[lug] software engineering

[Nate Duehr]

David Morris wrote:
> On Mon, Nov 13, 2006 at 05:32:43PM -0700, Bear Giles wrote:
>> The levels where it starts getting difficult?... they're
>> management, not engineering. Don't get me wrong, it's
>> usually still people on the engineering side of the house,
>> but when you're trying to make sure 14 teams are all
>> working on the same problem that's a management issue, not
>> something you solve with your grandfather's slide rule.
> 
> At the risk of going back to prehistory I'm so far behind in
> reading my email........
> 
> The concept you're striving for here is Systems Engineering,
> a field newer than even software engineering.  At least as
> an official engineering discipline, people have *done* it
> for quite some time with varying degrees of success.

You think it's really that new?  Or was the role filled (even if they 
didn't have the title) in traditional "Engineering" organizations by 
Principal and Head Engineers with a lot of passion for the end-result?

I've never had the joy of working for an organization that ever had this 
as an official role in software engineering, but I *have* seen the role 
itself filled under other names in older, more discipline-focused sorts 
of engineering.  An example would be my own "world" of Telecommunications.

Most telecom operations have people labeled "Subject Matter Experts" who 
have varying degrees of authority and power within the organization to 
recommend and suggest (even approve) various forms of work done to 
operational/production systems.  They are the first to go to training 
and or even work with the engineers directly during development to learn 
the system and recommend changes when they're in that "development" 
cycle, and they're the first and last say on a lot of things in the 
"production" environment later on.

Who fills this role in software engineering in larger organizations, is 
what I'm wondering -- someone who's overseen the entire life-cycle of a 
product?  I don't see many people (none, really) who are tasked with 
this type of job.

> In short, the traditional engineer is focused primarily on
> how something should be done, while a systems engineer
> focuses more on the question of why.
> 
> In the field of Aerospace (where I work), this is a vital
> role because of the need to get everything right the first
> time in an extremely complex system.  Aerospace companies
> are still working out how systems engineering works into the
> mix and just finding out that its a good role to have around
> on small scale projects as well as large.  I've worked on
> projects with good systems engineering support and others
> with next to none and the difference it makes is
> astronomical (sometimes literally, in this industry! ;)

Who made those decisions before they had an officially titled "Systems 
Engineer"?  I'm thinking specifically about Aerospace here, since you 
mention it's your field... I've read a lot of things about Kelly 
Johnson's Lockheed Skunk-works, for example.  Was it handled by someone 
like Kelly himself back then because his "name was on the door"?

I ask because it doesn't seem like a whole lot of projects today want a 
single "name above the door".  It's so much easier to spread the blame 
for crappy software around to "the team", it seems.

No matter how hard an individual team member tries to ask those "tough" 
questions, when it comes time to ship, they're just a member of "the 
team" and they're rocking the boat.   Sometimes the smart folks just 
keep their heads down and their powder dry for another day's battles 
when deadlines loom -- and bad things ship.

I understand Aerospace has some goals that must be met just to make the 
mission happen -- just keeping the pilot alive in an SR-71 forced a lot 
of "obvious" Engineering questions and design realities, I'm sure.

In software, we don't seem to have our act together yet -- a 
"successful" software project typically only means the thing did some 
percentage of the desired result and manages to stay running to say, 
six-nines... that magical number that got a lot of marketing traction in 
the last few years.  Or it can mean something completely different.

Do you think it's possible to state in black and white a list of 
requirements that EVERY "successful" piece of software has to have?  I 
don't, and I think that's one of the dilemmas of software engineers. 
What's "successful"?  If the software does everything that was required 
of it, but is a nightmare to maintain for the operations people -- is 
that successful?  Is it successful if a customer later complains that 
it's too hard to maintain?  Is the answer ALWAYS, "the next version is 
coming, welcome to our software treadmill?"

> I'm not certain how much systems engineering (as an official
> position on the project) has been adopted in the commercial
> software industry, but I haven't heard much about it.  I
> imagine that as time goes on we'll see an increase in this
> role and a direct increase in software quality along with
> it.

You haven't heard much about it, and I've certainly never seen it.  I've 
seen cross-functional teams produce some pretty nice products that took 
a lot of things into account, if they're given the resources and 
authority to make whatever changes necessary to the product, and then 
have a director-level person who can communicate to Sales and Marketing 
about what the product is REALLY going to do, long before it's 
released... worked on a team like that once.  Was the representative for 
"support"... caught a lot of things... like, "How are we going to access 
this appliance remotely to handle service contracts?" ... and "I 
understand this six page document to reload the database, but if the 
system's down and you have to walk a customer through it, you're going 
to have one very confused/pissed off customer... could this be 
scripted?"  (And yeah, I ended up writing the script... in REXX.  That 
was odd.  Long story about how it came to be REXX, but it was an OS/2 
box...)

Thanks for the insights.  I keep thinking there's some way to slowly 
move software off the never-ending treadmill of buggy releases and have 
much better quality than we have today, overall -- but I think the 
problem here is the industry isn't set up for it.

I think customers tend to hate the whole, "That will be fixed in the 
next version (which will break three other things and make your life 
complex because you need to figure out how to take the downtime 
necessary to upgrade" syndrome, but I don't see any way to get software 
engineering as we know it today beyond that stage.

I keep meaning to see what (if any) good reading there is on the team 
that built MS Excel.  As much as our little Linux list might not like 
MS, you have to admit -- after about the second or third version of 
Excel, the thing generally "did" what its end-users wanted, relatively 
bug-free.  I can't think of any time I've ever heard anyone complain 
about Excel in terms of how "buggy" it is.  I'm sure something advanced 
that only a small percentage of users use in it has some bugs somewhere, 
but for the vast majority of the customer-base, Excel works "correctly" 
for them.  How'd they do that seems like a useful question.  Apache 
might be another good example, but I'm not sure... users don't directly 
interact with Apache itself... maybe a bad example.  Hard to compare 
desktop apps to server apps, I guess.

Nate