[lug] Daily system crash....

[Nate Duehr]

On Jan 28, 2008, at 11:54 PM, David Morris wrote:

> Those programmers (who have no concept of what engineering
> is) are all missing one basic fact:  It takes several orders
> of magnitude more effort to fix a problem after it exists,
> than to prevent it from occurring in the first place.

Ohh, as a career "support" guy, I wish I could get that through just  
10% more of the engineers heads I've worked with over the years... any  
more than that, and I'd probably be out of a job, but just a FEW more  
would be nice... heh heh.

> The problem, of course, is that corporations have no
> incentive to create bug-free software that does everything
> the users want the first time around because consumers won't
> accept the required sales cost.

Hell, they actually have an incentive against EVER getting it right...  
in order to have sales NEXT year, you need the next version to  
release!  Anything missed in this year's version will be a fancy new  
"feature" in next year's.  Then about 5 years in, when the cruft gets  
too much to manage and the engineers are bored and/or have moved on to  
other things, it'll be time to start all over on "Project X" that does  
the EXACT same things, and makes the same design and implementation  
errors the original did, hopefully minus a handful.

LOL... no, I'm not cynical about software...


> Which leads, of course, to the fact that consumers are, as a
> class, total morons.  People are willing to pay thousands of
> dollars on the buy-support-upgrade-replace cycle if it is
> spread across a span of years, yet refuse to shell out a
> mere few hundred dollars on a product that works right the
> first time around.


Sorry have to say it... Apple Macintosh.

Okay, now I'm REALLY going to get stuff thrown at me... hahaha...


> /end rant

Good rant!  :-)

Heh heh... I don't want to come off as jaded, but it has been serving  
me well for years... being a bit snarky in your THINKING (maybe not  
out loud at work, depending on who's in earshot) about what crap is  
going to come flying over the wall from engineering next, serves very  
well to plan for it... get out the rain slicker, load up some test  
systems, hunt down some of their more common bugs/screwups and make  
sure whether or not they made them again, file a few bugs that'll go  
nowhere for a couple of revs because they're TIRED of fixing the same  
mistakes over and over... and prepare for how to present it all to the  
customer as something "new and wonderful... you'll like it... it has  
some of the same problems the last one did, but now it runs on a  
clustered system and has some auto-update features and..."

And away you go into another 5 year cycle... even more wild, SOMETIMES  
new engineering managers really DO think the new thing they were  
assigned to oversee is really new.

You'd think they'd know... every company that ever owned a computer  
has written code to create things like billing systems, customer  
tracking, and to do whatever features their product does, since the  
founding of their company -- or they wouldn't be in business.  A new  
pile of spaghetti code that does all of that, is just the next one in  
a long succession of systems that all did that... the hardware gets  
faster/bigger, but the goals of most software projects don't.  Even  
better, the engineers are very often practicing "code re-use"  
anyway... so the same bugs just come along for the ride!

I'm not saying this is "bad", just the way it is... you probably know  
and agree.  Anyone who hasn't worked in Dilbert-land of large corp  
America... on software products or product support/tech support.

And it's actually fun in a way... if you let it be.  The above makes  
it sound like it isn't.  It is.  For the really politically- 
incorrect... the support staff can often place bets (either real or  
just for intangibles) on EXACTLY the bugs any particular engineer will  
do again and again... "I bet you that DB code was written by.... X",  
once you see one engineer's common bugs, you'll see 'em again and  
again... especially if your organization is open enough that support  
staff can look at source code.  (Of course, junior support staff is  
usually still just climbing the slope of the learning curve and has no  
time for such things, but senior staff is often asked to "dig deeper"  
before sending in bug reports to engineers, and often that exercise  
will find the same name in the comments for the same type/style of  
bug, year in and year out... and their own managers and QA staff  
continually miss it.)

(Our perennial problem is that our engineers seem to think data  
networks are always guaranteed to be there and to be working... error  
handling of dead IP networks is ALWAYS the first thing I mess with  
whenever new code comes out, because those first few revisions ALWAYS  
suck at handling sudden losses of network.  Yanking the Ethernet  
cables out of new systems is priority #1 when I'm asked to test or  
load test something.  Networks ALWAYS fail in the real world.  And  
real-time telco systems ALWAYS hate that... they're like arch-rivals,  
and always have been!  Network failover schemes are a second-best fun  
thing to try to break... plug in the network cable, unplug it, plug it  
in, unplug it... wow, the failover mechanism is completely confused  
and everything's down, but both networks are up and live after me  
messing with the cables 20 times... why hasn't it recovered???  Heh  
heh... evil.  Pure evil.  I would have loved to work in QA, and break  
stuff... but they don't pay as well for behind the scenes jobs as they  
do for ones where you have to talk to customers!)

--
Nate Duehr
nate at natetech.com