[lug] On Testing (was: Freedom formatting (was: ubuntu wireless discovery tool))

[Rob Nagler]

Tom Christiansen writes:
> To: Rob Nagler <nagler@[snip]>
> Cc: Boulder Linux UG <lug at lug.boulder.co.us>

I let this go the first time, but it's a symptom of the disease.  Why
would I need two copies?  Gmail hides one of the two copies from me.
Yet, they still go out, and they clutter the Internet.  My MUAs
(both Gmail and Emacs VM) respect the reply-to, doesn't yours?

> Just in case, these messages all have as their owner password a very
> secret word which any old hacker out there will immediately
> guess. :)

I was just at a nice TG dinner with a bunch of families.  One of the
kids solves Rubik's cube in about 30 seconds.  He's competitive at the
state level.  I haven't ever solved Rubik's cube nor guessed passwords
nor solved pretty much any puzzle out there -- and therefore never be
hired by Google.  I could write a program to brute force it, but
wouldn't it just be easier to include it in the email?  The people who
want to misquote you, can, of course, just by writing whatever they
feel like.  Newspapers do this all the time. :)

I had to resort to OCRing and retyping again.

> I'm not sure courtesy is ever about hiding, per se.  I think it's
> about showing respect for one's fellow in a sens sensitive and
> compassionate way.

I thought you were speaking about MUA developers being discourteous by
writing software that didn't solve the rendering problem properly.  I
was writing to this explicitly, not the general need for courtesy in
society, which I agree with.

> Hm. Fear of failing? Fear of trying to compose a legible message but
> failing and so they don't even try? I'm afraid I don't follow. I do
> think the prime motivator is laziness of the bad sort coupled with a
> selfish disregard for others' time and trouble. 

Again, I was talking about the MUAs themselves.  I don't think users
should be responsible for formatting.  They should focus on the
content, which can't be automated.

Having lived in a foreign country for a long time, I found it was
practical to assume native speakers would want to figure out what I
was saying in my poor execution of their language.  The alternative
was to study the language(s) for several years before uttering a word
to native speakers.  That wasn't practical for me.

My experience with email is quite similar.  I delete a lot of mail I'm
uninterested in.  When I want to read something, I figure it behooves
me to figure out what the author is trying to say.  I don't think one
person can be expected to produce a document which all others can read
easily.  Even the concept of "legible message" is personal.  Many of
the people I communicate with have threaded MUAs.  Others, don't.
Some people hate top-posting.  Others hate bottom-posting.  Some Love
Reply-To, and others hate it.  I figure it's best to do get out there
with something, and hope for the best.  ("You miss 100% of the shots
you don't take.") If someone really doesn't want to understand me,
they'll just do so anyway.

Back to fear.  I think laziness is fear of failing.  Procrastination
is the same.  "Selfish disregard" is fear again.  Stripping away the
layers of the onion is often difficult, but I believe if you look hard
enough, you'll find that fear originates most anti-social behaviour.

> But that will just get me talked about, and I would very much rather
> not do that.

Anybody posting to this list wants to be heard publicly.  As such, the
posts and the authors are being talked about.  We are all seeking
status.  Unfortunately, you can't seek status and not be talked about.

> Hm. Maybe. That seems too facile to account for *all* the crappy
> software.  I'm not saying you're wrong. I'm just saying there are
> other reasons in addition to yours. And I'm not ranking them; I'm
> not smart enough for that.

Perhaps I said that too strongly.  I think lack of testing is the main
reason we have so much crappy software.  It starts with the dearth of
testing courses in school.  Tests change the nature of software
development from "thought experiment" to "executable fact".  This is a
significant shift in focus.  We at bivio spend a lot of time
discussing the most important cases to test, and not so much time
talking about how to satisfy said test cases.

Today, it's possible to create reams of code.  We have extremely
powerful tools to help people (programmers and non-programmers) do
amazing things with computers.  I wouldn't take this away, just like I
wouldn't go back to a strongly-typed language.  At the same time, I
think these people get stuck with piles of code after a while, and it
becomes their raison d'etre.  We've all seen it.  The guy who is in
charge of Module XYZ or The Spreadsheet, because he's the only one who
knows what it does.  The problem is that the guy might actually be
pretty smart, and could be doing productive work, but he's stuck
maintaining Module XYZ for the rest of his time at the organization.
When he leaves, oh well.  Time to rewrite it, and the cycle continues.

> It specifically lacks any failure tests, which are absolutely
> critical.

FYI, "failure tests" are known as "deviance tests" in the testing
literature.

I'm not sure any test is "absolutely critical".  That's the hardest
part of testing: hypothesis creation.  The reason programming is easy
is that you are given the hypotheses, and all you have to do is deduce
the code that satisfies them.  Testing is exactly the opposite.  Test
creation is inductive reasoning (or more likely abductive reasoning --
thanks Scott!).  It's very difficult to figure out what you need to
test, and even when you do this, you need to figure out the
experimental apparatus you need to create to test the hypothesis.
And, of course, debug the apparatus to make sure it is not faulty.

> Text::Autoformat [...] dumb buggy web-based email interfaces [...]
> really ought to be using Damian's algorithm.

I believe this problem is hard, and possibly NP-hard.  I don't think
anybody can say they have the best solution.  There are tradeoffs, and
I attached an example of how autoformatters can go wrong (which comes
from the Text::Autoformat page, btw).

> An Empirical Study of the Reliability of UNIX Utilities [...]
> fuzzy testing.

I know of Bart Miller's work.  I think fuzzy testing illustrates the
problem but it isn't a solution.  Unix utilities solve simple
problems, often with well-defined solutions.  The bugs that fuzzy
testing detect would not exist if the libraries used by the utilities
were more sophisticated and, of course, unit-tested.

> Languages like Perl or Python or Ruby do not compare with the
> guarantees provided by C++ or Java.

While your point was about availability vs security, I think this
statement supports a myth.  Dynamic languages are as reliable, if not
more so, than statically-typed languages.  The "guarantees" static
typing provides are trivial.  The static-typing adherents rely on a
false foundation.  The problem is not whether the machine will execute
the code correctly, it's whether the algorithms match user
expectations.  Again, the types of problems static typing solves would
be eliminated by high-level, unit-tested libraries.

> Hope you're all enjoying a warm and pleasant Thanksgiving with your
> families.

Ditto. :)

Rob

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 20091126-autoformat-err.png
Type: image/png
Size: 52569 bytes
Desc: not available
URL: <http://lists.lug.boulder.co.us/pipermail/lug/attachments/20091126/41a692b6/attachment.png>