[lug] Running a mixed Python environment

[Rob Nagler]

On Tue, Sep 15, 2020 at 7:28 PM Davide Del Vento wrote:
>> I'd be happy to pay maintainers to not break interfaces. I don't think
money is the issue.

> Have you offered anybody?

I have paid quite a bit for free software including contributing
non-breaking bug fixes. You can offer people a free fix, and they'll reject
it so it's more about their opinion than about the money, I believe.

> Back to "you are making it sound much easier than it is". You cannot
compare simple standalone utilities for which performance and hardware are
irrelevant (or close to) and no change is needed other than a rare bugfix,
to that huge piece of tangled libraries that is tensorflow, its
dependencies and the mess of hw circuits it runs on and the pressure to
achieve faster and better results or being left in the dust. Perl or the
Linux kernel, fair enough (but how about Perl6?)

You bring up Linux, which is way more complex from any number of dimensions
and almost 30 years old. The fact that Tensorflow is such a mess makes no
sense to me.

I put Perl6 in the Py3 camp. Perl5 will be around forever. It's open
source, and there's Docker.

> So you have to have very strong motivations to do it.

I work with a very small group of programmers on many, very complex
systems. We cannot throw money or time at any of the problems. We have to
make things backwards compatible.

> Perhaps it can be easy in some specific domains like yours, or you are
quite good at making the best (or at least not too bad) choices at the
beginning, or you have factored tha pain and cost as a normal part of the
business.

One of our systems integrates scientific codes into a GUI. These codes are
difficult to use, which is why we put GUIs on them. We are not
computational scientists, and we do not understand the codes, the
algorithms, or the underlying physics. We have to import and export native
data files. When we discover a bug in our interpretation, we have fixups
for old data so we can import old canonicalized (non-native) files that
have been exported or are sitting on disk in our database. This is work,
but it is not much work. The hard part is getting the codes.

Recently, in another system, we broke backwards compatibility in our tax
software. One of our customers tried to generate a tax return for an
investment club from 1999. The problem was that the PDF generator we use
could not read the original PDF (FDF) tax forms that the government had
provided. We removed the ability to generate 1999 tax returns. However,
people can still generate 2000-2019 tax returns.

So yes, we do break backward compatibility, very judiciously.

> In anything major, people make mistakes and release them and fixing them
requires breaking backward compatibility and/or creating a horrible mess of
APIs doing the same thing in multiple ways (my irritation with Perl) some
good, some bad and some ok -- with confusing API names.

Here are some common mistakes: naming "creat" and strcpy. You live with
"creat". Nobody has died from it not being named "create". strcpy style API
bugs on the other hand probably has killed people so you introduce strncpy
and  advertise it and maybe output warnings.

At Tandem they had a language called TAL, which allowed you to add any
number of parameters to a function, but it didn't have named parameters so
you'd have function calls with 15 or 20 commas in a row to ignore
positional arguments before you got to the one you needed. It was horrible,
but Tandem was backward compatible, and that kept the customers coming
back. There are still Nonstop systems running, I'm pretty sure.

> We just keep banging our heads on the keyboards each time a new version
of Tensorflow is released and a user asks for it.... and we are resigned,
as I said...

This is true, and agreed.

The part where we disagree: how much time does it take to maintain
backwards compatibility. My argument is that almost all of it is vanity,
e.g. the OrderedDict to dict changes in Python 3.8. Leave it alone. Nobody
will die if they continue to get an OrderedDict back from an API.

It's a lot like those signs in neighborhoods: Drive 25, it won't kill ya.
People just think their immediate problem (getting from point A to point B)
is so important that they get tunnel vision, and ignore things that are
often more important, like a child's life. Backwards compatibility is
similar: I'm going to remove this function, because it bugs me to have to
update it and this other function. I just broke backwards compatibility,
and probably ruined the days of many other people. It's not a child's life,
but it's the same effect: tunnel vision.

Rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lug.boulder.co.us/pipermail/lug/attachments/20200915/86633d9b/attachment.html>