[lug] Running a mixed Python environment

[Davide Del Vento]

Because of `python -c 'import this' | head -4 | tail -1 # until it works`
let me say that I agree with you that backward compatibility is highly
desirable and should be a design goal. Where we disagree is that you are
making it sound much easier than it is, IMHO.

> It's painful, but why should they do what you and I, the unpaying
> customers, are asking?
> I'd be happy to pay maintainers to not break interfaces. I don't think
> money is the issue.
>

Have you offered anybody?


> It's more about expectations and assumptions.
>

More on this later.


> > But I've lived on the other side (even paid), and I can tell you, it's
> not pretty and I think for most project is not going to happen. The example
> you provide of the opposite I think are (and will remain) the rare
> exceptions.
>
> The list is quite long of compatible interfaces. You just don't think
> about them, because well, they are compatible. Consider that 'ls -l' has
> worked for decades. They could have changed it to do what 'ls -lh' does,
> but they didn't. Many scripts depend on that format and that the output is
> in bytes, not human readable GBs, TBs, etc. Maddenlingly, 'rm -r' crosses
> file systems, when 'rm -r --one-file-system' should be the default. It's
> fine that they didn't fix this. I could go on about this, but a tremendous
> amount of software never ever breaks APIs.
>

Back to "you are making it sound much easier than it is". You cannot
compare simple standalone utilities for which performance and hardware are
irrelevant (or close to) and no change is needed other than a rare bugfix,
to that huge piece of tangled libraries that is tensorflow, its
dependencies and the mess of hw circuits it runs on and the pressure to
achieve faster and better results or being left in the dust. Perl or the
Linux kernel, fair enough (but how about Perl6?)


> The big difference between Guido and Larry was that Guido was a professor
> and Larry was a sysadmin. Perl has a culture of backwards compatibility.
> Python has a culture of breakage. The famous line from Ken Thompson "I'd
> spell creat with an e" is an example of the culture of backwards
> compatibility. Unix has a terrible naming culture, but it is an admirable
> culture of backwards compatibility. This statement sums up both.
>

That I agree. Some communities (and companies) miss that culture, like
Yegge's article said.


> You can make your own software backwards compatible. It's not hard. I've
> been doing it for a few decades now. :)
>

I love to hear that, but it's not been my experience. Yegge also said (like
I do) that it's not a trivial task. It's hard and a pain. So you have to
have very strong motivations to do it. Perhaps it can be easy in some
specific domains like yours, or you are quite good at making the best (or
at least not too bad) choices at the beginning, or you have factored tha
pain and cost as a normal part of the business. But it's not always like
that, in my experience. In anything major, people make mistakes and release
them and fixing them requires breaking backward compatibility and/or
creating a horrible mess of APIs doing the same thing in multiple ways (my
irritation with Perl) some good, some bad and some ok -- with confusing API
names.


> > Case example: two applications utilizing the same underlying
> communication library -- one application wanting bit-for-bit
> reproducibility first, the other wanting highest performance first. How to
> update the library with these two competing needs? Major-minor numbers
> allow that, chronological versioning doesn't. As you know this is NCAR's
> bread'n butter
>
> I would say you have a configuration parameter that sets the speed, just
> like in a Tesla.
>
[....]
>
Back in the day, I worked extensively on operating systems and networking
> code. The one rule was "don't break it". Linus understands this, because he
> read code from people like Andy Tannebaum, who understood this. I am
> confident that one can design
>
communication libraries today that don't break compatibility while
> providing for different users' needs.
>

I think I picked the wrong example and we are now going into the nitty
gritty details of software, hardware, drivers, and how these things
interact together. Not interesting for the purpose of this conversation
IMHO.


> NCAR has a large FORTRAN culture, which is renowned for its backward
> compatibility. Why has this not permeated the Python or communications
> library world at NCAR?
>

The communication libraries I am talking about are hardware, firmware,
drivers and a little bit of software on top of that, which vendors
implement, not NCAR. We are mere users.

The Python world is not specific to NCAR. That community has not been
permeated by backward compatibility culture from the start for the reason
you said (Guido), and it did not embrace it later because it's hard and not
fun. Compare "Look, ma, I wrote code that make me fly: `python -c 'import
antigravity'`" to "Ma, I fixed a bug without breaking backward
compatibility". Yes, I am as horrified as you and Yegge by Guido's response
to Yegge's question, and that's the culture part. But compare Perl's
community to Python's: the first is "boring professionals doing work" the
second is "cool kids having fun" -- or at least that is what I see. Now the
culture part could mitigate the problem (not solve it), and in the past I
was really angry at seeing the lack of that culture. There are many
offenders there, both in the Open Source community (the one I disliked most
were GNOME and KDE) and in private companies (Google among the first). Even
worse, it seems to me that there is a culture of
change-for-the-sake-of-change to make things "look nicer", especially with
GUIs. You might say GUIs are not APIs, but I disagree. I really hated such
changes. But now I am resigned.... There are two of us who are responsible
of python and python libraries installs at NCAR (for my sanity the other
guy does most of the work). We just keep banging our heads on the keyboards
each time a new version of Tensorflow is released and a user asks for
it.... and we are resigned, as I said...

Cheers,
Davide
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lug.boulder.co.us/pipermail/lug/attachments/20200915/e4e327c5/attachment.html>