No free lunch for programming libraries

Robert Fischer has a bad day with REXML: "In this language core library, both versions are broke! The solution is for me to reach in and make a change to the core library so that we avoid a null. In the standard Ruby deployment, using the standard core XML processing library, there is no way to write out XML. It is impossible because of bugs in the library.

The worst part?

THAT STUPID BUG IN THEIR CORE LIBRARY WOULD HAVE BEEN FIXED WITH STATIC TYPING. Even more if you have a type system which can check nulls for you."

I think it would be unfair to paint Ruby with REXML's brush. Even if static typing would fix some problems, I can't imagine Rails built on a language with a static type system. As we'll see in a minute you can write a good XML library without static typing. According to DHH "I think Rails feels, smells, and tastes like it does exactly because its very Ruby-like". Now the dilemma:

"I’ve cut Ruby on Rails a lot of slack, because the bugs and the awkwardness are the price that you pay to hang out with the beer-swilling hipsters that make out that community and roll with the rapid rate of development. And, really, it is the best web framework I’ve dealt with, mainly because it provides the most comprehensive and extensible set of functionality of any framework, and it’s got a solid community to back it up."

Fwiw, this is the reason I default to Django - similar to  Robert's experience with Rails, I find it provides a comprehensive and extensible set of functionality, and it’s got a solid community to back it up. Plus it's fun. Django is written in Python and Python's libraries tend to be well baked. In particular there are ones outside distro that rock. If there's a better XML lib than Elementtree, I don't know what it is. If there's a better httplib than httplib2, I don't know what it is. PIL. Feedparser. BeautifulSoup. Numeric. And so on. I guess I'm lucky in my choices.

It can take a few attempts to get the design of libraries right - going back to XML for a second, look at how long it took to get to XOM and XStream in Java (neither of which are in the JDK). Embedding libraries with immature designs, usually due to reaction to industry trends, has been a problem for the JDK. Once a lib ships in that core it's very hard to root it out. But when you look at search and megadata problems, no community has anything comparable to mg4j/lucene/hadoop. I remember a time when Python's XML support was up in the air and Unicode was "up the BOM". Python still has problems with installation, whereas Ruby has gems

That said, library problems are transient problems. They can be fixed.

Tags:

5 Comments


    As for better libs than ElementTree... I prefer lxml, because of the support for validation, relaxng, full xpath and some ET compatibility. See http://codespeak.net/lxml/


    > If there's a better XML lib than Elementtree (...)

    Not everyone's cup of tea. I prefer amara.

    http://uche.ogbuji.net/tech/4suite/am...


    Re: "THAT STUPID BUG IN THEIR CORE LIBRARY WOULD HAVE BEEN FIXED WITH STATIC TYPING."

    In this particular example he's wrong not about static typing, but wrong because bugs in libraries aren't fixed unless someone is fixing them, and from what I understand of REXML its biggest problem is that no one is fixing the bugs.

    Is this something about XML libraries? Python has gone through the same pain with PyXML (most of what makes up the xml package in the Python stdlib).


    Ian: "Is this something about XML libraries?"

    I'd very much think so. Building an XML library seems trivial at the beginning (hey, it's just some angle brackets!), but gets surprisingly difficult once your past the initial effort. Producing a good tree model for XML is also non-trivial.

    I'd guess that in many cases people start with something easy that somehow works, but find themselves in quite a mess of non-fitting pieces soon. And then there's the question of who's really willing to put in all the effort and diligence it takes to fix stuff up.

    And this is not limited to free software - the amount of commercial software packages that get XML wrong in one way or another is astonishing.


    "I think it would be unfair to paint Ruby with REXML's brush."

    As I said in my post -- if it wasn't in their core library, I'd be willing to cut it some slack. But the fact that it's in their core library puts a heavy expectation of quality: a library goes into the core because it is among the best available as a general solution. And among the best in this language is still can't-keep-compile-bugs-outs bad.

    "Even if static typing would fix some problems, I can't imagine Rails built on a language with a static type system."

    I'm not so sure about that. What feature of Rails is so difficult to conceive of in a static context? Please keep in mind that static context != Java.

    If you've got a particular feature you're interested in, drop it as a comment here so that I look into it later:
    http://enfranchisedmind.com/blog/2008...