« QOTD | Main | Subversion tips: working with branches »

I think I figured out the list comprehensions thing...

Warning - this one is about programming language esoterica.

For the life of me I've never been able to understand why anyone would pefer:

  print [x*x for x in range(1, 10)]


  print map(lambda x:x*x, range(1,10))

and so much so, that if lambda, map() and their ilk ever got dumped from Python, I'd probably be looking at Ruby or Lisp as my upgrade path from Python 2.4. Which is to say, it would *suck* if the day Python turns mainstream is the same day I stop using it outside work. So I turned it around - what is it I like about lambda, map(), et al, beyond some questionable handwaving about functional programming? At least that way, I don't have to go around pestering people about language trivia.

I asked myself, is lambda a cargo cult?


I think I have my preference figured out - it's the "for x in y" bit within the listcomp. That exposes an implementation detail - namely the mechanics of processing a list. I'd much prefer to arrange the function, the sequence and the data structure in concert and let the computer have it, instead of telling the computer how to work the list. Seeing a "for" inside my list comp is like seeing ".cfm?entry=12" in my URL. Yup, it's pretty thin justification. At one level, it *just doesn't matter*. Listcomps are seriously useful. At another one, it matters a lot - how do I || a listcomp? That said, it's nice to know for 99% of the work I happen to do, using list comprehensions over anonyomous functions won't matter. Which is to say, I'm truly being a fusspot about this.

if you are scratching your head now going wtf, that's not your fault, it's mine - this is all quite esoteric, like discussing the relative earthiness of red wines, without the side effect of drinking the stuff.

February 12, 2006 12:59 AM


Ian Bicking
(February 12, 2006 02:05 AM #)

It's not a hard coding in a list to the process, it's hard coded a sequence of items, you can even think of it as a "stream" (especially with generator expressions, though list comprehensions have always consumed a stream even if they do not produce one).

Certainly there's still operations that can't be expressed in list comprehensions or generator expressions. itertools provides a lot of these. Most of them do not require functions, though a couple do that cannot be expressed in terms of comprehensions. Specifically: dropwhile/takewhile and groupby (I'm not sure about starmap, but if I understand it, it's actually *much* easier and clearer to express in a comprehension).

So I still think you are mostly attached emotionally to lambda, not for any real quality of it. Unless you come up with a real use case -- that will help prove your case, and actually provide a useful point of discussion.

Miles Sabin
(February 12, 2006 02:08 AM #)

Well, if you did this in Haskell it'd look like,

show [ x*x | x <- [1..10]]

not a whiff of an imperative-seeming 'for' and quite obviously a direct transcription of what you'd write if you were doing a bit of set theory,

{ x*x | x in { 1..10 } }

I think you're just getting hung up on syntax.

Bill de hOra
(February 12, 2006 01:43 PM #)

Miles, I think you're right :)

Bill de hOra
(February 12, 2006 01:57 PM #)

Ian, what I'm doing here is working things through ("how come I like lamda more?"), not building a case ("listcomps objectively suck"). I do like lambda, I know I can parallelize a map() function, I think listcomp syntax is the oddest thing in Python (after scoping). All IMO. As far as I know listcomps are more expressive that Python lambdas but I don't find them more beautiful. Unless they're written in Haskell perhaps. That's some nice code.

Aristotle Pagaltzis
(February 12, 2006 05:05 PM #)

How do you express chained maps as list comprehensions?

Ian Bicking
(February 12, 2006 08:49 PM #)

A use case isn't something complicated -- it's just a reasonably realistic example, implemented in both forms, where you argue that the map/lambda form is better for some reason. Maybe just "real example" is a better term.

Lately when people have suggested new syntaxes or basic functions for Python, some people have looked into the Python standard library (as an example of a bunch of code written by a wide variety of people) and found places where the new stuff is applicable, then reimplmented that portion. Then you compare. This is a lot easier to think about -- in all ways -- than an abstract case.

Steve Downey
(February 13, 2006 06:37 PM #)

List comprehension and map are connected by definition. Using Haskell syntax

    [t | x <- u]

is defined to be
    map (\x -> t) u

So, what you're smelling is the syntactic sugar between the two forms.
For comprehensions with more than one qualifier, the definition is a little more complicated:
    [t | p, q]

    join [ [t | q] | p ]

where join for lists is just concatenating them together.

However, it has been observed that using list comprehensions is often the result of thinking about a problem imperitively, assiging results to variables and looping over them, rather than thinking in terms of the transformations involved. So that 'for' may be bothering you for a reason.