« Our dichotomy opens the combat | Main | Smalltalk by other means »

Django debugging, and MVCP frameworks

If you happen to be the only other person writing a django app without using the free (as in labour) RDBMS mapping stuff (yup, it's RDF + XML), here's a gotcha that might interest you.

If you see a stacktrace that ends like this:

    ...
    File "E:\lamp\python\Python24\lib\site-packages\django\core\meta.py", 
      line 303, in get_all_related_objects
        for klass in mod._MODELS:

  AttributeError: 'module' object has no attribute '_MODELS'

You're in for a treat. What the means is that a model file in one of your applications looks like this.

   from django.core import meta

   # Create your models here.

because you haven't put anything in it yet. When I came across this I was calling the top page of an app named 'units':

  http://localhost:8000/units/

Previously that had been working fine. I had added an app called contacts recently:

  http://localhost:8000/contacts/

and that was working fine - it was really just a placeholder returning a raw string while I was testing the URL dispatching. Here's the view stub for contacts.py:

  from django.utils.httpwrappers import HttpResponse

  def index(request):
      return HttpResponse("true")

But units/ was failing now. I hadn't changed the units view (honest!):

  from django.core import template_loader
  from django.core.extensions import DjangoContext
  from django.utils.httpwrappers import HttpResponse

  from mobius.apps.units.models import units

  def index(request):
      unit_list = units.get_summary_list()
      unit_list.sort()
      t = template_loader.get_template('units/index')
      c = DjangoContext(request, {
          'unit_list': unit_list,
      })
      return HttpResponse(t.render(c))

    ...

Nothing much fancy there. Did I say I hadn't changed it?

So I run off and check the main.py configuration, the URL mappings (the main pain source to date), and that the test runnner webserver was configured properly with envars and pythonpath. All fine. Take the contacts app away - units works. Put the contacts app back - contacts works, units fails. No joy: I'm left with a sort of Heisenbug - a new working app is breaking a previously working app.

Time to walk the execution stack in Django.

Here's what happens. When the /contacts/ URL is called, Django will look up its regex mappings and dispatch to the view that meets the pattern. Core will invoke the view and it will run just fine - bear in mind that the contact's model is never loaded. When the /units/ URL is called, Django will look up its regex mappings and dispatch to the view that meets the pattern as before unless that view is also calling DjangoContext, whereupon we go down the rabbithole.

Using DjangoContext results in a new execution path that causes each of your app's models to be introspected and reflected over using meta.py. What Django is doing is looking for all the related models that might be in scope for this request, and if so those models will be loaded, and if needed, out of the RDBMS.

Now, it's important to note that in this latter case, all the available apps are in scope to be reflected over, not just the one you called via the URL. Hence, if any of the model files has no content, this results in Django's introspector crashing out of iteration and sending the message "AttributeError: 'module' object has no attribute '_MODELS'".

Here's what I did to fix the problem:

  from django.core import meta

  # Create your models here.

  class Contact(meta.Model):
      fields = (
          meta.CharField('name', maxlength=256),
          meta.DateTimeField('email', maxlength=256),
      )
      def __repr__(self):
          return "contact"

All that Contact class does is stop Django's introspector crashing when units is invoked. The units/ view will work fine now.

So here's the deal: yes you can compose and plug apps in Django, and it's very nice, but an app with no attributes in its model can cause breakage in working apps that require a model check.

I'm still debating whether this is a real bug and is worth filing. I guess it can be dealt with by having Django's application code generator put some kind of stub attribute into the model. On the upside I got to learn a lot about Django internals.

Django nonetheless, is looking good. It has first rate support for URL design, intuitive URL/function mapping, a decent template language with pluggable templating an option, composable webapps (aka portlets, aka plugins), and on the whole is a productive environment. And it's not a 1.0 yet. I have some quibbles:

i18n: i18n isn't there yet (it's coming, I hear). While the java frameworks do lack on the productivity side, they generally do the right thing by i18n.

Layout: Django breaks your app into three folders: views/models/urls. If your app is called polls you get this layout polls/views/polls.py, polls/models/polls.py, polls/urls/polls.py. That's a lot of polls. Now, having three files called polls.py and trying to figure out which is which in the editor can get irritating. I like the idea of split directories in principle - in practice the less directories the better. I really doubt the Django people are going to do anything about this (they did build it after all, not me). But a whole module for a tuple of url regexes? My best guess it enables app composition and the name-munging needed for automation.

Community: I will be interested to see how the Django team deal with the influence of feature requests and bug reports. So far they are fixing them quickly and attributing in the commit comments, but at some point the Django team need to think about how the project could function and support a community without them.

RDBMS Models: as the issue I started with indicates, if you are not using a RDBMS to back your webapp's models you are a second class citizen in Django. Django has enough features and niceness to make living there ok, but you'll never be able to put it out of your mind. I imagine this could result in implementation by exhaustion: "oh screw it, slap it in the RDBMS, I is tired".

MVCP frameworks

This issue of the RDBMS is terribly conflicting, if you believe to some extent as I do, that RDBMSes are not neccessarily your first choice place to put long-lived data, or at least not the only choice. Ruby on Rails and Trails also present this dilemma. I really recommend the Prag's Ruby on Rails book for anyone building any kind of webapp, but it's interesting to see them promote a framework that defaults to using a RDBMS. If you know your agile history, you'll know that starting with a RDBMS is considered questionable. Something's afoot - or not. It's tempting to conclude that for all the talk about web MVC, you can't easily automate to the level of the next generation web frameworks without putting your Persistence mechanism up there with the Model. It seems to reach the levels of productivity implied by Rails, Trails and Django, we're talking about MVCP frameworks, where P stands for Persistence. It's for each to decide whether that's a good tradeoff. Open data is king, and putting data into a silo to gain a quick productivity win early on is dubious. It's not clear how good the support in any of these frameworks is for alternate persistence mechanisms.


August 1, 2005 12:54 PM

Comments

Ian Bicking
(August 1, 2005 11:39 PM #)

Well, an RDBMS is in practice one of the most tranparent data silos you'll find. It's generally more transparent than XML or other flat files, as it is self-describing and constrained. Of course if you do crazy things with it it's not so hot, but all of these frameworks are careful about sensible mappings.

As far as the bug, I'd certainly say you should report it. You were doing something innocent, and it produced a confusing error; for a *library* that's a real and important bug. It's equivalent to a UI bug in a GUI application.

Robert Leftwich
(August 2, 2005 12:46 AM #)

I'm curious - what are you using for the RDF+XML backend?

As an aside, I've just put together a small web app using Aquarium as the MVC and 4Suite/Amara as the RDF+XML backend as a proof of concept. Sweet! The ability to generate RDF on any XML doc combined with the Versa query language and Amara's intuitive Python/XML binding is very powerful.

Bill de hOra
(August 2, 2005 01:32 PM #)

"I'm curious - what are you using for the RDF+XML backend?"

Currently rdflib+sparta. I'm noodling on 4Suite and redland tho'.

Bill de hOra
(August 2, 2005 02:12 PM #)

"Well, an RDBMS is in practice one of the most tranparent data silos you'll find. It's generally more transparent than XML or other flat files, as it is self-describing and constrained."

I disagree; in the context I'm thinking about, every RDBMS has a private interface and wire protocol and that has to be ported to every language you might to use to access it. When I think silos, I think RDBMSes. How you talk to an RDBMS is specific to that RDBMS.

Adrian Holovaty
(August 3, 2005 06:46 AM #)

Hey, thanks for checking out Django! Sweet.

We are *very much* interested in providing XML backends as an alternative to database backends. That sort of flexibility would be outstanding, and there's certainly an audience for it. Please bring this up on the django-developers list -- http://groups-beta.google.com/group/django-developers -- if you have a second. Anything would help, from broad implementation ideas to concrete patches.

Lemme address some of your issues --

* I18N -- We're very committed to getting this done. Problem is, we don't know the best way to do it; this is very new to us. People in the community have made suggestions here and there, but nobody feels comfortable taking on the *whole* problem. If you could contribute to that effort, *awesome*. Check out our developers list, whose link I've pasted above.

* LAYOUT -- Yeah, I'm not too crazy about the default module layout myself, and I'm the one who arranged it. We're very open to suggestions on this. Most of the layout doesn't really matter as far as Django is concerned, so we have a fair amount of flexibility in changing it.

* COMMUNITY -- The IRC channel is very-much self-sustaining at this point, in terms of people helping other people. Jacob (co-developer) and I might be open to giving people commit access, if that's what you're implying, but we're still getting our toes wet, getting comfortable.

* THE BUG YOU POINTED OUT -- I agree that that's a bug. Thanks for bringing it up. There are two ways to fix it: Change the default code generator to create "stub" models, and/or change the model parser to allow empty models. Both changes sound good to me, and I'll get to them ASAP.

Adrian Holovaty
(August 4, 2005 04:33 AM #)

I've fixed the bug you brought up. Thanks again for reporting it!

Post a comment

(you may use HTML tags for style)




Remember Me?

Trackback Pings

TrackBack URL for this entry:
http://www.dehora.net/mt/mt-tb.cgi/1615