« Google numbers | Main | links for 2007-07-20 »

Design for the web

Exhibit 1: Wads and Wads about using ETags to reduce bandwith and workload with Spring and Hibernate. Too much to distill into a quote. But Gavin Terrill's article is a great read; he does things like making sure not to use any machine/physical context to calc the Etag, so it'll be consistent across a cluster of servers. Frankly, awareness of this sort of thing is lacking in the Java space. As Floyd Marinescu observed in the comments: "It would be cool to see a generalized etag caching framework added to some of today's modern Java webframeworks." Yes it would.

Exhibit 2: Django's support for ETags, which I can quote: "django.middleware.common.CommonMiddleware: Handles ETags based on the USE_ETAGS setting. If USE_ETAGS is set to True, Django will calculate an ETag for each request by MD5-hashing the page content, and it'll take care of sending Not Modified responses, if appropriate." That's it - you're done.

Exhibit 3: Rails support for Etag, again quotable in full: "Rendering will automatically insert the etag header on 200 OK responses. The etag is calculated using MD5 of the response body. If a request comes in that has a matching etag, the response will be changed to a 304 Not Modified and the response body will be set to an empty string."

The relative verbosity of programming languages isn't the interesting thing; nor is typing doctrine. What's interesting is the culture of frameworks and what different communities deem valuable. My sense of it is that on Java, too many web frameworks - think JSF, or Struts 1.x - consider the Web something you work around using software patterns. The goal is get off the web, and back into middleware. Whereas a framework like Django or Rails is purpose-built for the Web; integrating with the internal enterprise is a non-goal.

ETag support is just one example; there are so many things frameworks like Rails/Django do ranging from architectural patterns around state management, to URL design, to testing, to template dispatching, to result pagination, right down to table coloring that the cumulative effect on productivity is startling. I suspect designing for the Web instead of around it is at least as important as language choice.

It's hard to explain sometimes just how time-consuming it can be to get Web things done on some Java frameworks. This post will be a handy thing to point at next time I'm lost for words :)


July 20, 2007 06:18 AM

Comments

Eric Allam
(July 18, 2007 02:54 PM #)

Also note how many lines of code it took to implement ETags in Rails. If you look at the changeset, a considerble about more code went towards testing and documentation than actual implementation. The code on the Java side looks to be much more complicated. I haven't looked at the Django code to implement, but I'd imagine its way closer to rails than Java.

Gabe da Silveira
(July 18, 2007 03:58 PM #)

I think that's why I never got into Java, even though it was used for a good third of my computer science classes. I've been into the web for almost 15 years now, and it's always been about what cool stuff can the web do, rather than what cool software can I bring to the web.

You've really put your finger on something here about the philosophy of Java frameworks. The nice thing with Java is one language can do almost anything. That's great if you don't ever want to leave the Java world. But it does overcomplicate a problem domain like the web where the ecosystem (HTML, CSS, Javascript) is so organic and bug-ridden.

The most interesting aspect is how the nature of the web has allowed a relatively unknown language like Ruby to spring to the forefront after more than a decade of obscurity. If someone in the Java camp had looked at PHP 7 or 8 years ago and said, "why can't the easy stuff be easy?" instead of "don't these people know anything about software engineering?", then maybe Rails would have no foothold today. In and of itself that's not so spectacular, but considering the decline of desktop software, and the rapid adoption of web-based solutions, frameworks like Rails and Django really are in a position to drive general-purpose language adoption within organizations.

Other great languages (lisp, smalltalk) that sank into obscurity may never have had such an opportunity.

Luke Plant
(July 18, 2007 04:37 PM #)

Another example of this kind of conciseness with Django is the Cross Site Request Forgery protection that you get with Django (enabling it is one line in your config file, and after that you basically stop worrying about CSRF).

If you count by 'words', the implementation is about 200 words of non-comment code, the documentation about 450 (disclaimer: I wrote it).

See:

http://code.djangoproject.com/browser/django/trunk/django/contrib/csrf/middleware.py
http://www.djangoproject.com/documentation/csrf/

Luke Plant
(July 18, 2007 04:39 PM #)

Another example of this kind of conciseness with Django is the Cross Site Request Forgery protection that you get with Django (enabling it is one line in your config file, and after that you basically stop worrying about CSRF).

If you count by 'words', the implementation is about 200 words of non-comment code, the documentation about 450 (disclaimer: I wrote it).

See:

http://code.djangoproject.com/browser/django/trunk/django/contrib/csrf/middleware.py
http://www.djangoproject.com/documentation/csrf/

Jerome Louvel
(July 18, 2007 05:26 PM #)

Java already has Restlet, a complete and dedicated REST framework. It can be used either as a standalone API or inside a Servlet container.

Also, it already has an advanced support for E-Tags. You just need to expose the tag of your representations/variants and the Restlet engine will take care of setting the status with conditional requests.

See the Variant and Tag classes:
http://www.restlet.org/documentation/1.0/api/org/restlet/resource/Variant.html
http://www.restlet.org/documentation/1.0/api/org/restlet/data/Tag.html

Restlet home:
http://www.restlet.org

James Abley
(July 18, 2007 08:14 PM #)

I'm currently looking at implementing conditional GET for some applicaitions I've inherited. We're using Java for this. I know Python fairly well; Ruby less so and Rails not at all. I've been reading Sam for a couple of years, so the importance of ETag support has been drummed into me.

But in my view, both the Rails and the Django implementation suffer from doing too much work. They are nice in that you can get the benefits in terms of saving network traffic. But what about not bothering to create a response in the first place? For that, you need to have a good definition in your application model of object identity; effectively defining the equivalent of the Java equals contract for each resource. To take the simile further, if you override equals, then you should override hashCode (or ETag). I think Joe talks about this very well and it's something I keep going back to when talking to people about this.

adw
(July 20, 2007 03:25 PM #)

I've been enjoying reading many of the recent articles on ETags but when I try to examine how to apply this to my work I start to struggle. Take a typical 'enterprisey' web app that connects to a db and one or more non-db resources, be it SAP/Siebel/BPEL engines/whatever. (that's typical for me, may not be typical for you....if so, good for you! :) ). You get to step 5 in the InfoQ description of how ETags work and make your life better :

5. Server examines the ETag and determines that the page hasn't changed since last time the client requested it, so sends back a response of 304 (Not Modified) with an empty body.

Hmmm.....suddenly things get tricky. I connect to db's that are also written to by other applications, or I connect to other applications that have their own interfaces for updating data. Suddenly the amount of work it takes to verify 'has the data on this page changed' starts to approach the amount of work it takes to just generate the page.

This was meant to be a comment on why ETags aren't embraced by Java web frameworks. Most of the Java web frameworks have come out of the corporate IT development world, or were created with that environment in mind. Concepts like 'has this page changed' suddenly get a lot tougher.


(July 21, 2007 03:10 AM #)

My understanding has been its easy for a framework to automatcially handle etags for a static file, since the framework can look at the last modified time of the file to detect when it got changed (and hence the etag is potentially stale).

The problem comes with dynamically generated responses. The user has to tell the framework when the dynamically generated response has changed... this is where the bulk of the complexity lies according to me. In such cases the framework automatically generating etags based on MD5 does not solve the real problems.
Furthermore, it might be more efficient for the app to generate a domain-specific etag without buffering the response in order to compute the etag.

Am I missing something?

Thanks
Vish

Post a comment

(you may use HTML tags for style)




Remember Me?

Trackback Pings

TrackBack URL for this entry:
http://www.dehora.net/mt/mt-tb.cgi/2120