« Heaven - 111 | Main | Scrapemonkey »

Hitting reload is the framework job

I'm building a simple enough web app, to manage some project related data. Withoug claiming the ability to see around corners, I'm quite sure this app will grow over time because the data it's working against is nebulous and that will push for more and more views. The main decision so far has been to keep data in XML+RDF - for example there is project data in DOAP files and person details are in FOAF and bits of DC are scattered about, RIG chunks, and so on.

I'm telling myself I'm using RDF+XML because I want to be able to pull data in from anywhere. That's true, but to be brutally honest I can't be bothered designing and maintaining yet another relational schema for yet another webapp - doing so is starting to make as much sense as designing my own filesystem or TP monitor. Life's too short, too short to be working on technology that can only possibly make sense when you're in dressed in combats and vans listening to Pearljam pretending it's still the nineties... there's a real wish to conduct oneself at a higher level of abstraction before complete dementia sets in. What's the point in designing tables for a webapp when an RDF-backed store will manage the data for you and RDF queries will come back as tabular data anyway? There are RDF triple stores that will handle in the order 10^6 statements - Leigh Dodds is doing some research on that, up to 10^8 by the looks of things. If I need queries instead of hacking out iterators+fiters I'll use versa/itql/rdql. Now, saying I never want to design another relational schema again is not to say I don't want to use a database. Most of these RDF triple stores are in fact using an RDBMS in the background, as the filesystem and indexer, it's just that the relational schema in use is not exposed to the application.

Can't say I'm too fussed about having a nice object model for the domain either. Yes, it's heresy not to have an object model for the domain - out of the corner of my eye, as I write this, I can see that Eric Evans' book is trying to wriggle off the shelf and wallop me upside my head.

Other than not using an RDBMS directly, and not being too fussed about objects, I wanted these capabilties:

  • Login+sessions
  • Easy XML out
  • Easy URL design
  • Easy URL/action mapping
  • Easy Atom subscriptions on views
  • Save query as bookmark
  • Save filters as bookmark
  • Provide adequate opportunity to look at some frameworks

I didn't care much about:

  • Protecting graphic designers from code
  • Protecting coders from user interfaces
  • Planetary class scaling
  • Shopping carts
  • Pet store transactions

More heresy - no doubt this project will be a disasterous conflagaration of worst practices. I can't wait.


Once you know what the web app is about, you then have to decide what to write it in. That alone is a research exercise. I looked at RoR and I don't know, maybe a dozen Java and Python frameworks.

RoR: RoR is really nice, but a lot of the value is tied up in hooking into an RDBMS schema via ActiveRecord. I'm not using one of those, I'm using XML+RDF, so that takes me off The Golden Path. And even the RoR guys will tell you want to stay on The Golden Path. Maybe I could write an ActiveGraph (Redland RDF has Ruby bindings), but who knows what else would get rewritten by the by - a lot of RoR magic is in mapping the DB - when that's gone a lot of value goes with it. Please yes, I got past the demo and understand that scaffolding is only one part, I just don't understand what the value beyond that is. In RoR, The Golden Path is the system value. [update: Bruce D'Arcus pointed me at Obie Fernadez' musings; seems like there's some pent up demand for RDF on Rails]

Python: The state of the web frameworks in Python is nearly as confusing as Java, no small feat. There's lots of 'em and I have no good sense where the community interest really lies. If you are doing CMS, Plone wins hands down, but I'm not doing a CMS. Zope's a parallel world and I'd have to get around zodb for RDF. Twisted is fine for building servers, but pushes too much back onto an app developer even with Nevow. CherryPy I'm still playing with, it looks nicest so far, and feels closest to a 'done' thing. Greg Wilson has also noticed this excess in the Python world recently, and he thinks the Python community needs get on message - that would not be my conclusion. Python is also overflowing in templating languages, of which Clearsilver, Tal and Cheetah are notable - tho' I'd like to try out Kid - Ryan Tomayko cracks me up.

Java: I know the Java space better than Python. Struts is out for reasons of verbosity and sanity retention - there's that XML config file format (if you need a graph, use a programming language or RDF). By the same criteria, JSF is also out, never mind it's unproven, insofar as the answer to MVC on the Web is not neccessarily even more MVC on the Web. Tapestry looked interesting but it's squarely targeted at HTML output and stopping code monkeys and graphics monkeys squabbling over who gets which bananas. The latter is a non-requirement and the HTML only thing I don't entirely get, even though I gather Tapestry is very focused on non-programmers. I read Howard say somewhere once he doesn't believe in multi-site output - maybe I've been too long in the Atom world but it seems to me publishing as Atom/RSS is becoming a requirement. Spring webflow doesn't seem to offer any value for a two tiered web app where objects are going to be incidental by design - Spring also has that graphlike XML config going on as well. Webflow by the way, is the one part of Spring which I gather has stated it is not targeted for simplicity - instead it's for complex application flows. The rot starts there.

I've complained about Java web frameworks before, especially this obsession with MVC, but much of the issue is with Java, not the framework designs. There are two many steps involved in deploying a servlet app, or making changes to a running one, or updating half-a-dozen XML files - it's ridiculous. All I want to do is edit and hit reload. Letting me hit reload is the system job. To get there means scripting support.

(Non) Conclusion: RoR's database dependency and Object MVC obsession isn't working for me. of course if you read the 37Signals weblog (and you should) your conclusion could be that I'm approaching this entirely wrong and I should start with a user interface as the requirements and work back. But you know, slick UIs are like nice shoes - awfully sexy but awfully transient - you put them on and still no-one sees the real you. Whereas data will set you free. Less surreally - I don't care what the UI is made out of as long as its not arduous and doesn't dictate how this data is going to be structured.

CherryPy is the closest thing to a decision I've settled on in the Python world, but I'm really not sure. Not at all. [update: I am getting a ton of feedback on the Python side of things, enough to have me going back to reassess as I clearly don't have enough of a clue]

The best option for Java seems to be WebWork (one of my favourite web applications is built on it). WebWork is a big improvement over Struts and the Java world is really missing a trick by going straight to JSF. The main problem is setting things up for scripting, to get out from under the compile/deploy/load loop. PyServlet comes with Jython, but that leaves things configured backways, by putting the Jython at the front of processing chain; whereas it would be better to script the action code to be invoked by a framework. Then the framework is dealing with skinning, sessions, stringtables and the like. I'm thinking about embed a JythonActionSupport into WebWork so it can call out to .py files implementing the ActionSupport interface. WebWork uses proxy generation to create and fill out your action objects, hopefully there won't be problems integrating that magic with Jython.

Generally, the web frameworks situation is depressing unless you're a researcher. If someone thinks there's a framework I should be looking at, shout. Deparalysis imminent.


June 18, 2005 07:04 PM

Comments

Bruce D'Arcus
(June 18, 2005 07:27 PM #)

Bill -- did you see the stuff on RDF and RoR over here? It seems more than one person is working on implementing something like ActiveRecord for RDF, which would be a good thing not only for the RDF world, but also for RoR (which I agree is too RDBMS focused right now).

Bruce D'Arcus
(June 18, 2005 07:28 PM #)

Oops. try this link instead.

kellan
(June 18, 2005 07:56 PM #)

RoR+RDF is the holy grail in my book, and I think it would be a **huge** mindshare boost for RDF.

Nothing like providing smooth RDF integration which everyone knows they *should* use to the sexy new technology that everyone *wants* to use to get a win-win.

T. Joad
(June 18, 2005 08:27 PM #)

It's somewhat saddening that people so often think in terms of [neat idea] + RoR, rather than [neat idea] + general Ruby abstraction layer.


Anyway, Bill, you may want to look at Nitro/Og, which might better lend itself to an RDF storage system (though such a mapping does not yet exist).


It seems to make it easier than, say, Rails, to work out your domain model first in code and then move to some particular persistnce mechanism when needed.

Parand Tony Darugar
(June 18, 2005 10:44 PM #)

So what are you going with after all? CherryPy or some Java framework?

Mike
(June 18, 2005 10:51 PM #)

If you're willing to use RoR and Ruby, then you might be willing to use Perl and Catalyst (new article at Perl.com).

Ramon Felciano
(June 19, 2005 12:37 AM #)

Don't forget Spyce at
">http://spyce.sourceforge.net/ as a Python framework, especially for quickly putting together small sites. It is lightweight, stable, minimalistic, and reasonably complete. It has a simple action processing framework, and can plug in to Cheetah for templating (which I expect you could also use for your XML out).

It is view-centric (model 1 I think), so if you are religious about a MVC-like action vs view separation, it may not be a good fit. Definitely worth knowing about tho.

Ian Bicking
(June 19, 2005 02:19 AM #)

I think you'll be happy enough with CherryPy, Webware, Quixote, or one of the other similar frameworks. I don't think Spyce will appeal to you, since I suspect you will put code before templates, since obviously UI is not your first concern. For templating I'd choose ZPT or Cheetah -- both are mature and usable. ZPT is more XMLish, for better and worse.

I think the choice is mattering less; or at least that's what I'm trying to make happen with Paste (http://pythonpaste.org) -- I'm not quite there yet, but I'm confident it will happen, and be applicable to whatever you choose.

Robert Leftwich
(June 19, 2005 03:22 AM #)

Aquarium (http://aquarium.sourceforge.net/) is definitely worth adding to your Python list. It has excellent support for automatically reloading and compiling Cheetah templates and the reload feature can be used on any module that doesn't maintain critical state (e.g. reloading model classes). I've recently committed a URL connector that was heavily influenced by RoR's Routes (see http://sourceforge.net/mailarchive/forum.php?thread_id=7276170&forum_id=38053
for a brief description) that makes URL design and action mapping even easier. While it ships with a database-backed model implementation it doesn't mandate its use and plugging in another model is easy. Well worth checking out!

Mik Lernout
(June 19, 2005 11:44 AM #)

Bill,

You might want to have a look at Fulgora (http://narcissus.futurestreet.org/) a proof on concept I am working on. It's like PHP (no redeployments) but runs in the JVM (on a servlet engine), and adds MVC , application state, filters...

Still very mich a concept, but I am looking for input...

Mik

Howard M. Lewis Ship
(June 19, 2005 02:07 PM #)

I don't find the statement "Tapestry is focused on non-programmers" to be accurate. It's designed around team development, especially HTML vs. Java type teams. You most certainly still program, you just don't do any of the plumbing.

Tapestry's deepest rut is HTML, but anything that looks even vaguely markup-related is reasonable. Any you can always fall back to raw servlets where that's appropriate.

What I don't believe in is a single page/component that auto-magically adjusts itself for different devices and/or markup languages. Those approaches always seem to be like coding inside a case statement, and/or they bulldoze over the more subtle nuances. If you want a Tapestry application to serve desktop HTML clients and phone-based WML clients ... build two distinct apps and share code in the back end. Your users will thank you for not making them interact with a FrankenUI.

Leigh Dodds
(June 19, 2005 08:43 PM #)

"...to be brutally honest I can't be bothered designing and maintaining yet another relational schema for yet another a webapp..."

Bingo! This is one of the arguments in the "anti-hype, pragmatic RDF pitch" that I've rehearsed (i.e. bored!) people with a few times over beer, but not written up yet.

I see moving to RDF, as an standardised abstract model -- never mind the syntax -- as one way that I can push a whole lot of tedious, even irrelevant issues down a layer in the stack. "Outsourcing your data model design" is one way I've pitched this view.

I've spent far too much time on too many projects haggling over data models, revising code to deal with minor naming changes, and then writing integration code to map data structures between applications. By moving to RDF I can largely avoid many of these issues.

The data is easier to manipulate, but while I've lost a custom schema, I've not necessarily lost all the benefits of a standard RDBMS. With a relational back-end I can still be certain of stability, ease of backups, etc, etc. As RDF has a formal model, query languages (SPARQL) haven't been lost either.

This is a big win in my book and contributes a lot to productivity, especially at the start of a project.

The "pragmatism" in this argument is that notions of a machine processable web, complete with machine based reasoning doesn't enter into it. There are immediates benefits to be gained.

But, by using a formalism upon which rule languages and other neat features can be layered, I *may* just have hooked into a grander scheme. If so, great. That'll probably be a whole other set of issues that I can push down a layer.

The other side to this pitch is the way that RDF+REST ("R & R") tie nicely together via URIs. But this comment is long enough already!

Dan Hatfield
(June 20, 2005 10:21 PM #)

"...to be brutally honest I can't be bothered designing and maintaining yet another relational schema for yet another a webapp..."

And, amen to this:
"What's the point in designing tables for a webapp when an RDF-backed store will manage the data for you and RDF queries will come back as tabular data anyway? "

But URI's will be the death of this...we have to do better than this


(from the JENA RDQL examples page):
SELECT ?resource, ?familyName
WHERE (?resource info:age ?age)
(?resource, vCard:N ?y) , (?y, <vCard:Family>, ?familyName)
AND ?age >= 24
USING info FOR <http://somewhere/peopleInfo#>
vCard FOR <http://www.w3.org/2001/vcard-rdf/3.0#>

I mean ouch...no one wants to write this stuff...
RDQL needs a good subsitution or shorthand scheme for URI's or something...perhaps I missed it? I'm rather new to RDQL/RDF.
Otherwise, I'll spend all my time debugging my own typos in URI's alone..

Dan

Dan Hatfield
(June 21, 2005 01:05 AM #)

Well, what I attempted cut and paste from the jena RDQL tutorial seems to be getting edited out. My point was this - if we have to code full URI syntax into these RDQL queries then I feel to see how this really helps me. I can't imagine anyone wanting to write queries with embedded URI's...

Bill de hra
(June 21, 2005 08:56 AM #)

Dan,

"no one wants to write this stuff."

some people think the problem lies in using SQL syntax to operate over graphs as much as the URIs rather than a graph/path language. You should check out Versa as an alternative. Admittedly, SQL-like is an easier pitch. That said, it would be nier to declare the USING clauses at the top of the query; that way they wouldn't look any worse that a Java/C# import mechanism.

Bill de hra
(June 21, 2005 08:58 AM #)

Parand,

"So what are you going with after all? "

I thought I knew, but now you guys all came along and confused me! No... I'm still thinking on it :)

Bill de hra
(June 21, 2005 09:04 AM #)

Howard,

"I don't find the statement "Tapestry is focused on non-programmers" to be accurate. "

That was off-the-cuff. I'll fix up the entry.

"What I don't believe in is a single page/component that auto-magically adjusts itself for different devices and/or markup languages."

Ah, ok. In OO land we know the fix for that - replace switch with polymorphism. The RESTian in me says that should be done using HTTP conneg and the server should dispatch on the requested media-type and/or UA string.

But this belies a problem with a lot of frameworks that want to abstract out HTTP - you lose information in the headers that can be used to inform the app logic. It's not so bad when you're writing to a session based webflow, you just pull context from the session but it's a real problem if you;re writing a service that is going to used by people and client automatons. I think we need to understand that HTTP, as something that constrains and informs your app, is sometimes the solution, not the problem. This is the only think I really dislike about WebWork - it thinks HTTP is a problem.

Bill de hra
(June 21, 2005 09:06 AM #)

Ian,

"since obviously UI is not your first concern"

Whoa, that's not what I intended to say. What I intended to say was that I don't place too much importance on keeping firewalling graphics people and programmer people.

Dan Hatfield
(June 21, 2005 02:56 PM #)

A USING clause would fit the bill nicely...like you. I probably should have been more clear as ot my needs - using a tripe-store as a flexible extensible data model makes a lot of sense to me. So, when looking for a query language to return a result set of RDF data, SQL seems to work for me when dealing with business data (Versa being more for navigation of the RDF tree, if I'm not mistaken).
Anyway, I wait in hushed anticipation for your decision. I'm hoping you'll go for a Ruby on Rails extension as it would be the excuse I needed to explore RoR.
Dan

Robert Brewer
(June 21, 2005 04:55 PM #)

Bill,

CherryPy is nearly ready for a 2.1 release which I think you'll find works well with Atom. We've sped up request-handling by a factor of 3 to 4 (depending on which server you use), and included support for arbitrary HTTP methods. If you have any questions about CherryPy features or direction (or hacking ;), feel free to ask me in email or on #cherrypy.

Leigh Dodds
(June 21, 2005 08:15 PM #)

Dan,

You should look at SPARQL. It's still SQL like, but the PREFIX mapping (similar to RDQL USING) goes at the top of the query. Much easier to read, IMO.

Dan Hatfield
(June 22, 2005 02:48 AM #)

Leigh,
I'll check it out. Thanks for the heads up!
Dan

Leslie Hensley
(June 22, 2005 06:21 PM #)

Bill,

Have you looked at Seaside? It is at least part of the answer to the MVC madness that has gripped so many web developers. If moving to Smalltalk is too big a commitment (it was for me), then check out the Java based clone that I'm working on, Lakeshore.

Patrick Lightbody
(July 8, 2005 09:57 PM #)

Bill,
We're trying to make WebWork more "restful", starting with better support for URLs. Check out the start of that work:

https://webwork.dev.java.net/source/browse/webwork/src/java/com/opensymphony/webwork/dispatcher/mapper/

Patrick

Eric Hanson
(July 9, 2005 03:03 AM #)

C'mon now, what's wrong with Pearl Jam??

Not my favorite band or anything, but we wonder why RDF can't kick this Ivory Towers rep...

Vincent D Murphy
(July 16, 2005 07:37 PM #)

Django is a python web application framework. Its got a very nice website with excellent documentation.


It seems like Rails, and better in some respects. For example, the database is derived from the model definition; I think this is more amenable to using an RDF database (ActiveOntology notwithstanding). Definitely worth a look.

Harry Fuecks
(October 3, 2005 05:42 AM #)

You may find this useful if you haven't seen it: http://www.wiwiss.fu-berlin.de/suhl/bizer/toolkits/index.htm "Developers Guide to Semantic Web Toolkits
for different Programming Languages"

The author of that comparison is also the author of RAP (http://www.wiwiss.fu-berlin.de/suhl/bizer/rdfapi/) - RDF API for PHP - presumably he's doing the comparison because RAP fares well.

Payday Loans
(February 6, 2006 09:08 PM #)

You have done a great job with your blog.
Payday Loan http://www.payday-loan-place.com

Payday Loans
(February 7, 2006 12:37 AM #)

You have done a great job with your blog.
Payday Loan http://www.payday-loan-place.com

Larry
(March 26, 2006 02:44 PM #)

Mattew
(April 22, 2006 03:44 PM #)

Trackback Pings

TrackBack URL for this entry:
http://www.dehora.net/mt/mt-tb.cgi/1592

Listed below are links to weblogs that reference Hitting reload is the framework job:

» A Group of Unrelated Items from franklinmint.fm
i HATE it when people say davin: Do you hate the use of "slippery slope" in general or only it's... [Read More]

Tracked on June 21, 2005 04:12 PM

» RDF and Library Metadata Interoperability from Lost Boy
I've been having an interesting email discussion with Bruce D'Arcus and Richard Newman as a result of Bruce pointing us both at this posting on Metadata Interoperability by Kevin Clarke. I thought I'd write up some points to respond to that article her... [Read More]

Tracked on July 6, 2005 02:15 PM

» Bill de hra on Web frameworks from Mchel Foghl's Weblog
This is an interesting posting Bill de hra: Web frameworks reloaded. Just use.... In it Bill de hra describes Plone, Struts, Django and Rails - his favourite four web programming frameworks. In a similar posting last year Hitting reload is the framew... [Read More]

Tracked on February 6, 2006 01:21 PM