« WWW cubed: syndication and scale | Main | Writing Servlets with Jython tutorial »

Slab! Interoperation and evolution in web architecture

Mark Nottingham on POST being special:

From the standpoint of interface semantics, the difference here is really just one between saying 'POST machineMgmtFormat' and 'MANAGEMACHINE.' In the uniform approach, the service-specific semantics are pushed down into the type (media type) and content (entity body) of the data. In the specific approach, they're surfaced in the interface itself.
This isn't a big difference.

Mark is commenting on Don Box's expectations of the insights that could result from the WS-Transfer specification. I would say the difference is substantial. The reason for this is is that determining a trade off between interoperability and evolvability in a language (or protocol) is a critical axis, particulary in computer languages. If we go back to the fundamental underpinnings of a language or protocol verb set, we arrive at speech act theory, the theory of how we communicate with other actors to achieve a goal. In crafting such acts of speech we find two basic approaches to the creation of verbs or action words. The first is that the the verb set is open to addition, but not to modification. Anyone is free to add new verbs to the language. The second is that the verb set is fixed - no additions, no modifications. The latter enhances the uniformity and interoperability of the language at the cost of evolvability. The reason this matters is simple enough - the use-value of a verb is a function of the number of clients and servers that share an understanding of it. Not quite Metcalfe's law, but along the same lines. the reason it matters even more in computer languages is also simple - computers for the most part do have have the ability to learn new verbs or bootstrap them into the language the way people using natural languages do - thus the need for precise specification.

While HTTP does allow for addition, practically speaking, the verb set is fixed. It has taken years for WebDAV additions to HTTP* to penetrate more than a fraction of the Web. Other efforts, such as HTTPR, an extension for reliable messaging, have gone nowhere. Even within the mandated verb set of HTTP itself, we find the availibility of verbs varies widely (notably PUT and DELETE) with entire eco-systems (such as mobile device clients) having only a subset. One can argue that the active verb set of HTTP comprises a subset of 3 verbs - HEAD, GET, POST - anything else is dead tongue.

The problem with HTTP POST, and what makes it special, is that it is a semantic catchall. What makes POST a uniform speech act is ironically the absence of interesting semantics and lack of specificity. Although it has specifications that are helpful to people when dealing with caches and state management, there's no controlled means of defining what one is actually saying with it, without some further and prior agreement between client and server. The reality is that POST has been overloaded and abused to get systems talking even where such systems would have done better with an alternate verb - and the result is that in many systems the POST speech act is close to meaningless. WS-Transfer aims to throw some light into this void by providing a means to add consistent meaning to operations that would often be drilled through POST. In particular this may prove valuable for use with web services toolkits which are often designed to hide the networking aspect of communications from the developer.

While they may share a common basis, WS-Transfer and HTTP are fudamentally different in mechanism and this is not a design consideration we should so easily gloss over. This is because we now have a sufficient understanding of languages are protocols designed for use in open and distributed systems to be concerned that despite WS-Transfer looking like a good thing on the drawing board it might flounder in the trenches.

Why would this happen? It comes down to how the trade off between inteoperation and evolution affects the way extensions are managed. Neither HTTP or WS-Transfer has as strong story to tell as one might expect. On the one hand, REST advocates laud HTTP's and the Web's success as an architectural triumph and will happily point to WS-Transfer and say "I told you so". They are much more circumspect in their musings of how people actually use HTTP to get things done, which is not consistent with, and sometimes directly contrary to, the REST architecture. On the other hand, promoters of WS-Transfer see it as targetting those who need something more specific than POST or who are firmly grounded and trained in middleware practices and not network protocols. WS-Transfer however does not offer a consistent means of addition - it describes a structural means for declaring new verbs, without an ability to declare what they mean. And it is difficult to see WS-Transfer stopping at the handful of verbs it defines currently. It thus may if left unfettered, act as a distributed Tower of Babel, affecting people's ability to communicate. This is compounded by the fact that it is considered good practice in object oriented middleware to be highly specific in the methods used on objects - verb uniformity is not a goal, quite the opposite in fact. In one sense, HTTP represents a more conservative and risk-averse approach to the evolution/interoperation tradeoff.

In computer jargon, this means of addition is often called 'semantics', though philosphers and mathematicians may take umbrage at that (they have a very specific definition of the term). It essentially defines how you may extend a language in terms of its existing primitives and is when successful based, universally it seems, on formal logic. There are existing technologies that strive to achieve this such as FIPA-ACL, Lisp and OWL. Each is successful in its own way (Lisp's ability to extend itself consistently has allowed it to remain current for going on 50 years), but are often considered obscure or academic by mainstream technologists and practitioners. RDF, which can be used to make logical assertions about URIs, is ideally suited for articulating the semantics of any extended WS-Transfer speech acts. This is because WS-Transfer uses URIs to name its verbs.

Where WS-Transfer may prove useful is in breaking down barriers and getting architectural factions to the table by raising awareness that there are issues with both REST and Webservices approaches. The debate between REST/Web and Webservices/Middleware communities has often been acrimonious. Aside from complications resulting from the strategic and commercial agendas imposed by the industry that have resulted in a plethora of competing and inconsistent web services 'standards', the core technical debate has been arcane. It is not always obvious to outsiders and system stakeholders why some kind of agreement can't be forged. While the architects and vendors are busy in argument, customers and practitioners are frequently left with little by the way of clear advice on how to either construct new systems or integrate existing ones. The outcome is that systems are being built, week in, week out, than cross the Web/Middleware boundary without being informed by both approaches and where they are approriate. This implies projects with excess risks and costs, wasted effort, re-learning of best practices or what is already in the state of the art. This is all the more important now that systems that incorporate web and middleware aspects are increasingly the norm (the size of the industry sector affected is significant). Any effort that raises mutual understanding is most welcome.

* WebDAV folks are often adamant that the verbs WebDAV specifies are HTTP verbs - there are no WebDAV verbs.

Update: Originally I asserted in the first sentence of the entry, that Mark Nottingham thought that WS-Transfer matters. Mark was kind enough to point out he never actually says that (see comments), so I've removed the assertion.

October 10, 2004 03:27 PM


Mark Nottingham
(October 10, 2004 05:24 PM #)

Did I say WS-Transfer matters?

Jon Hanna
(October 12, 2004 03:59 PM #)

First time I've ever heard REST advocates described as "circumspect" before. Really advocating REST means directly engaging with such violations of REST architecture and showing how they are suboptimal or downright harmful (we don't need to advocate REST as the architectural style of the web - the point of Fielding's defining it - that's already won, we just need to point out that it's won).

David Ryan
(October 13, 2004 09:12 PM #)

Your comment, "computers for the most part do have have the ability to learn new verbs or bootstrap" is interesting. A verb could be considered any data in reality. My work has consisted of developing something called Argot, designed to agree on the format of binary data between client and server. In a way agreement of syntax can actually infer agreement of semantics. Given a type name, and type description it is possible to agree that the format of the data is the same. While this does not guarantee the semantic meaning, it does provide the ground work. Infact the semantic meaning may be different between client and server; agreeing on the syntax is often enough.

The problem that you seem to infer is that encoding meaning across boundries of protocols, HTTP, WS-Transfer is open to interpretation. Argot develops a common data format agreement between systems. In effect allowing verbs to be exended using strongly typed semantics.

I'd be interested in knowing if you think that a system that provides agreement of verbs and data in general offers any advancement over the current WS-* mismatch of concepts.

Mark Baker
(October 14, 2004 07:50 PM #)

I think you're mischaracterizing POST. By far the most prominent use of it is with HTML forms; I'm sure that has to account for 99% of all POST traffic. And in every single instance of a form, POST semantics are being used propertly ("Providing a block of data, such as the result of submitting a form, to a data-handling process").

POST is not a catch all as specified. It's semantics are extremely general, but that's not the same as ambiguous. GET, PUT, and DELETE are equally as general (they're uniform too) but you wouldn't say they were ambiguous would you? POST is simply a name given to the application semantic of a document being processed by a document processor. That's it.

Now, it just so happens, in the cases where you need to use HTTP as a tunnel, that POST is the best possible fit since its semantics are the closest match to an arbitrary non-uniform invocation. But it must be acknowledge that you're not using POST semantics.

Trackback Pings

TrackBack URL for this entry:

Listed below are links to weblogs that reference Slab! Interoperation and evolution in web architecture:

» Wrong path to WS standards from Middleware Matters
Over in his latest blog entry, entitled Slab! Interoperation and evolution in web architecture, Bill de hra has written yet another must-read piece. (If you're not subscribed to his blog, then you're missing out on some generally great insights.) I'd ... [Read More]

Tracked on October 12, 2004 04:59 AM

» Bill de hOra On WS-Transfer from Don Box's Spoutlet
TITLE: Bill de hOra On WS-Transfer URL: http://pluralsight.com/blogs/dbox/archive/2004/10/13/2761.aspx IP: BLOG NAME: Don Box's Spoutlet DATE: 10/13/2004 05:27:15 AM [Read More]

Tracked on October 13, 2004 05:27 AM

» The difference between Web and Middleware from The Now Economy
Mark Nottingham wrote: From the standpoint of interface semantics, the difference here is really just one between saying “POST machineMgmtFormat” and “MANAGEMACHINE.” In the uniform approach, the service-specific semantics are pushed down into ... [Read More]

Tracked on October 14, 2004 08:58 PM

» Still RESTless from XML Eye for the Object Guy
TITLE: Still RESTless URL: http://pluralsight.com/blogs/tewald/archive/0001/01/01/2871.aspx IP: BLOG NAME: XML Eye for the Object Guy DATE: 10/18/2004 04:39:05 PM [Read More]

Tracked on October 18, 2004 04:39 PM

» WS-Transfer from Michael Curry SOA Blog
The interesting (and endless) debates around WS-* specifications continues with WS-Transfer. [Read More]

Tracked on October 20, 2004 05:36 PM

» REST feet forward from Raw
Tim Bray has put already himself forward as a loyal oppositionist when it comes to WS-*, and is now letting James Governor do the talking: SOAP is boring, wake up Big Vendors or get niched. Predictably there was a backlash, well actually quite a well-r... [Read More]

Tracked on February 17, 2005 10:36 AM