Patterns of Web Architecture

Eric Newcomer: "This afternoon I finally caught up up on Steve Vinoski's recent article and blog entries about the "evils" of RPC. If you aren't already among those who have read them thoroughly, I'd encourage you to. Including the comments, it's one of the best discussions of the merits and demerits of RPC and REST that I've ever seen. The core of his argument is that the RPC abstraction is not helpful - in fact the opposite. Explicit programming is preferable when creating distributed applications.

As someone in the middle of designing another RPC based system (Distributed OSGi), though, I'd like to weigh in with a few thoughts. ;-)

As I've said before, the distributed OSGi design does not really represent a new distributed computing system. The goal of distributed OSGi is to standardize how existing distributed software systems work with OSGi, extending the OSGi programming model to include remote services (today the standard only describes single JVM service invocations).

Because the design center for OSGi services is the Java interface, RPC or request/response systems are a more natural fit than asynchronous messaging. In fact because we are trying to finish up our work this year ;-) we have postponed tackling the requirement for asynchronous communication systems to the next version."

That's interesting; I didn't know there was a distributed OSGi. I wonder how the model would tee up with the protocol buffers approach, which when I looked at that first I thought was an RPC, but on a second glance it's more like an IDL that is encourages mustIgnore versioning and doesn't really say much about how the protocol messages are shipped about (or put another way, I wish Google would expose how they use protobufs :).

Sanjiva July 2008: "REST is now beyond the peak of the hype curve and is sliding down. Waay down."


March 2006: "Hype could be a problem for REST if the 'industry' gets on board with it as a commmercial factor that requires ticking off rather than some dorky engineering stuff – 'Do they have REST? Check.' So will keeping an eye out for land-grabbing on the term 'REST', so it remains a crisp technical term "

REST hype helps no one. It makes discussing what are the good tradeoffs to make sure "Web APIs" (a non-sequitor) are simulataneously optimal for programmers and those to need to get something done, as well as being stable interfaces over time for businesses and other who need to take a long term view. That "REST" is hyped makes it harder for people to make good decisions. I suppose what's funny now is that "not doing REST properly" was almost a badge of honour a few years back.

Sanjiva, June 2007: "Is that RESTful or not?" A good question from a good post, worth reading.

I like checklists:

  • Are URIs used as the standard identifier for objects?
  • Does the design use a client server model?
  • Does the client store just the session state and the state of the application?
  • Does the server store just the state of the object?
  • Does the server provide well documented content types that can be selected dynamically?
  • Does the design support code on demand?
  • Do clients and servers communicate use a fixed set of methods, where each method's side effects and idempotence aspects are explicity defined.
  • Can caches be introduced without understanding the content, and the server/client states?
  • Is there more than one method?
  • Is the architecture layered, and can intermediaries be introduced between clients and servers without altering the client server interaction?
  • Stateless 1: Does each client-server interaction provide all the information needed for the network connectors between the client and the server to convey the request?
  • Stateless 2: Does each client-server interaction provide all the information needed to process the request on the server?
  • Stateless 2: Does each representation sent to the client provide the full state of the resource at that time?
  • Are all methods called on object URIs rather than URIs containing methods?

Those are some of the important design questions, once you get beyond surface things like URI formats and which formats are "open". Most or all of these need to be yes. Some are non negotiable.

Anne Thomas Manes: "The REST architectural style defines a number of basic rules (constraints), and if you adhere to these rules, your applications will exhibit a number of desirable characteristics, such as simplicity, scalability, performance, evolvability, visibility, portability, and reliability. [...] Non-RESTful POX applications violate these basic rules. First, they don't define a URI for every resource. And second, they don't constrain the interactions to the methods defined in the uniform interface. Instead they define a single URL that represents an operation that can be performed on any number of unnamed resources. Essentially they are tunneling RPC calls through the URL"


Practical decisions that the REST architecture does not answer directly for service and system designers:

  • How should collections of objects be represented to clients?
  • How should a service be exposed?
  • What formats should be used?
  • Should I negotiate content types using URI parameters or headers?
  • How should authenticated access to and update of resource state be done?
  • How are resources instantiated?
  • How are private domain objects mapped to standard content types so as allow layering, implementation hiding and reduce dissonance between the internal object domain and the exposed on the wire representations?
  • How can resource state or an individual field be updated efficiently?
  • When should I use POST?
  • How can updates be idempotent, or cached?
  • Can I use parameter based identification and be consistent with REST?
  • How do I return partial versus full content? Should I? Specifically how do I represent data graphs to optimise the network architecture?
  • How can a client define a relationship between two objects?
  • How can I make good use of cookies?
  • Should I use forms post or a data structure such as XML/JSON to create objects?
  • What's the best way to send messages over HTTP?
  • How can I represent a queue?

I'm sure there are others.

Stefan Tilkov: "weirdly enough, building your own protocol using Web services is a lot easier than understanding and using HTTP correctly. REST and RESTful HTTP are not easy"

Sanjiva: "The world is inherently heterogeneous, even in languages and language runtimes. There are 3 core platforms in existence today: C, JVM and .Net CLR. Every language runs on top of one of those. Sticking your head in the sand in only one of those will automatically limit the market you can address."

I think you have to add javascript to that list; if the browser is not a globally deployed platform, then I don't know what is.

Ajax Patterns: RestfulService: "Being a broad architectural style, REST will always have different interpretations. The ambiguity is exacerbated by the fact that there aren't nearly enough HTTP methods to support common operations. The most common example is the lack of a search method, meaning that different people will design search in different ways. Given that REST aims to unify service architecture, any ambiguity must be seen as weakening the argument for REST.

Another issue is portability - while GET and POST are standard, you may encounter browsers and servers which can't deal consistently with DELETE, PUT, and others, if they're supported at all.

The main alternative to REST is RPC (see RPC Service]). RPC is equally broad in definition, but the essential idea is that services are exposed at procedures. You end up POSTing into verb-like URLs such as /match/createMatch?matchId=1995 instead of RESTful, noun-like URLs such as /match/1995. In fact, the difference is significant enough that some service providers, such as Amazon, actually provide separate APIs for each. As a general rule, any set of services could be exposed as either REST or RPC; it's just a question of API clarity and ease-of-implementation. Note also there is some overlap; as discussed in the RPC Service solution, RPC can still follow certain RESTful principles.

From an implementation perspective, REST and RPC differ in that REST requires some explicit design, whereas RPC tends to follow directly from the back-end software model. In the example, it's likely there will be a MatchcreateMatch() method - that's just how most server-side software gets done. So it's a no-brainer to tack on a /match/createMatch web service that mediates between the client and the back-end method. In fact, there are many frameworks that will completely automate the process for you. 

With REST, there's no direct mapping between web service and backend implementation - an impedence mismatch. You need to take a step back and explicitly design your API to be RESTful. If you're following feature-driven design, the API is the first thing you'll produce anyway, since the design will be "pulled" by the needs of clients, rather than "pushed" from the available technology. Once you've designed the API, the web service implementation will effectively be a kind of Adaptor (see Gamma et. al) to the back-end services."

Ironically the browser and HTML is one of the main reasons "non-REST" approaches to APIs are commonplace, especially the REST-RPC hybrid style - a lot of infrastructure and tooling is optimised for HTML form posting and not HTTP itself.

Diego Doval: "The Ning platform is one giant API that runs on what we call the core, a collection of hundreds of servers and dozens of server types that power the platform, presenting a homogeneous view of data and services through a variety of HTTP-based REST APIs.

The Ning APIs are built using REST, but PHP applications running on Ning have access to them through a thin layer of PHP code that simplifies their use from within PHP. However, it is possible to use the Ning REST APIs directly from PHP, it's just a little more cumbersome."

Benjamin Carlyle: "The least common change in a REST architecture is to change the set of methods. Why? Because it’s hard to know what to do when faced with a method you don’t understand. You can deal with this to some extent by returning a response to indicate that you didn’t understand, or only partially understood. The client may be able to refashion the request to take into account the protocol mismatch. However, method changes require special and individual attention. There are few hard and fast rules as to what strategy will work. This is one of the reasons why methods in REST architecture tend to be so heavily restricted: Every one must pull its weight, as reinventing or altering the set makes it difficult to continue moving an architecture forward.

The other set of common changes is a restructure in server URI-spaces. However, this is not a protocol change. The clients still work. They just need to be reconfigured to point to the new URLs. Note that this can be done in-protocol via redirection response codes.

In short, I think that SOA is fine and a proven technology when it is possible to upgrade your whole architecture in one go to the new protocols. I think that REST is the only proven technology when only a tiny fraction of the overall architecture can be upgraded in a single sitting. You can’t upgrade the whole Web. REST accommodates this.

Note that if you don’t have a definitive list of all content types and all methods in use across your architecture, you aren’t doing REST yet. REST uses standard methods and standard media types. It is still up to your architecture to define the sets, but they must be defined and controlled. Not only that, but they need to be defined and controlled separately so that the set of methods and the set of media types can evolve independently."

We need some more books that are written for programmers who need to get something done on the web and want to do so with as much principle as makes sense for them; along the lines of Effective Java, PoEAA or Ajax Patterns. My working title would be Patterns of Web Architecture, or "POWA" ;)






    hmm .. never thought you would change into reddit/slashdot .. what is this ? some aggregator ! :D

    No offense here, but I don't like to follow Sanjeeva's words .. he has flip flopped WAAAAY too many times for me ...

    To the previous commentor: Please show me how I've flip flopped.

    I'm a scientist. I learn from my experience. I learn from others experiences. I learn from mistakes. I am not stupid enough to think that I have all the answers for any question right now.

    I do evolve my thinking over time.

    If that's your definition of flip flopping, then guilty as charged.

    If you don't evolve your thinking over time, then think again about yourself.

    But if you think I'll ever flip and say WS-* is stupid or useless then you're gonna be waiting for a long time. It won't happen when I am done with WSO2 or whatever ... I'm not like Steve Vinoski who flipped against RPC the day he left Iona.

    Despite what people like Tim Bray appear to desire, REST won't kill WS-*. They both have their place in the world and both are here to stay.

    Sanjiva Weerawarana.
    CEO, WSO2.

    "We need some more books that are written for programmers who need to get something done on the web and want to do so with as much principle as makes sense for them; along the lines of Effective Java, PoEAA or Ajax Patterns. My working title would be Patterns of Web Architecture, or "POWA" ;)"

    +1 !

    I have been thinking the same thing lately, and the thought had already occurred to me to nominate you for the task. Isn't much of it already written (in blog entries here)? I'd assume Addison-Wesley would jump at it.

    --peter keane

    @Sanjiva: since you claim to be a scientist, you at least ought to do your homework. You might want to read:

    which I wrote a full year and a half before I left IONA. Oh, and there was this one too:

    which I wrote nearly 5 years before I left IONA. You should at least get your facts straight before issuing your uninformed personal attacks.

    Steve, my sincere apologies to you for that uncalled-for personal attack. I was out of line.



    I love the patterns idea. I think <a href="">Restful Web Services</a> deserves a plus as the first step in the right direction, and would be well complemented by a companion pattern book.