xmlhack: SOAP Web Services: built on a contradiction?

The fifth annual Forum XML, on November 12-14 in Paris, featured a consistent theme--that SOAP-based Web Services seem more than ever based on a contradiction, even as they are being positioned by vendors as a foundation for both loosely coupled systems and a new RPC architecture.

This observation is not new and was already the subject of the article published with the title "Web Services missing the point?" after the last year's Forum XML conference. To some extent, Jim Green provided further confirmation: When answering the question "what are Web Services?", he identified not two but four definitions, which are only partially compatible:

loosely-coupled software components (according to the Gartner Group)
about using best-of-breed development tools (according to development tools vendors)
APIs to packaged apps (according to packaged application vendors)
what you use to get your systems to communicate (according to integration vendors)

Many other presentations confirmed a double mismatch between the theoretical vision of loosely coupled systems based on existing standards and the practical implementation based on technologies which are usually associated with Web Services (SOAP, UDDI, WSDL, ...):

The RPC approach which is one of the selling points of SOAP can't be called loosely coupled.
If the SOAP layer can use existing protocols such as HTTP, the usage that is done of these protocols is such that the infrastructure needs to be totally refactored to be suited for Web Services.

This second point was highlighted in presentations by Cyril Dhénin "Web Services, which gains for which costs?", Didier Girard "Architectures and Web Services" and Jean-Philippe Moresmau "Quality of Web Services" which did show the limitations of existing web infrastructures in a context of Web Services and demonstrate the need to develop a new infrastructure to develop, deploy and administer Web Services.

Questioned on this point, Cyril Dhénin said he saw this as a new example of the usual shift between the management and technical domains. An explanation also given under a slightly different angle by Jean-Marie Chauvet, who elaborated on the distinction between the strategical long term vision of Web Services as a way to manage the services of the extended enterprise by exposing their business objects through Web Services and the tactical short term vision of developers using them as a new RPC mechanism.

Noting that the tactical vision is already there, Chauvet said that the long term vision is still far away and that a lot needs to be done to avoid falling "from the dot.com bubble to XML Babel."

Aside from mismatch between business and technical views or between strategy and tactics, the final word will come from concrete implementations--and these may take unexpected paths. Three customer stories were presented during the session: Sidé Brahimi for the French Department of Defense showed a BizTalk based architecture that one could call "classical", and Stéphane Bidoul for Elia and Benoît Marchal and Patrick Bacher for France Telecom presented architectures exchanging XML documents over raw HTTP. They preferred plain XML to SOAP for the sake of simplicity--and because of the unfinished state of SOAP within the W3C Recommendation track.

The architecture of these applications is similar to what was described by Roger Costello and mentioned in my article last year and is now well known under the name "REST".

Like SOAP, REST uses existing protocols, but unlike SOAP, REST uses HTTP as human users would use it and this can make quite a difference in the amount of infrastructure refactoring needed to deploy Web Services. Furthermore, unlike SOAP, REST doesn't attempt to reinvent RPC mechanisms and is inherently loosely coupled.

Three examples are not enough to extrapolate, but it is very likely that most of the Web Services applications deployed so far are relying on REST architectures (whether they use the name or not) and that far from the glare of the spotlights this architecture is probably still dominating the market of Web Services.

Behind the marketing power deployed on the SOAP/UDDI/WSDL triple by organizations which have often a financial interest to see their customers renew their web infrastructures, one may wonder how long this will last.

Throwing some spanners;

REST is a concept not a technology or a standard.

SOAP has a document "style" of exchange so could be used to build REST like web services (assuming the implementations support it).

WSDL supports any messaging protocol - i.e. XML-RPC is not dead it just needs a schema.

WSDL could be used to describe REST type web services.

Two years down the line who, apart from MS, has made any money with SOAP? What successful new business models do we have?

Re: SOAP Web Services: built on a contradiction? (Yves Reynhout - 20:50, 17 Dec 2002)

I think we need to realize that one aspect of "Web Services" is that it's a document-oriented message exchange. At the end of the day RPC is just a concept that sits on top of that. Ofcourse, we need RPC as much as we need some sort of WSDL (not liking it's current incarnation though) to enforce and define contracts by which parties can invoke each other, just like an xml schema could be the contract for document exchange between parties. We're essentially trying to solve two things, namely document exchange and rpc function calls (which use document exchange under the hood). From reading the proposed standards (regarding Web Services), I sometimes get the feeling most people jumped on the bandwagon to soon. We have not yet fully figured out how the two (doc & rpc) fit together nor how they actually relate. Or maybe we have, but haven't found the right way to describe all that. So naturally we're going through some growing pains. This might sound strange, but HTTP+XML isn't very helpfull as it allows you to cranck out your own standards pretty easily. Many people are not satisfied with the standards work that has been done upon till now, and are trying to do something about it (which they are entitled to ofcourse). Hopefully, we'll be laughing at these kind of discussions in a couple of years ...

Re: SOAP Web Services: built on a contradiction? (John R. Daily - 15:03, 17 Dec 2002)

It's not hard to tell the losing side in any debate: just look for personal vitriol, knee-jerk overreaction to assertions that aren't even being made, and bogus statistics.

I don't know anything about the REST vs. SOAP discussion, but the SOAP advocates have convinced me to take a look at REST.

i think his point was that the SOAP architecture is bloated and solutions are overly-hyped. if the point of web services is for loosely coupled applications, created with best-of-breed tools, to communicate via http then the web-services "solution" has become a problem.

it's really becoming a problem microsoft getting involved with standards. they are a big force and whatever they endorse becomes the latest buzz phrase. van der Vlist is right, many companies have staked the bank on soap/uddi/wsdl and they _need_ it to happen. it's going to be a financial disaster if it all comes crumbling down. so they are shoving this crap down clients throats.

what was wrong with xml-rpc again? what was the point of using http? what was the vision for ubiquitous computing? you microsofties ... so easily led.

The REST crowd is wasting their time. SOAP-based Web Services are created using technologies and standards created for a Web Service environment. REST is nice for the Web but forget about it when it comes to Web Services. The SOAP crowd enjoys having all the benefits of SOAP messaging and RPC in one location.

I thought REST for Web Services would die out after every major vendor released SOAP based products. REST is really nice and I too am a big fan of "the right tool for the job" but at the same time I am not going to make things hard on myself and learn something no one will support. You need to plan ahead and realize that business partners move with the crowd and right now the crowd is following SOAP.

If you want to use REST that's up to you, but try convincing your business partners to switch from SOAP and see what happens. A convincing argument will not make them throw away all the investments they put in SOAP to try something that you tell them works better for some things.

The bottom line is that the right tool for the job is the tool that will attract the most customers or get you the most money. From both the service creator and the service consumer points-of-view, SOAP is widely supported across platforms, languages, and applications. The majority of people know the Web's architecture but don't even know what REST is. It's too hard to push an architecture for Web Services that developers and businesses were not considering as an option.

There is a SOAP vs. REST links page on my site: http://www.myspotter.com/soap_rest.shtml for those interested.

> 90% people I know use SOAP and like it. 8% don't > know what it us. 2% have long beards and are REST > fanatics.

Amen. Add to that the fact that these goddamn 2-percenter REST fanatics are matematically twice as dangerous as the so-called 1-percenter motorcycle "clubs" (Hells Angels and the like), and it's clear we potentially have got a serious public safety problem on our hands.

Microsoft: act swiftly -- I recommend you launch a media blitz to educate the public about this risk, and suggest you choose Pat Boone as your spokesman.

With any luck, it will be at least as successful as the campaign you launched to educate the public and government about the serious threat to the American Way of Life posed by the GNU General Public License and open-source software. (btw, choosing Bizarro Superman as the spokesman for that campaign was brilliant.)

I am certainly not a REST folk and not fanatic of anything (even of XML) except of using the right tool for the right purpose.

Do you remember another great idea that came up a while ago and was named "XML data islands"? XML data islands were an attempt to highjack HTML to get the benefits of XML. Nice demos have been made but their generalization has been a failure because you got the downsides of both HTML and XML without really getting their benefits.

IMO, SOAP is to HTTP what XML islands were to HTML, ie a way to highjack HTTP to get the benefets of XML which gets you the downsides of both HTTP and of deploying a new protocol without getting the benefits.

REST is one of the alternatives and is wonderful for exchanges when HTTP is the right choice. When HTTP isn't the right choice I would certainly not recommend HTTP: instead of emulating and reinventing features already implemented elsewhere why not use anything but HTTP?

CORBA and COM are still alive, more widely implemented than most think and can easily be used for RPC.

At the protocol layer, if you need features which are not included in HTTP you'd better use BEEP which is to HTTP what XML is to HTML. That's the right tool to do the job and eventually it might do to SOAP what XML has done to XML islands.

Of course, you can use SOAP over BEEP, but what would be the point? What would be the point of using XML islands within XML? A lot of SOAP complexity is about reinventing missing HTTP features and a XML application layer designed to sit on top of BEEP would be much simpler, flexible and powerful than SOAP.

I have learned not to trust people who say that there is only one way of doing things. Don't trust people who say that HTTP, SOAP, REST, CORBA, COM, BEEP or XML is the only way but please choose the best tool to do the job. Like XML data island, SOAP is a terrible hack to use a hammer to pin a screw...

For some reason you REST folks just don't get it, and live in a world of chatter on newsgroup, and not in practical places of work.

Basically RESY is there, and will always be there. But for us folks who want to do much more than REST, let us use SOAP. It may not be as pure as REST but by god it is practical and solves a lot of issues. It opens up interoperability and all sorts of extensions.

Basically, my word to you is shut up, stop putting people off SOAP, and get a life.

90% people I know use SOAP and like it. 8% don't know what it us. 2% have long beards and are REST fanatics.

At my company we are using a REST-based interface to allow the import and export of data to and from our trading system. We considered using SOAP, but felt that it would create an unnecessary extra layer of complexity without offering any benefits over a REST architecture. In fact SOAP's RPC approach seems to have many short comings versus the Representational State Transfer approach.

We use HTTPS as the transport protocol, with unique descriptive URLs representing each data resource in the system. We are using HTTP commands in a way that is cogizant with their protocol semantics, i.e. GET is used to retrieve the XML representation of a resource, PUT is used to submit a new resource to the system. A nice feature of the semantically correct usuage of HTTP commands is the ability to do informal testing using a web browser.

We marshal and unmarshal XML straight from our Java object model using Castor, an open source XML binding tool, (http://castor.exolab.org/). Currently we are not using schemas or DTDs, instead processes on either side of the interface are responsible for ensuring the validity of the data.

In client integration work the biggest issue we have found is the reconciliation of differences between our system's data model and the client data model. This issue has nothing to do with the choice of interface architecture, but exemplifies a key aspect of an integration project: how to interpret data outputed from another process in a way that creates meaningful input for my process. In our experience making the XML as visible as possible makes this task easier.

Re: SOAP Web Services: built on a contradiction? (Terris Linenbach - 17:57, 17 Nov 2002)

The tools producers (IBM, Microsoft, BEA, Apache) are stuck with the majority of their customers who are Corba, RMI, and COM developers who already understand RPC very well. Since RPC hides transport details which is, by far, its most popular feature, the early architects of the SOAP toolkits thought it would be in their customers' best interests to completely hide the fact that XML was being used at all.

However, the most successful uses of XML seem to involve the direct generation, transport, and manipulation of XML documents, possibly for the following reasons:

1. Large documents can only be parsed via stream-based paradigms because RAM is still in short supply, and that won't ever change -- this is why databases exist in the first place.

2. Documents are a good fit for loosely coupled and asynchronous business processes. RPC paradigms fall over in many asynchronous scenarios; even "publish and subscribe" paradigms tend to fail across business boundaries.

3. Writing HTTP code is easier than using WSDL (even with the various SOAP toolkits)

Doc-Literal and WS-Attachments (or the forthcoming SOAP attachments) are promising steps in SOAP's growth towards the real XML universe. Every major player at Microsoft, BEA, and IBM that I have spoken with agree that the document paradigm is more powerful and interesting than the RPC paradigm. However, their massive customer base probably won't be able to make the shift.

SAX and XML Schema are too hard to deal with. It's still too hard to create XML documents from data sources. And, most damning of all, the various toolkits and IDEs make it almost impossible to get at the raw XML messages. WS-Attachments can be used as a solution by using the SOAP message for routing/security and using attachments for the actual data which can be parsed via DOM, SAX, or however the developer _chooses_.

The recent "XML Writer" components in both Java and .NET have made creating XML documents a lot simpler than pasting strings together. Pull parsers have made SAX processing incrementally easier, but it's still too difficult.

The organization that brings REST to the masses under the SOAP marketing machine will be a hero in my opinion. I don't think it's that far off. The first thing that needs to happen is widespread understanding of XML Schema which many xml-deviants are skeptical of. We also need decent and cheap XML Schema aware tools for visualization, editing, and validation of both schema and instances.