Write Once, Serve Everyone

Web services promise to integrate businesses, connect all kinds of devices, and rally the computer industry around standards. You've heard the hype -- here's what the hubbub's all about.

The bread and butter of many companies is service: offering a unique and valuable expertise or resource to others. Indeed, in the past six or seven years, the Internet has enabled many traditional and new service companies to reap even greater profits from their know-how. Examples are everywhere: credit card authorization (through services like PayPal and c2it) is now available to any one; online auctions linking buyer and seller are commonplace; any bank worth its salt offers Web-based banking to its customers; and online travel brokers and “e-tickets” have literally recasted an entire industry — much to the dismay of your local travel agent.

With widespread Internet access, those services remained much the same, but reach expanded. Availability brought users, users drove usage, and usage, of course, drove profit. (Yes, there were many failures, but just look at eBay’s revenues and profits as an example of what can be done.) Usage has also driven change. Like the saying, “A rising tide lifts all boats,” Internet usage and adoption has forced all of us to think and act and conduct business differently. Even if your product is “tangible” (a car, a piece of software, consumer credit, or genomic data), the phenomena of 24/7, ubiquitous, on-demand service has likely changed your business, too. Or is. Or will — no doubt about that. There’s no going back to the days before the Internet boom. In fact, more and more devices get connected to the Internet every day. Connectivity is becoming pervasive, and devices of all stripes are becoming more interconnected.

If you don’t believe in a more interconnected Internet, you just might be a heretic. Certainly HP, Microsoft, IBM, and Sun are believers. In fact, those companies are betting on a vastly interconnected Internet. Their new religion is Web services. Web services promise to provide information, transactions, and computation to any client — a remote server, a desktop application, a web page, an intelligent agent, or a wireless transceiver — from any server.

Web services can let you provide your knowledge and expertise simply, directly, and globally while avoiding the costs of building special intercommunication infrastructure (specialized servers, specialized protocol, specialized business rules) for every client you acquire. In other words, “Write Once, Serve Everyone.”

Web Services…At Your Service

Yeah, you’ve probably heard that hype before. So, what are Web services really? In a nutshell, Web services are simply a new kind of Web application. Web services provide electronic and programmatic access to your business, whatever your business may be. What makes Web services superior to other Web applications is standards. Web services use standard Internet protocols and data formats (which we’ll examine in detail) for all intercommunication. As we will see, Web services are based on four core platform-independent technologies that provide data representation, intercommunication, description, and discovery.

XML and a protocol called SOAP (Simple Object Access Protocol, described below) provide data representation and intercommunication, respectively. Taken together, XML and SOAP is the universal dialect of Web services. WSDL (Web Services Description Language) is another XML-based syntax that describes the names and properties of a Web service. WSDL describes how to call a Web service and how to interpret the results. Finally, UDDI (Universal Description, Discovery, and Integration) provides a “yellow pages”-like directory for Web services. With UDDI, an application can dynamically and programmatically discover available Web services.

Because all Web services “speak the same language”, Web services are infinitely interoperable. Any Web service can interact with any other Web service. In fact, because Web services are also described and published in a standardized way, they can be located and invoked on-the-fly across the Web.

That’s quite a series of claims. Let’s examine each one a little more closely, look at SOAP, WSDL, and UDDI, and ultimately learn why Sun, Microsoft, and IBM are touting Web services as the next big thing. (To learn how to build a Web service with Linux, Apache, and Java, see the accompanying feature “Building a Web Service with Apache AXIS” on page 30.)

SOAP: A Clean, Refreshing Standard

As mentioned above, any Web service can interact with any other Web service. To clarify, it’s probably more correct to say that any device that supports TCP/IP, HTTP (or FTP or SMTP), and SOAP can both serve and access Web services. While Web services can be implemented over an arbitrary protocol, the combination of TCP/IP and HTTP is superior for one simple reason: ubiquity. The Web is already built on it, it’s firewall-friendly, and all devices speak it. Since TCP/IP and HTTP are universally available, Web services simply re-use that existing infrastructure for all intercommunication.

But TCP/IP and HTTP just provide a connection and an agreed-upon way to exchange information. The conversation — the actual format and content of all Web service requests and responses — is expressed in SOAP. SOAP allows one device to invoke procedures advertised by another device.

To be sure, remote procedure calls are nothing new. So, why SOAP?


  • SOAP is an XML-based protocol. XML, the lingua franca of Internet data exchange, is ideal for communication between heterogenous partners. XML clearly separates content from presentation, and because it is highly structured, XML allows data to be manipulated reliably and automatically. (XML is readily parsed by any number of programming languages.) XML is also text-based, making it much easier to debug and pass through firewalls. (For more information on how XML documents are structured, please see our July article, located on the Web at http://www.linux-mag.com/2001-07/xml_basics_01.html, and for information on parsing XML, check out our October 2001 article at http://www.linux-mag.com/2001-10/xmldom_01.html.)

  • SOAP is a universally-accepted W3C (World Wide Web Consortium) standard. Other remote procedure call technologies are proprietary or are not supported across a wide variety of operating systems or hardware platforms. SOAP, however, is vendor agnostic, language independent, and unencumbered by operating system peculiarities.

All of the major vendors are supporting SOAP. Indeed, the entire Microsoft .NET platform is a platform for deploying Web services. Whether you write in C# or Visual Basic or C++, .NET makes it very easy for you to deploy your components as Web services. And because of SOAP, your Perl script running on Linux won’t have any problem talking to one of Redmond’s finest.

Simple SOAP examples have appeared in earlier issues of Linux Magazine. Our October 2001 article, “Cleaning Up With SOAP”, located on the Web at http://www.linuxmagazine.com/2001-10/soap_01.html, contains a rudimentary example of using SOAP to access a stock quote Web service. Rather than create more arbitrary examples here, let’s take a little field trip over to Google (http://www.google.com) and see how the Google search engine can be accessed via SOAP.

Another Side of Google, Please

Recently, the Google index of more than two billion web pages was made available to application developers as a Web service. Using the Google SOAP API, you can now embed Web searches and spell-checking (among other features) into any Internet application. All requests and responses take the form of SOAP messages. Everything else — the user interface of how to conduct searches and display results — is strictly up to you.

For our example, we’ll write a Perl client that connects to Google and runs a search (if you don’t use Perl, you can find a list of Google implementations in virtually every programming language at http://www.soapware.org/directory/4/services). If you’d like to play with this example on your own Linux machine, go to the Perl CPAN (http://www.cpan.org) and download and install the SOAP::Lite Version 0.52 (or later) module (and any of its prerequisites). You’ll also need to go to Google (http://www.google.com/apis) and follow their instructions to download their SDK and register for a Google developer’s account. Setup should take less than ten minutes.

As mentioned above, to call a Web service, we need to specify the server, the service (a single server can host many Web services), and the method (the remote procedure) we want to call. If the method requires parameters, we need to pass those along, too. Listing One shows a Perl script that calls Google’s GoogleSearchAction to perform a doGoogleSearch on whatever is provided on the command line.




Listing One: Calling the Google Web service from Perl


1 #! /usr/bin/perl
2
3 use SOAP::Lite;
4
5 my $key = “0000000000000000000000000000000″; # <– PUT YOUR KEY HERE
6 my $query = join(” “, @ARGV);
7
8 my $result = SOAP::Lite
9 -> proxy(‘http://api.google.com/search/beta2‘)
10 -> uri(‘urn:GoogleSearch’)
11 -> on_action(sub{‘urn:GoogleSearchAction’})
12 -> call(‘doGoogleSearch’ =>
13 SOAP::Data->new(name => ‘key’, type => ‘xsd:string’, attr => {}, value => $key),
14 SOAP::Data->new(name => ‘q’, type => ‘xsd:string’, attr => {}, value => $query),
15 SOAP::Data->new(name => ‘start’, type => ‘xsd:int’, attr => {}, value => 0),
16 SOAP::Data->new(name => ‘maxResults’, type => ‘xsd:int’, attr => {}, value => 1),
17 SOAP::Data->new(name => ‘filter’, type => ‘xsd:boolean’, attr => {}, value => ‘false’),
18 SOAP::Data->new(name => ‘restrict’, type => ‘xsd:string’, attr => {}, value => ”),
19 SOAP::Data->new(name => ‘safeSearch’, type => ‘xsd:boolean’, attr => {}, value => ‘false’),
20 SOAP::Data->new(name => ‘lr’, type => ‘xsd:string’, attr => {}, value => ”),
21 SOAP::Data->new(name => ‘ie’, type => ‘xsd:string’, attr => {}, value => ‘latin1′),
22 SOAP::Data->new(name => ‘oe’, type => ‘xsd:string’, attr => {}, value => ‘latin1′),)
23 -> result
24 ;
25
26 if (defined($result->{resultElements})) {
27 print join “\n”,
28 “Found:”,
29 $result->{resultElements}->[0]->{title},
30 $result->{resultElements}->[0]->{URL} . “\n”
31 };

Save the code in Listing One to a file called rawsoap.pl. Before running the code, make sure to replace the value of $key with the key sent to you by Google (a key of all zeroes will not be autheticated). Running the command rawsoap.pl linux magazine produces the following output.


Found:
Linux Magazine: Open Source. Open Standards.
http://www.linux-mag.com/

If you want to see what a SOAP request looks like in detail, see “A Sidebar of SOAP”.




A Sidebar of SOAP

Every SOAP message is an XML document, and all SOAP messages have the same structure. All SOAP messages are sent in an envelope; the contents of the envelope are a header and a single body.

The SOAP header can contain zero or more header entries that can extend the message format without coupling them to the body or changing the well-established structure of SOAP messages. For example, header entries can be used for billing, encryption, authentication, auditing, and so on.

The body of the SOAP message is the actual payload, the application-specific information. Every SOAP message has one body. The body, in turn, also contains a single element, the name of the remote procedure to call. The elements contained within the procedure call are its parameters. The names of the parameters are significant, but the order is not. Each parameter has an xsi:type attribute that indicates type.

For example, the HTTP headers and the SOAP request generated by the code in Listing One is shown below in Figure Two. The URI in the POST header helps the server route the incoming message (in case the server is handling other requests other than SOAP messages). Similarly, the SOAPAction header is also a kind of hint; it is entirely up to the server to interpret SOAPAction and respond. Finally, notice that the proxy is not contained in the SOAP message, but was used in the HTTP header.




Figure Two: A SOAP message being sent to the Google search Web service


POST http://api.google.com/search/beta2
Accept: text/xml
Accept: multipart/*
Content-Length: 894
Content-Type: text/xml; charset=utf-8
SOAPAction: urn:GoogleSearchAction

<?xml version=’1.0′ encoding=’UTF-8′?>
<SOAP-ENV:Envelope
xmlns:SOAP-ENV=”http://schemas.xmlsoap.org/soap/envelope/
xmlns:xsi=”http://www.w3.org/1999/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/1999/XMLSchema“>
<SOAP-ENV:Body>
<ns1:doGoogleSearch xmlns:ns1=”urn:GoogleSearch”
SOAP-ENV:encodingStyle=”http://schemas.xmlsoap.org/soap/encoding/“>
<key xsi:type=”xsd:string”>00000000000000000000000000000000</key>
<q xsi:type=”xsd:string”>linux magazine</q>
<start xsi:type=”xsd:int”>0</start>
<maxResults xsi:type=”xsd:int”>10</maxResults>
<filter xsi:type=”xsd:boolean”>true</filter>
<restrict xsi:type=”xsd:string”></restrict>
<safeSearch xsi:type=”xsd:boolean”>false</safeSearch>
<lr xsi:type=”xsd:string”></lr>
<ie xsi:type=”xsd:string”>latin1</ie>
<oe xsi:type=”xsd:string”>latin1</oe>
</ns1:doGoogleSearch>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

SOAP-ENV:Envelope is the SOAP envelope element. The namespace of the envelope must be http://schemas.xmlsoap.org/soap/envelope, or the SOAP message will be rejected by all Web services. SOAP-ENV:Body is the body of the SOAP message. Notice that the SOAP body contains one element, doGoogleSearch, the name of the procedure we’re calling. Its elements are constructed from the names and values specified in lines 13-22 in Listing One. (The string of zeroes in the key element will be replaced with your Google developer key.)

While the Google SOAP request is fairly terse, the response is quite lengthy (and is omitted due to space constraints.) The response includes some special HTTP headers, and an envelope and body just like the request. In the case of Google, the response is especially verbose because it defines all of the data types that the Google service can return. As we’ve seen, SOAP messages are self-describing. The Google SOAP client will receive enough information to interpret and reconstruct arbitrarily complex return values.

If you want to see the complete SOAP response, modify the sample code in Listing One and add the line


-> on_debug(sub{print@_})

immediately after line 10. Once you turn on debugging, the HTTP headers and the SOAP messages will be printed to stdout.

Hey, What’s your Sign?

With XML and SOAP in place, any application can talk to any service. Indeed, after reading about SOAP you might imagine either end of a Web services connection as a “black box”, with SOAP acting as a universal API.

Certainly, SOAP simplifies how two devices talk to each other, but there is still a need to coordinate what the devices “say” to each other. After all, a Web service is useless if used incorrectly (just like any other API).

Specificity is always going to be needed: you can’t call an API without knowing the names of its methods and the number and types of arguments of each entry point. However, if the Web is going to be widely populated with Web services, with many similar Web services competing for service requests, it is unrealistic, likely intractable, to code specifically for each Web service.

(As an example, glance back at the Perl code in Listing One. To call the Google service, we had to specify a lot of parameters. In some cases, such as key and maxResults, we even had to specify the type of the argument: the type attributes were xsd:string and xsd:int, respectively. If any of those parameters are incorrect, the call to the Web service fails.)

While SOAP and XML have certainly separated implementation (platform, operating system, programming language) from operation, Web services aren’t black boxes — not yet. What’s needed is self-description.The capabilities and interfaces of each Web service must be described in a standardized way. Moreover, each Web service must describe itself in a format that any application can understand.

WSDL (Web Service Description Language) is the XML-based syntax that provides description. As SOAP is an agnostic messaging platform, WSDL is an agnostic interface platform. Based on XML again, WSDL describes a service’s methods and messages in an abstract way — independent of message formats and network protocols.

Figure One shows a portion of the WSDL file for Google’s Web search service (the entire WSDL file can be downloaded from Google). Notice the element <message name=”doGoogleSearch”>. It contains elements whose names should look very familiar — it defines how to construct a message to call the doGoogleSearch method we used in the Perl code in Listing One.




Figure One: A snippet of Google’s WSDL file


<!– Types for search – result elements, directory categories –>
<types>
<xsd:schema xmlns=”http://www.w3.org/2001/XMLSchema
targetNamespace=”urn:GoogleSearch”>
<xsd:complexType name=”ResultElement”>
<xsd:all>
<xsd:element name=”summary” type=”xsd:string”/>
<xsd:element name=”URL” type=”xsd:string”/>
<xsd:element name=”snippet” type=”xsd:string”/>
<xsd:element name=”title” type=”xsd:string”/>
<xsd:element name=”cachedSize” type=”xsd:string”/>
<xsd:element name=”relatedInformationPresent” type=”xsd:boolean”/>
<xsd:element name=”hostName” type=”xsd:string”/>
<xsd:element name=”directoryCategory” type=”typens:DirectoryCategory”/>
<xsd:element name=”directoryTitle” type=”xsd:string”/>
</xsd:all>
</xsd:complexType>
</types>

<!– Messages for Google Web APIs – cached page, search, spelling. –>
<message name=”doGoogleSearch”>
<part name=”key” type=”xsd:string”/>
<part name=”q” type=”xsd:string”/>
<part name=”start” type=”xsd:int”/>
<part name=”maxResults” type=”xsd:int”/>
<part name=”filter” type=”xsd:boolean”/>
<part name=”restrict” type=”xsd:string”/>
<part name=”safeSearch” type=”xsd:boolean”/>
<part name=”lr” type=”xsd:string”/>
<part name=”ie” type=”xsd:string”/>
<part name=”oe” type=”xsd:string”/>
</message>

<!– Port for Google Web APIs, “GoogleSearch” –>
<portType name=”GoogleSearchPort”>
<operation name=”doGoogleSearch”>
<input message=”typens:doGoogleSearch”/>
<output message=”typens:doGoogleSearchResponse”/>
</operation>
</portType>

<!– Endpoint for Google Web APIs –>
<service name=”GoogleSearchService”>
<port name=”GoogleSearchPort” binding=”typens:GoogleSearchBinding”>
<soap:address location=”http://api.google.com/search/beta2“/>
</port>
</service>

Briefly, each WSDL file is divided into five sections: types, messages, port types, services, and bindings. You can find examples of the first four sections in Figure One.

TYPES: If your Web service uses its own data types, you must define those data types in the <types> element. Complex types in the WSDL file are composed of XML Schema Instance primitive data types like xsd:string and xsd: boolean. For example, look at the definition for ResultElement in Figure One. It is a complex data type (hence the xsd:complexType) composed of nine primitive types. You can think of ResultElement as a C struct. If the Google search service finds any Web pages, each matching item will be a ResultElement record. If you look at Listing Two, you’ll see that we refer to elements title and url. (More background on XML schemas can be found in our February 2001 article, “XML Schema Languages”, located online at http://www.linuxmagazine.com/2002-02/xml_01.html.)

MESSAGES: This section specifies the API of the Web service. Here, you’ll find a list of methods that each service provides, and a list of arguments (with types) for each method.

PORTTYPES: PortTypes tie services to messages. For example, in Figure One, the method doGoogleSearch uses one kind of message, doGoogleSearch for requests (input), and responds with another kind of message, doGoogleSearchResponse (omitted due to space constraints) as output.

BINDINGS: This section describes how to call each method defined in the portTypes section.

SERVICES: Services tie services and ports to specific URLs. (For a complete description of WSDL files, you can read the WSDL specification at http://www.w3.org/TR/wsdl.)




Listing Two: A greatly simplified Perl client using the Google WSDL file


#!/usr/bin/perl -w
use strict;
use SOAP::Lite;
use XML::Parser::Expat;

my $prsr = new XML::Parser::Expat;
my $key = “00000000000000000000000000000000″; # <– PUT YOUR KEY HERE
my $query = $prsr -> xml_escape(join(” “, @ARGV) || “google api”);

my $service = SOAP::Lite
-> service(‘file:./GoogleSearch.wsdl’);

my $result = $service -> doGoogleSearch( $key, $query, 0, 1, “false”, “”, “false”, “”, “latin1″, “latin1″);

# $result is hash of the return structure.
# Each result is an element in the array keyed by ‘resultElements’.
# See the WSDL for more details.
if(defined($result->{resultElements})) {
print join “\n”,
“Found:”,
$result->{resultElements}->[0]->{title},
$result->{resultElements}->[0]->{URL} . “\n”
}

As you can see, a WSDL file contains all of the information necessary to contact a Web service dynamically. The WSDL file exactly specifies what URL to contact, how to communicate with the service, what messages to send and what messages to receive, how to construct a message, and how to form the service-specific data types.

WSDL files can be read and processed programmatically — effectively making Web services little black boxes. For example, the Perl code in Listing Two performs the same work as Listing One, but uses the Google WSDL file to craft the call to the Google service automatically. WSDL files are contracts, and both parties to the “contract” (the client and the server) must honor its terms reciprocally.

You can also think of a WSDL file as a public specification of the service, and build code to that specification. In fact, a number of code generators do just that: given a published WSDL file, the code generators automatically create stub code for your server or client application. Indeed, the code in Listing One was derived from Perl’s stubmaker.pl. (You can see the AXIS Java code generator, WSDL2Java, in the “Creating Web Services with AXIS” article on pg. 30. Code generators even exist to create Enterprise Java Bean server skeletons based on a WSDL file.)

PC Seeks WS for Meaningful Relationship

So far, we’ve learned that XML provides data representation, SOAP provides intercommunication, and WSDL provides description. The last core technology to discuss is UDDI, Universal Discovery, Description, and Integration. As its name implies, UDDI provides discovery, a dynamic process to find Web services that meet certain criteria.

If WSDL is a “yellow pages” advertisement, then UDDI is the Yellow Pages. WSDL syntax describes all of the services a service provider supplies; UDDI syntax accumulates the names of service providers and descriptions of each service into a registry. A special service provider called a service broker hosts the registry and provides all of the processing power and know-how to match a service requestor (like your Web server or your cell phone) with a service provider.

Service providers and service requestors can be any kind of device or application. XML, SOAP, WSDL, and UDDI render the issues of platform and operating system choice moot.

While CORBA, COM, and EJB may underlie Web services, each of those proprietary solutions already or will soon offer “wrappers” that provide XML-, SOAP-, WSDL-, and UDDI-based equivalent interfaces. In time, XML, SOAP, WSDL, and UDDI will be favored over proprietary solutions.

The Web is Your Oyster

Web services offer great opportunity to all of the Internet’s constituencies. Service providers — think anyone with content or a unique expertise — can “write once, serve everyone” on the Internet. Service brokers — think of a virtual eBay or a pervasive Priceline — can connect service providers to service requestors, adding value for both consumer and producer. Service requestors can be just about anything — a cell phone, a bioinformatics search engine, an automated inventory system, even an old-fashioned Web browser.

As bandwidth increases, as newer and friendlier devices emerge, as information becomes more varied, dynamic, and important, Web services are an ideal platform for moving the Web further into the mainstream and into the “back office” (where e-business really occurs).

In a sense, the Web services universe just had its “Big Bang”, and developers and businesses are beginning to take notice. Developers are playing with code, building better tools, and deploying the first commercially-available Web services. Businesses are are “kicking the tires” and making their initial investments. “Web services” is an official industry buzzword — with Sun, IBM, and Microsoft (and others) generating the buzz.

In these early days of Web services, you’d be wise to stay informed and keep track of emerging trends. Look for standards that are embraced widely by industry; stay focused on technologies that have substantial user bases, technical support, and financial backing; try to seize upon frameworks that let you focus on your business, not the business of creating or re-creating reference implementations. Talk to the vendors and make informed decisions.

And while it’s too early to predict which technology provider will prevail — Sun, Microsoft, IBM, or Open Source — one thing’s for sure: the Internet is evolving again.




Resources

Axis http://xml.apache.org

ebXML http://www.ebxml.org

Google Web Service http://www.google.com/apis

IBM Developer Works http://www.ibm.com/developerworks/webservices

Microsoft .NET http://www.microsoft.com/net

Perl SOAP::Lite http://www.soaplite.com, http://www.linuxmagazine.com/2001-10/soap_01.html

Sun ONE http://www.sun.com/sunone

The UDDI Community http://www.uddi.org

W3C Web Services Specifications http://www.w3c.org/TR/SOAP, http://www.w3c.org/TR/wsdl

Web Services Architect www.webservicesarchitect.com

XMethods (a list of publically-available Web services) http://www.xmethods.com



Martin Streicher is an Executive Editor at Linux Magazine. He also owns and operates a family of four. He can be reached at mstreicher@linux-mag.com.

Fatal error: Call to undefined function aa_author_bios() in /opt/apache/dms/b2b/linux-mag.com/site/www/htdocs/wp-content/themes/linuxmag/single.php on line 62