Imagine all of the processing power within your enterprise - from every large and small server and cluster in every datacenter, to every networked personal computer - all available to work on solving the day's business problems. That's the notion of an enterprise grid, and if the Enterprise Grid Alliance (EGA) fulfills its mission, a company-wide computing farm will be a reality.
Imagine that in addition to all of the information of the World Wide Web, your Internet connection gave you the sum of all the computing power that was simultaneously connected. After all, at any given moment, some large percentage of the world’s computers is simply sitting idle. Because of grid computing, this isn’t such a far-fetched idea, even though the realities of security and privacy certainly make it impractical or at least a distant wish.
So instead, imagine all of the processing power within your enterprise — from every large and small server and cluster in every datacenter, to every networked personal computer on desktops in every office worldwide, all carefully protected behind your precisely configured firewall — all available to solve the day’s business problems. If the Enterprise Grid Alliance (EGA) fulfills its mission, such a company-wide computing farm won’t be such an impractical idea.
You might ask, “Why would anyone want to do this?” It’s unlikely that the average desktop user cares about harvesting extra cycles from a remote computer to help read email, but for resource-hungry enterprise applications, the ability to get more done with less has tremendous potential and appeal. Does a day go by when some datacenter manager isn’t asked to cut the budget while providing better performance?
Grid computing, the idea that computing services can be made as available and ubiquitous as any utility, is not a new concept. As early as 1969, when the first node of the ARPANet, the predecessor to the current Internet, was announced, a young UCLA engineering professor, Len Kleinrock, stated, “We will probably see the spread of ‘computer utilities,’ which, like present electric and telephone utilities, will service individual homes and offices across the country.” Kleinrock is credited as the father of packet switching, the basis of modern computer networks and the backbone of any grid.
So if the potential for a grid of shared computing resources has been recognized for over thirty years, what’s been done about it? Certainly research scientists and academics have created successful supercomputing grids for compute-intensive tasks such as bioengineering and molecular dynamics. And in what is probably the best known example of a scavenging grid (a grid that scavenges the network looking for idle CPUs from which to harvest cycles), SETI@home, the search for signals from extraterrestrial life, has been using the idle cycles from volunteer PCs to crunch terabytes of raw data at essentially no cost.
Grid for Mainstream Business: The EGA at Work
As with most new technologies, the pioneers are those who can spend significant time keeping a novel system running. But grid has matured and is moving out of the labs and into corporate settings. In a sector dominated by open source software, there are now commercial tools available to turn disparate resources into coherent grids, and robust software to take care of the necessary provisioning and monitoring, and even to help grid-enable existing applications. Additionally, the formation of the Enterprise Grid Alliance is proof that businesses are interested. “The founding members of the consortium all have products in this space and customers who are leading-edge adopters,” said Donald Deutsch, the first president of the Enterprise Grid Alliance and Vice President for standards, strategy, and architecture at Oracle Corp. “We all see grid computing within the enterprise as a clear need, but it’s currently useful only to the most technologically involved. What we have to do to push this down to the next level within the company.”
The job of the EGA, if you will, is to “grow the market,” and the only way to do that is to standardize grid computing and turn it into a genuine utility. More specifically, Deutsch expects the EGA to start showing results within twelve to eighteen months of their founding, which was announced at the end of April 2004. There are currently twenty founding members and plans to form a variety of working groups to define a reference model that includes component provisioning, data provisioning, utility accounting, and grid security. Even basic elements need to be codified, such as ontology and frameworks for a common set of terms and architectural components. Finally, there needs to be common programming interfaces (APIs) to raise the level of interoperability for most common activities for grids.
The Grid Today
As Peter ffoulkes, group marketing manager for high-performance technical computing (HPTC) at Sun Microsystems and a member of the EGA planning group, points out, “Technical problems of simple grids have been solved. Up to 500 nodes is pretty much commodity stuff these days. Multi-department grids have also been solved.”
Sun itself is marketing its N1 Grid Initiative (“n computers serving as 1″). At the same time, other big name software vendors are also actively supplying grid solutions: HP offers its Adaptive Enterprise; Oracle’s latest database offering is called 10g, with the “g”‘ meaning grid-enabled; and IBM (which is not yet a member of the EGA) features its Grid Toolbox. There’s serious money being spent to develop and market more comprehensive solutions, but most solutions aren’t yet aimed directly at enterprise customers.
ffoulkes further notes that much remains to be done. “Global grid with security is not solved. Security and accounting in commercial grids are not solved. Standards are not quite mature enough and haven’t been tested at a level of scaling for enterprise apps. Moving to the nirvana of the network as the computer (Sun’s particular mantra), is still on the horizon,” said ffoulkes. “You still don’t have what you really need to run a bank and bet the business on it.”
Much of what is available commercially is based on open source software from the Globus Alliance. Based at the Argonne National Laboratory, home of one of the early supercomputing centers, most of the affiliates are universities and research institutions. It’s a high-powered group that has taken a leadership role in grid research, tools, and applications, but they aren’t focused on problems of the enterprise. The Globus Toolkit, which has matured to version 3 and is considered the de facto industry standard, includes software services and libraries for resource monitoring, discovery, and management, plus security and file management.
For its “adaptive enterprise” customers, HP adds additional management software to the Globus Toolkit. “We’re very involved in web services,” said Sara Murphy, HP’s Marketing Manager for grid computing in high-performance technical computing. “Grid is moving into a web services model. A couple of years ago there was a real shift in grid standards to merge with web services. Globus is moving that way, and service-oriented architectures provide the underpinnings on which enterprise grids can be built.”
Where to Begin?
There’s general agreement, at least among the current members of the EGA and the analysts they work with, that grid computing is enormously valuable, but not a panacea. “It doesn’t just snap together overnight, but there are real economic savings in sharing resources,” said Murphy. “There’s a risk in terms of being over-hyped and creating unrealistic expectations.”
While the potential for grid in the enterprise is tremendous, it’s not yet a “bet your business” technology. “Initially, our research finds customers thinking about grid in non-mission critical areas, like supply chain, ERP, or data warehouse,” said John Humphries, IDC Research Manager of high performance and commercial systems. “Many grids today are used by industrial or high-performance customers, in applications such as financial services for things like derivatives, risk assessment, complex large batch-oriented applications. People are talking about it, but it’s not there yet.”
Humphries describes the big difference in where grids are now and where they need to be to support enterprise computing as simply the difference between a large, batch job orientation and a transactional orientation. “You’re pooling resources in big chunks in the HPC world, and provisioning for smaller chunks, like transactions, is an added complexity. Boeing, Ford, Morgan Stanley, Exxon/Mobil all have HPC divisions that can use grids. But in the supply chain, ERP and CRM applications need to be grid-enabled to create a successful enterprise grid.”
Despite the obvious difficulties, IT leaders find the promise of grid computing irresistible. John Anthony, architect in the P&C Enterprise Development, Strategic, Planning and Architecture unit of The Hartford Financial Services Group, Inc., started out by looking at disaster recovery and the demands of high availability. “Organizations tend to purchase large, name-brand servers that are sometimes underutilized due to the behavioral characteristics of software [that is to say, that many applications are I/O bound rather than CPU bound], and the problem is often exacerbated by operational processes such as reserving capacity for disaster recovery” said Anthony.
So a company might plan to use only 45 percent of the capacity of an expensive resource so that in case of a disaster, the entire load could be moved to an identical resource elsewhere. The result is an acceptable 90 percent peak load. Not very efficient, but safe.
The Hartford is evaluating the potential benefits of grid technology and its impact on the business. As grid technologies mature, Anthony believes that more organizations will assess and adopt grid solutions to solve specific needs. “The first deliverable is to provide a position for the enterprise with respect to grid computing. The second is to identify first candidates for benefits.” Underwriting and actuarial applications are likely areas.
Chicken or Egg?
It would be nice if all you had to do to improve computing performance and efficiency was to interconnect a bunch of hardware and let it divvy up the load. With the latest developments in hardware clustering and blade technology, this part of the problem is essentially solved. There really are ways to use less-expensive, commodity hardware to gain advantages of scale at relatively low costs.
But these are homogeneous solutions grid computing gains a large measure of its power and flexibility by providing a heterogeneous solution. “There are sometimes intermediate steps, like moving to clusters, which are a solid technology today,” said HP’s Murphy. “A single utility datacenter sharing heterogeneous resources is a very powerful solution. Sharing resources across multiple datacenters is real grid technology, where the datacenter is grid-enabled to be a node on the grid.”
There’s that term again, “grid-enabled.” It’s one thing to create a completely new application to take advantage of the scalable processing-power of the grid, but what about the applications that matter most to enterprise datacenters? Fortunately, there’s help here, as well.
“Grid is an architecture, a whole way of thinking about computing,” said Benny Souder, Oracle VP of distributed database development. “It’s a pervasive thing. You need all your software infrastructure to really support it.” Thus the introduction of Oracle’s 10g product line. This also points up the importance of all resources, including data resources, being grid-enabled.
“You can allocate CPU both to the application SMP tier and the database tier, and you can reallocate by taking the SMP boxes apart,” continued Souder. “With standard servers in clusters you can add a server to the database tier or the application tier with no down time. Just reallocate when the load changes.” Souder is the founder of the EGA and recruited all the current members. He’s still hoping to add IBM and Microsoft, both members of The Global Grid Forum (GGF), to the EGA.
The GGF’s mission is the support and development of standards for grid technology, and its members are the who’s who of the grid world. But it has traditionally been a world dominated, once again, by research and HPC applications. However, the theme for the next GGF meeting, GGF12, set for September in Brussels, is “Grids Deployed in the Enterprise.” It’s the next logical step in the application of grids.
“Grid is an evolving technology, but it has potential for immediate traction,” said Al Bunshaft, IBM VP of Grid Computing Sales and Strategy. IBM has identified five focus areas, business analytics, engineering and design, research and development, government development, and enterprise optimization, as their initial targets for enterprise grids.
“Predominance of grid in the enterprise today is around making applications more efficient and effective so that end-users can be more productive,” said Bunshaft. “You get better, more seamless access to data through virtualization of data. They’re just starting to use grid as a way to balance multiple simultaneous apps. Servers are usually used on average ten to fifteen percent of capacity, so there’s huge potential.”
Notice that Bunshaft is not talking primarily about the cost savings driving many of the research projects that rely on grid. Grids have repeatedly lowered the costs of supercomputing, but IBM’s focus is on efficiency and that difficult-to-define entity, Quality of Service (QoS). Ever since the dawn of the datacenter, the focus of IT has been on centralized control. But more and more enterprises are decentralized and computing resources are by necessity becoming dispersed. Grid provides a mechanism for the corporate computing model to mirror a more far-flung and heterogeneous business model.
Dan Kusnetzky, Program Vice President for System Software at IDC, makes the following illustrative comparison. “If I was trying to get forty tons of gravel from New Jersey across the Hudson river to New York City, I could use one truck and fight traffic and the lines at the toll booth to move it in. Or, I could divide it into 40 1-ton loads, give the drivers radios to communicate, and they can get there faster, because they’re more agile and can find the most efficient route.” A grid-oriented database is a good example of an application that can be moved as gravel. “But if it’s a 40-ton statue being sent to a museum, you can’t break it up or you get gravel.” Most current applications fall into this monolithic category; in other words, they break if not properly prepared for grid transport.
This is one of the reasons that grid is an evolutionary and not a revolutionary or ubiquitous technology. Kusnetzky thinks that the “pain point” driving IT managers to grid is ultimately cost savings. “The insatiable demand to lower costs may drive this more rapidly than we had thought.” The pain point may be hardware costs, software acquisitions, or staff-related costs, but in each case, Kusnetzky sees a trend toward utility computing, either through the use of computing clusters instead of monolithic computing, open source, and software service models, or by automation through distributed maintenance. “Utility computing is the infrastructure. It’s inherently more complex, but if the technology is well-designed, a fewer number of people can manage it; self-healing, self-managing environments with the appearance of a simple, unified entity.”
Real-World Enterprise Grid
As examples of grids at work in the enterprise, there are many successes in the fields of financial analysis, aerospace and automotive engineering, and pharmaceuticals and health care. While these are important applications that reap significant savings while producing impressive performance gains, they tend to serve homogeneous user groups. In other words, they represent only a first step in grid utilization. Serving a disparate user base adds another layer of complexity, but not an insoluble one.
“We provide a core application for the State of Arkansas, and our primary users are state and federal agencies, as well as the public in general,” said Fred Limp, Director of the Center for Advanced Spatial Technologies and a professor at the University of Arkansas. “We are like any large organization in that we tie together lots of databases with different architectures.”
Limp’s grid supports multi-terabyte databases of geospatial data. It is both computationally-intensive and data-bound, depending on the users’ needs. It supports multiple capabilities across a unified system, the essence of grid.
“We’ve been working in the geocomputational area for more than a decade. In about 1998, we moved to Oracle and then worked with the San Diego Supercomputing Center to try clustering, but it wasn’t robust.” Limp hits on one of the classic limitations of early grids. A lack of comprehensive standards. “It was an R&D environment, and they kept changing their code.”
As Oracle added capabilities to support spatial extensions and then grid computing, Arkansas’ GeoStore application was born. It combines everything from city-level street data, to demographic, economic, agricultural, and geographic data for the entire state. “There are a lot of users connecting directly to the database and using it as a back-end for their own custom applications,” said Limp. The use is constant, but spikes, for instance during tornado season, show the advantages of the distributed nature of the grid. “On the hardware side, we’ve been able to decrease cost by a factor of 10.”
GeoStore uses a firewall-protected configuration of large Sun Solaris Ultra-enterprise systems located both on the university campus in Fayetteville and in the state capital in Little Rock, where a majority of the users are. The same tornadoes that bring peak demand make multiple installations a critical component of their infrastructure.
It’s true that Limp’s background as a professor makes him an atypical datacenter manager, but his application is like so many other mission-critical enterprise functions. “Any large organization is like this,” he said. “We’d like to have our systems talk together, but can’t because there are technical boundaries. Boundaries to collaboration are eliminated with grid.”
Grid Sneaks into the Enterprise on Little Cat Feet
“Grid computing will bring a flexibility and efficiency and almost certainly cost effectiveness that is going to be stunning,” said EGA’s Deutsch. “We’re talking about a scalable, automatically-allocable resource that executes critical applications seamlessly.”
You have to wonder if his ability to make this kind of pie-in-the-sky statement got Deutsch his job. On the other hand, Deutsch admits that the vision of computing as utility is the ultimate holy grail rather than a current reality.
To achieve the ultimate, the EGA has the future mapped out as follows: Currently, grids support technical computing applications within the enterprise. In the first year, grids will support commercial computing within a single enterprise. In the second year, grid will be extended in two directions: across and between enterprise entities and including technical applications. The third year will come close to the ultimate goal of utility computing, where grid will make ubiquitous computing readily accessible.
If you’re an old-timer, this might sound to you a little like the early problems with transactional computing, where one-to-one access was followed by one-to-many, many-to-one, and the ultimate goal of many-to-many. But in this case, instead of data in relations, we’re talking about computational nodes on networks. It’s file-locking and data integrity versus access security and facility protection — a much higher level of abstraction with a significant risk factor that makes the problem exponentially more complex. Fortunately, computer and network power is exponentially more capable, and it seems more a matter of when rather than if the grid takes its place as a standard tool in enterprise computing.
“It’s a natural, easy, evolutionary step to grid,” says Souder. He adds, “The really cool thing is that grid lets you align IT with the business. You no longer have to align to your systems. It’s there when you want it and it meets your needs.”
Clay Andres writes about technology from his home in Northwestern, CT. He is the author of numerous books on computing and is also an editor and designer. Email Clay at firstname.lastname@example.org.