Brave New Worlds

A number of open source projects, including WorldForge and NeL, are redefining the frontiers of massively-multiplayer gaming. Take a peek at the games, the gearheads, and the gizmos that make play work

Life online is certainly getting complex. As if spam, spim, crackers, and worms weren’t enough, you now have to worry about dragons, thieves, spells, and Sith lords. It’s almost enough to make you pull the RJ45 plug. But don’t. Dragons aren’t a new Windows exploit (yet), and you don’t need to protect your hard disk from the Sith. And while you might run into such beasties online, have no fear: you’ll have a trusty axe, light saber, or passel of pigs to protect you.

Over the last several years, massively-multiplayer online role-playing games (MMORPGs) have grown more and more elaborate and even more popular. In games such as EverQuest, Star Wars Galaxies, and the classic Ultima Online, thousands of people go online simultaneously to battle, socialize, trade, cooperate, and compete in vast worlds that exist only in the lines of code and bits of data that define them. In a MMORPG, you don’t have to remain Bob Accountant or Jane Programmer. You can be a bounty hunter, an elf, a Jedi, and Bob can be Jane or vice versa.

But the richness of modern MMORPGs comes at a price: as worlds become increasingly complex, so does the code required to create them. Indeed, developing the software, designing the settings, and maintaining the day-to-day infrastructure of cutting-edge MMORPGs is a massive endeavor. Usually, large teams of developers and artists toil for eighteen months or more to produce game software (which then sells for upwards of $40 per player, plus a monthly subscription fee). And while most MMORPGs allows for mods, or user customizations of some sort, the games are otherwise closed systems. Given the complexity and propriety of the software, building a substantial MMORPG is simply beyond the reach of most independent developers and gamers.

But as with other closed and complex systems — operating systems, web browsers, and databases — open source developers are tinkering with MMORPGs, too. Two such projects, WorldForge and NeL, strive to put games back in the hands of gamers and prove that open source can play just as hard as the big boys.

Building Big Worlds Out of Small Ones

WorldForge (http://www.worldforge.org) is the earliest known effort to make an open-source MMORPG. Named for a community of programmers and artists who develop Linux-based client and server software for running online virtual worlds, the original goal of WorldForge was to create an open source clone of the popular Ultima Online. However, it quickly became obvious that building a game that would serve “just” thousands of players simultaneously was an unrealistic goal to pull off right away.

Figure One: The world editor for WorldForge

“We’re far away, if ever, from cloning a commercial MMPORG, even if we wanted to,” admits James Turner from Edinburgh, Scotland, who coordinates software development for the WorldForge game servers in his spare time.

Thus, the WorldForge community focuses on designing generic game client and server code that’s flexible enough to support almost any type of online role-playing game.

First, WorldForge programmers built a networking protocol to bind together the large numbers of clients and servers used in a MMORPG. Named Atlas, the WorldForge technology is dynamically extensible and self-defining, so servers can communicate new types of entities and operations to the client. At the same time, clients can flexibly support rule-sets and games without custom code, and new entities can be defined at will. This allows content creation to be much more iterative, and enables a much looser client-server relationship than is traditionally possible. New gameplay actions can be added simply by modifying the scripts associated with a game.

“Atlas is probably much more flexible and powerful than anything being used commercially,” claims Al Riddoch, a systems programmer for the University of Southampton, who coordinated the development of Acorn, WorldForge’s first game.

Acorn was an experiment for the WorldForge team, designed to teach them how to create a very basic role-playing game. In Acorn, players compete online to raise and sell pigs. Compared to the vast scale of Ultima Online, the size of Acorn’s world is much smaller, its game play is much simpler, and it serves a lesser number of players.

Figure Two: A screen from Acorn

Mason, the second WorldForge game, was created to develop code for building and manipulating objects in a role-playing game world. In Mason, graphics are polygonal 3D models, and game play features humans and orcs fighting over valuable resources, including trees, animals, and iron ore. Threats to player-characters come in the form of inanimate objects like deadly traps and poison. There’s no combat or spell-casting, because the game was intended to foster a social gaming structure unlike that of the typical “hack and slash” play found in most commercial MMORPGs.

“When power and money are the only objectives, things get boring very quickly,” opines Oliver White, a software engineer in Melbourne, Australia, who developed a program to import data from relational database management systems into the game-world model used in Mason. A lack of violent solutions to disagreements presents new challenges, he adds. “When your neighbor builds an outhouse on your prized strawberry patch and you can’t simply knock him on the head with a club, things become a bit more interesting.”

Acorn and Mason were built to hone the basic features of MMORPGs, little by little, over a series of simple games. The idea is that a series of successfully made mini-games ultimately realizes the bigger goal of creating a more large-scale online gaming experience.

“By doing the [games] on a low level with a small set of rules, we can avoid a lot of the balancing problems that arise from creating a large number of rules,” explains Hans Häggström, a software engineer in Helsinki, Finland, who handles the lead design for Mason.

Sticking to this “incremental” plan, WorldForge’s programmers believe they will eventually match most of the capabilities of the commercial MMORPGs. That may sound a tad optimistic or naive, but the code for the non-player characters (NPCs) in Acorn is as sophisticated as that used in many online RPGs, they claim.

“‘Cyphesis,’ the server for Acorn, uses a goal-based system for programming non-player characters. It provides for more advanced behavior than what’s typically displayed in commercial MMORPGs,” says Riddoch.

Two examples of Cyphesis’ abilities: the pig sellers in Acorn are only around during the game world’s morning and afternoon. At lunchtime and in the evening, they hang out at the inn. The pigs themselves have herding behavior (too), which means that if a group of pigs are grouped together for some time, they can be driven as a herd. “Acorn only scratches the surface of the capabilities of our code,” says Riddoch.

This level of sophistication has its drawback, though. “In spite of — or perhaps because of — these capabilities, the quantity of players that can be supported by Cyphesis may be rather limited, especially compared with the [commercial] MMORPG’s,” admits Bryce Harrington, WorldForge’s Project Coordinator and an engineer from Tigard, Oregon. “However, we didn’t intend Cyphesis to handle such intense loads. Its objective is to provide specialized AI services and thus run as a specialized subsystem for the main server.”

Guessing Games in the Development of Acorn and Mason

Designing Acorn and Mason presented three unique technical issues, which the WorldForge developers had to iron out:

1. DATABASE RELIABILITY. MMORPG servers use a database to store the state of the game world. The performance and reliability of such a database is critical. The WorldForge team had to find a good open source or freeware database to do the job.

“MySQL performs poorly under heavy load,” says Riddoch. “In particular, it doesn’t handle very large tables well, and doesn’t optimize complex queries well either. Until recently, PostgreSQL was too slow, though recent performance improvements now make it the preferred database for WorldForge servers.”

However, Harrington says, “PostgreSQL has its own sets of issues. It has many of the needed features MySQL lacks, but is a little stricter. For read-only applications, it can be significantly slower than MySQL, but we haven’t done any benchmarking or timing, so we can’t really say much there.”

Harrington adds, “Our database design is only modestly relational. Most of the data is stored in a small number of tables. Because of this, our database needs emphasize a few specific features, such as row-level locking and query caching. We believe PostgreSQL is a better fit than MySQL on these counts, but aren’t ruling out the possibility of other databases that may be even better suited to the task.”

2. CORE SERVERS. Lots of articles discuss client and server design for Quake-style games, but the WorldForge programmers found scant information elaborating how online games servers work. “There is very little information on such systems, whereas clients — especially on the graphics side — are quite well described,” says Turner.

Apparently, commercial online game developers are reticent about revealing the inner-workings of their servers, and using existing multi-user dungeon (MUD), object-oriented MUD (MOO), and multi-user shared hallucination (MUSH) code wasn’t of much help to the WorldForge developers either. So, designing the server code was a guessing game.

“While the basic functions [of the server] are obvious, we have seen all manner of proposals for game server architecture,” says Turner.

3. PERFORMANCE. With Acorn, the performance of the early versions of the game bogged down largely due to its artificial intelligence engine. Those initial results suggested that things could get worse in a game that featured more play elements.

“Until we start scaling up the number of clients, rule-sets, types of objects, number of objects, number of moving and colliding objects, we don’t really know which aspect is going to limit us,” admits Turner. “Will all these things scale equally? Will we spend 95 percent of our time doing collision detection? Everyone is writing their code with the expectation that it won’t hog vast amounts of CPU, but until we start scaling the world up, we’re just guessing. Still, we are trying to ‘optimize last’ — or at least not prematurely.”

“We are frequently told that Atlas is over-engineered and too slow, but I believe that in the long run, it will pay off,” says Riddoch. “As CPUs get faster, the cost of this flexibility will diminish, and performance will be less of a problem.”

NeL: Commercially Made, Free for the Taking

Giving away the engine that runs a game you plan to sell might not sound like good business sense, especially while in the midst of making the game. But the developers of Ryzom (http://www.ryzom.com) see things differently.

Figure Three: Creatures from Nevrax’s Ryzom

Like EverQuest and Star Wars Galaxies, Ryzom is a massively-multiplayer online game featuring immersive, 3D graphics. The game was created by Nevrax, which has offices in London, England and Paris, France.

While the game will sell like other, traditional computer games, the engine code for Ryzom, called NeL (for Nevrax Library) has already been released under the GNU General Public License (GPL).

So, how does Nevrax expect to make money? Nevrax says that the real value and return of any commercial MMORPG lies in the subscription fees that people pay to keep playing, not in the sale of the software.

“In a massively multiplayer online game, you’re not selling a software product — you’re selling a service, access to a virtual environment, and its community of players,” says Olivier Lejade, co-founder of Nevrax. “This overall experience is unique. That’s what you want to keep proprietary. Creating and especially maintaining the software necessary to do this can be a sunk cost. That’s why we think having NeL as free software makes sense.”

To that end, Nevrax has retained the copyright and other rights to Ryzom itself, but NeL is free for anyone to download, modify, and use for whatever purpose, be it commercial or non-commercial.

Sophisticated 3D Graphics
Delivered Online in Real-Time

NeL (http://www.nevrax.org) is a toolkit for the creation of 3D MMORPGs or similar online virtual-world environments that require client and server code. It runs on either Linux or Windows, and uses OpenGL as its 3D graphics renderer.

Cyril Corvazier, from Paris, France and the lead 3D programmer for Nevrax, says the goals for the NeL toolkit can be summarized in three points:

1. IMMERSION. NeL must be able to render high-quality 3D graphics.

2. MASSIVE. The graphics engine must sustain good performance, even when displaying a large number of players and other characters.

3. OPEN. The engine code must be accessible to the open source community to use and modify.

Like the WorldForge developers, the programmers of NeL say the primary challenge in creating an MMORPG engine lies in the design of its network infrastructure. The servers need to be very scalable, easy to maintain, and secure.

“This is not e-commerce and banking. Here, we have to secure gameplay. This goes from basic cracking protection to well-thought out game and policy design,” points out Vincent Caron, a software developer from Paris, France, who maintains the public releases of the NeL toolkit. “This needs transparency, humility, and a lot of testing — not to mention good system administration and game design skills, too.”

NeL’s technology is extensive. It comprises 3D graphics, networking, artificial intelligence, and audio code. Compared to a commercial MMORPG 3D graphics engine, such as EverQuest’s, NeL gives artists more flexibility in modeling outdoor settings, and has better rendering and lighting model capabilities, according to its developers.

Because of its aggressive goals for visual acuity, the toughest part about designing NeL has been figuring out how a large number of environments and character models can be most effectively rendered, and how to quickly transmit data affecting these elements to the player’s client. The key? Mastering the “data pipeline” of NeL.

“The data pipeline has to be smooth, because in an MMORPG, the engine handles thousands of small assets,” says Corvazier. “NeL provides an automatic process to build the runtime data from the artists’ database. The data production for NeL is a complex problem. NeL deals with twenty media file formats, most of them exported and built from the artist database.”

This production pathway is the critical part. Says Caron: “You end up with a lot of data in most MMORPG games. You need an efficient way to go from your artists’ data to the compiled game-set. And you need to do it in a continuous way, while your world is running.”

The Trials and Errors of Building an MMORPG Engine

The major features of the NeL toolkit include:

* A landscape renderer that creates environments with “patches” of land, blending together sub-divisions with vertex shaders. Landscapes can extend out for more than a kilometer (in virtual distance measurements), and physical features in the landscape, like holes in the ground and tunnels, can be added.

* A lighting system for indoor and outdoor environments, and a special effects manager.

* Multi-resolution mesh (MRM) technology, which automatically scales down the number of polygons in a character model (when appropriate) to conserve system processing power.

* A level-of-detail (LOD) balancing system that derives the best image quality for the number of models and special effects that need to be rendered on screen at once.

In addition, most model parameters can be animated by the user through a built-in animation system, and landscape, character, and building textures can be streamed from the hard disk during a game.

NeL is based on some assembly language, C++, the Standard Template Library (STL) and Runtime Type Identification (RTTI), and uses bash for the engine’s data builder system. Caron says this recipe was a good decision, since the project had to scale to great proportions. “It’s 500,000 lines of code today. As a personal note, I regret that only C++ was used. A bunch of high-level things could be done much faster with scripting languages. I’m mostly thinking about the [NeL] tools suite, which is much too painful to maintain.”

Adds Corvazier: “The tools in this project [are significant]. NeL has more lines of code in tools than in the engine. The tools require a lot of maintenance.”

Originally, the road map called for using Python in the development of NeL, but this was gradually dismissed as the toolkit evolved. Caron is of the opinion this shouldn’t have happened.

“A fair amount of code in an MMORPG is not in the critical path, because it’s simply gluing low-level functions like 3D, sound, and networking together. When you wrap your low-level C++ code into Python classes, you’re encouraged to make the simplest interface for both the C++ wrapper’s and Python coder’s sakes,” he explains. “The other point is that it’s easier to convert existing Python code into a native C++ implementation than the other way around. And a Python development cycle is many orders faster than C++’s. Compile times just get insane at 500,000 lines.”

“There really is no absolute way to do this,” says Lejade. “Even if you count the first MUDs, building persistent worlds is still a very recent craft. There’s still plenty of room for experimentation, trial, and error.”

Smaller, More Unique Worlds Made for You and by You

Business interests keep commercial MMORPGs afloat. Money is made from the initial sale of the client software, and additional revenue is generated by hooking players into pay-for-play subscription fees.

So how can an open source equivalent work? Both the developers of WorldForge and NeL believe that if players have access to the source code and are given the tools to create such worlds themselves, volunteer-run MMORPGs will sprout up across the Internet.

The biggest strength of the WorldForge and NeL projects is that their development communities can support online games that might not be viable in the marketplace. Homegrown, quirky games could fulfill the niche interests of smaller communities of online players.

“By providing technology, we hope to lower the barrier of entry, letting gamers focus on what really makes the difference in a ‘persistent world:’ the conception, the artistic design, and the administration of these worlds,” says NeL’s Lejade.

To these two teams, the ideal online virtual world is a community where players not only play, but can also conjure games at a whim to create a new experience for others to enjoy — the more unique these massive settings are, the better.

“What you’ll see is everyone’s idea of a good game coming to light,” says WorldForge’s White. “People with expertise in various subjects will be able to have their say on how to implement game rules, from fencing experts to physicists to ecologists to indigenous theologists.”

God, that sounds good.

Howard Wen (http://www.howardwen.com) writes about gaming and multimedia in Linux and open source. He previously wrote about Falcon’s Eye and Slash ‘Em in the December 2003 issue of Linux Magazine (available online at http://www.linux-mag.com/2003-12/nethack_01.html). You can read more of his articles on the O’Reilly Network (http://www.oreillynet.com).

Comments are closed.