"Money makes the world go around" is a well-known adage, and a similar aphorism could be made about the Internet. Indeed, in the last ten years, a mammoth industry has grown out of and because of the Internet. These days it's quite common to conduct a considerable part of one's personal business online, using the Web to manage utilities, bank and securities accounts, and to purchase everything from cruises to cars to compact discs and even candy.
“Money makes the world go around” is a well-known adage, and a similar aphorism could be made about the Internet. Indeed, in the last ten years, a mammoth industry has grown out of and because of the Internet. These days it’s quite common to conduct a considerable part of one’s personal business online, using the Web to manage utilities, bank and securities accounts, and to purchase everything from cruises to cars to compact discs and even candy.
Yes, even in the Internet age, money still makes the world go around — except now the money exchanges hands via cashless transactions performed through a series of web forms and automated interactions between banks and credit card companies.
The irony in all of this is that the bulk of the Internet’s infrastructure is largely based on free software. Open source runs mission-critical services like email, domain name resolution, and web serving, and in many cases, continues to outperform proprietary solutions, providing better reliability and security. That said, it might not come as a surprise that free software also conducts a great many electronic monetary transactions.
In this installment of “Out in the Open,” we introduce a company whose sole business is electronic commerce and whose operations are largely built on open source. As you’ll see, you don’t have to pay for software to enjoy success.
Columbus, Ohio-based 2Checkout.com (or 2CO, located at http://www.2checkout.com) offers electronic credit card processing services for over 25,000 web-based businesses in 145 countries. Since its founding in late 1999, 2CO has seen explosive growth, with 2002′s processing volume exceeding 2001 numbers by an impressive 500%, and 2003′s volume on track to double the 2002 mark. Expressed in dollars, the company processed roughly $12 million dollars in sales throughout 2001, while in the first 10 months of 2003, the company surpassed $100 million dollars, with the holiday season still to come. Current estimates hold that 2CO will process $140 million in 2003.
$140 million is no small feat, to be sure. However what makes 2CO’s story particularly intriguing is the company’s IT strategy. Not only do open source products play a critical role in all aspects of the company’s services, but open source is also heavily used for internal intra-office operations. Linux Magazine sat down with 2Checkout.com’s Director of Information Technology Bill Jones to talk about the company, its service offerings, and their ongoing commitment to open source technologies.
LINUX MAGAZINE: Please describe the business of 2CO.
BILL JONES: 2CO provides credit card processing services to web-based businesses who either can’t or don’t want to get individual merchant accounts. 2CO also provides a high degree of fraud checking, handles customer chargebacks and refunds, and allows people to link to our checkout pages using our secure certificates.
LM: How did 2CO come into existence?
JONES: The founder of our company, Alan Homewood, started a number of Internet-based companies in the late 1990s. As he was setting them up, he noticed that all of them needed to process credit cards online. So, instead of setting up different merchant accounts for each one, he started a company solely dedicated to providing merchant services for online sellers. 2CO took off from there.
LM: What projections have been made for 2004/2005? Assuming that continued growth is expected, what steps are being taken to ensure that the processing infrastructure can continue to perform?
JONES: Interestingly, 2CO’s home page offers periodic updates of the total revenues processed by its systems. According to recent numbers, it seems quite likely that the company will break the $100 million mark in 2003. We’ve more than doubled [2002's] processing volume, both in number of transactions and dollar value, and that followed a twenty-fold increase over 2001. Looking ahead to 2004/2005, we expect similar growth, doubling the dollars processed.
To handle this increase, we are increasing our connection speed to the Internet backbone four-fold in November , are performing a complete rewrite of our CGI scripts, and are completely redesigning our database structure. In addition, we are increasing the number and speed of our servers and are moving to a high availability web server farm structure and redundant, multi-processor (8 CPU) database servers. We’re also implementing a dedicated data warehouse for the sole purpose of speeding up the reporting that’s available to our customers.
To put this in perspective, in 2002, we were utilizing a total of seven servers for the entire business. By December 2003, we’re running seven servers just for our external web site, plus three database servers, four file servers, a server for our administrative intranet, a mail server, a support server, and more, for a projected total of 24 machines. And I have plans on the drawing board that will require at least six additional servers before June 2004.
LM: Describe 2Checkout’s platform. Which components are open source, which are closed source?
JONES: Our web servers run Apache, and our back-end coding is done in a combination of Perl, PHP, C, and C++. The web servers run on generic Pentium 4 and Athlon XP machines purchased from HP, Dell, and Micronux. Our database is MySQL 4.1.0 running on a Sun Enterprise 3500 and an Enterprise 4000, both running Sparc Linux. All of our support servers are either single- or dual-CPU Athlon XP machines. We also have some Sun Ultra 5 machines used for internal administration and some internal machines that run Windows XP, but those were placed by some of the software vendors we purchase from and only because Linux or FreeBSD solutions weren’t available.
LM: Was there a particular reason for running MySQL 4.1 on Sun hardware? And why Sparc Linux instead of Solaris?
JONES: We run our database on Sun hardware to get the benefits of the 64-bit UltraSparc CPUs and because we could get an extremely powerful machine for roughly 25% of the cost of equivalent Intel hardware.
We chose Sparc Linux over Solaris primarily due to the licensing costs involved in Solaris, which totaled roughly $32,000 for the two servers. Once we worked out some issues with the disk arrays, everything’s worked just fine.
On average, we process between two to four transactions per second, with database queries averaging between 300 to 400 per second, and at times spiking to around 1,500 per second.
LM: Is MySQL currently the sole database solution deployed at the company?
JONES: It’s our production database. We have small installations of SQL Server (for phone use and security), Sybase (for credit card processing), and Oracle (also for credit card processing). Commercial databases are used at 2CO because some third-party products depend on them. Otherwise, all of 2CO would run on MySQL.
LM: Please describe the decision-making process that led to the choice of open source software over similar proprietary solutions.
JONES: Our founder, owner, and CEO was the initial programmer and used open source from the beginning. So, there was no reluctance then, and certainly not now, to using open source.
Some of our early developers used some proprietary programs (we still have some FrontPage tags on a few of our older pages), but now, for all practical purposes, we’re all open source. Our internal staff works on diskless, thin clients hooked into a Linux file server running LTSP, and use Evolution, Mozilla, and OpenOffice for productivity software. Some of our management team runs Windows XP on desktops, but our company fileserver is a Linux machine running Samba.
LM: Overall, how would you rate the success of this decision?
JONES: We’re very satisfied with the decision. We did have some issues when we took our customer support staff off of Windows-based desktops to put them onto the Linux diskless thin client solution, but those issues were largely attributable to introducing a different user interface. Once those glitches were worked out during the first month, our workload with patches and virus issues went way down. (See the sidebar “The Linux Terminal Server Project” for more information on Linux thin clients.)
The Linux Terminal Server Project
The practice of deploying diskless or “thin” clients that act solely as interfaces to a central server upon which the operating system, user applications, and data is stored has been around for decades. Deploying thin clients can result in significant savings because desktop hardware demands are minimized and because system administration is largely centralized at the server. Thin client products include Citrix’s Metaframe and Microsoft’s Terminal Server, both of which are widely deployed and have unquestionably resulted in considerable savings for the organizations that use them.
The Linux Terminal Server Project (http://www.ltsp.org), better known by its acronym, LTSP, offers a solution for deploying thin clients running the Linux operating system. An open source project since 1999, the LTSP supports all of the prominent Linux distributions, including Red Hat, Mandrake, SuSE and Debian, among others. Recently chosen as “Best Open Source Project” at the August 2003 LinuxWorld Conference and Expo, LTSP continues to show considerable promise as an effective, Linux-based, thin client solution.
With the success we’re having running open source, the only thing that would drive us towards proprietary software is if we grow out of our current software choices.
LM: It’s interesting that open source solutions are used by the majority of the internal workstations. How else is free software used within 2CO?
JONES: We use qmail (http://www.qmail.org) and Exim (http://www.exim.org) for email, although we’re standardizing on qmail. All internal projects are based on open source solutions. Our DNS will be moving in-house shortly, and our plan is to run djbdns (http://www.djbdns.org). (See the sidebar “djbdns: An Alternative to BIND” for more information.)
djbdns: An Alternative to BIND
For many, the open source Berkeley Internet Name Domain (BIND) software product is synonymous with the domain name system. BIND implements the DNS protocols, which ultimately broker the resolution of domain names such as www.example.com to their respective IP addresses. However, BIND also has a long history of debilitating exploits and bugs, with the most recent serious issues being uncovered as recently as late 2002.
The oft-questionable quality of the BIND software has prompted various organizations to consider alternatives, one of which is djbdns, written by D.J. Bernstein (http://cr.yp.to/djbdns.html). Bernstein is so confident of his open source product that he’s offered $500 to the first person who publically reports a verifiable security hole in djbdns.
Bernstein is no slouch when it comes to software. He’s also the author of other popular open source products, including qmail and daemontools, and is a professor at the University of Illinois at Chicago. (In his spare time, Dr. Bernstein also fights court battles against the United States’ cryptography export regulations.) According to Bernstein’s web site, djbdns is used by over 1.8 million domains, including the Lycos search portal and Interland ISP.
LM: What has 2CO done to ensure system redundancy?
JONES: Everything is redundant. The web server is a high availability cluster of machines running LVS (http://www.linuxvirtualserver.org), and our databases are replicated, including replication to a read-only system. If one machine goes down, there is always a backup ready to be put into place, and in most cases this happens automatically. We back up data multiple times each day and always have an offsite backup. We have a group of servers at an offsite location just in case a disaster destroys all of our primary systems.
LM: Describe 2Checkout’s current hosting environment.
JONES: We’re connected via multiple T1s, but are moving to a DS3 as soon as we can get it put in. We have redundant service to geographically different POPs, along with other lower bandwidth backups. Also, we do have the offsite machines for additional redundancy.
LM: Can a web site integrate seamlessly with 2Checkout’s infrastructure? In other words, is it possible to perform transactions without actually leaving the client web site?
JONES: No, not at this time. Currently, the client web site is hands-off to us. We complete the transaction and then, depending upon the administrative settings the client’s made, we pass it back to their web site. We are exploring the possibility of an API that a client could use to perform a transaction without leaving their site, but that feature is in the early planning stages only.
LM: How does 2Checkout.com prevent fraud? Presumably this is done against an external data source?
JONES: Currently, we check every purchase against an IP locator database (down to the city and ISP level), a card level database, and other internal information, including previous fraud/return/chargeback history, both from the buyer’s perspective and the seller’s perspective. Each sale is assigned a fraud score and all orders that we flag for fraud are then personally reviewed by someone in our fraud department. Their actions include reviewing the order, possibly trying to contact the purchaser, or additional steps. They do have the option to delay or cancel the sale based on evidence they’ve collected. Of course, this is just an overview of our fraud protection procedures. There are other additional steps taken.
Check Out 2Checkout.com
2Checkout.com continues to perform exceedingly well in a sector that makes or breaks companies based on timeliness, effectiveness, and reliability. The fact that 2CO continues to expand while largely depending upon software available in the public domain makes the company’s success — and open source’s success — even more impressive.
Jason Gilmore is the author of the forthcoming book,
Beginning PHP 5 and MySQL: From Novice to Professional, due out by Apress in early 2004. Jon Shoberg is an avid developer of Perl, PHP, and Java applications for the Fisher College of Business at the Ohio State University. Special thanks to Bill Jones for providing valuable insight regarding 2Checkout.com. If you’d like to be considered for an upcoming “Out in the Open,” please send email to email@example.com.