Andrew Tridgell's implementation of the SMB protocol allowed Linux to "dance" with Windows, but it wasn't always easy.
It’s probably an urban legend, but there’s a story about an Australian company that tried to make a buck off of public toilets in Sydney. They attached coin boxes to the city’s rest rooms and tried to charge a 25 cent admission fee. Within a week almost every pay toilet in the city had its door smashed in or simply ripped off the wall.
Australians don’t let things get in their way.
Andrew Tridgell releases versions 0.5, and 1.0. Then he gets an X Terminal and stops development of server code.
Dan Shearer contacts Tridgell about a Linux port. Tridgell’s response to Shearer: “What’s Linux?”
Tridgell networks a Linux system to his wife’s PC, but has to search the University FTP archives to find a copy of his own server code so that they can share the printer. Soon afterward, Andrew announces NetBIOS for Unix.
Andrew’s smbserver 1.6 is released and then renamed Samba.
JR Conlin and Dave Fenwick start the SMB newsgroup, comp.
protocols.smb, a forum for discussing Samba development.
The first Samba logo, web pages, and FAQ are created by Paul Blackman.
Doom for Linux is released.
Linus Torvalds is attacked while trying to pet a
penguin during a visit to the National Aquarium in Canberra with Tridgell. Penguins don’t like that.
Work is started on
Samba NT Domain coding.
CIFS specification submitted
to IETF by Microsoft.
New Samba logo donated by Robert Gelardi.
Silicon Graphics (SGI) hires Team member
Jeremy Allison to work full-time on Samba.
Samba 2.0 released.
Tridgell finishes his PhD Thesis,
and gets a job with Linuxcare.
And back in December 1991 something was getting into Andrew Tridgell’s way. Andrew was 24 year old Ph.D. student in the Computer Sciences Laboratory at the Australian National University, Canberra. He had three computers and two filesharing protocols. And all three machines — a PC running DOS, a Sun Workstation, and a DECstation 3100 running Digital UNIX — needed to share files with each other. Tridgell had a program called Pathworks, that let the DEC box and the PC share files, but it was still hard for the PC and the Sun — which used the Network File System (NFS) protocol — to communicate. Getting DOS to handle two protocols at once was a nuisance, so Andrew took the direct route. Seized by the muse of the hacker, he did a little packet sniffing, and then set about creating a Pathworks-like program for the Sun machine. Samba’s creator did not realize it at the time, but he had written an implementation of the SMB protocol.
Within a few days, Tridgell had the PC reading files on his Sun machine. Within a week he was mounting disk space from a Sun to a PC. He published his code on the Internet and then put the project aside. From time to time someone would email him a question about the software, but the project was basically forgotten.
Forgotten for a few months, that is, until Dan Shearer sent Tridgell a message asking if he was doing a port for the nascent Linux operating system. This was 1992, and Tridgell had never even heard of Linux. He installed a copy on his PC, and after playing with it for a couple of months, became a convert. Tridgell eventually got hold of a couple of Ethernet cards and built a small home network between his Linux system and his wife’s PC. He recompiled the software he had written for SunOS and, to his great surprise, the two machines communicated. Now here was an interesting project: helping the Linux operating system to act as a file server to PC clients. This was cool. Fortified with new enthusiasm, Tridgell announced the “NetBIOS for UNIX” project.
NetBIOS for UNIX wasn’t exactly a catchy name. When Tridgell finally got hold of some proper Microsoft documentation, he discovered that the protocol he’d implemented was named SMB, so he changed NetBIOS for UNIX’s name to the slightly more ear-catching, “smbserver.” It turned out, however, that the name smbserver was already taken by a company in Federal Way, Washington named Syntax, Inc. After a quick dictionary search for all words containing the letters S, M, and B, Samba was born.
Within a few months more developers had signed on. Dave Fenwick and J.R. Conlin started the SMB newsgroup comp.protocols.smb as a forum for Samba developers and users. There were over one hundred people on the Samba mailing list by 1994. A year later there were fourteen times that many, and by 1996 the list had grown to over 3000. Today the samba list has over 6,000 recipients, and has spawned a whole sub-strata of new mailing lists, including samba-technical, samba-vms, samba-docs, samba-ntdom, and samba-announce.
Samba is made up of a number of NetBIOS programs that collectively implement the SMB protocol (see the “Protocols of Samba” sidebar, pg. 81, for an explanation of this and other Samba technologies). At its heart are two key programs, smbd and nmbd, which let UNIX machines provide file and print, authentication, name resolution, and service announcement (called browsing in Microsoft-speak) services to Windows clients.
By implementing these services on UNIX, (though Samba is now developed on Linux, it has been ported to almost every flavor of UNIX, including AIX, Solaris, and FreeBSD) Samba lets UNIX machines take the place of NT file and print servers. And it has been very successful at this. Open Source advocate Eric Raymond has called Samba Linux’s “stealth” weapon because Windows users don’t even realize they’re talking to a Linux server when Samba is running. Today commercial vendors — including BrainStorm Technologies, Cobalt Networks, Whistle Communications, Hewlett Packard, Silicon Graphics, and every major commercial Linux provider — are shipping or supporting Samba with their products.
Like Linux, Samba has grown from a graduate student’s side-project to a full-scale industry. In fact, the two projects have been catalysts for each other’s growth. “I think that the influence runs both ways,” observes Andrew Tridgell, “Samba didn’t really gain a lot of momentum until it was picked up by the Linux community, but Samba helped get Linux into many corporate environments.”
CIFS and SMB
Microsoft owns the SMB filesharing market in the same way they own the desktop market, and they manage both with the same flair. In 1996, they decided to rename the Server Message Block protocol the Common Internet File System (CIFS), a name that provides some insight into their marketing goals. Microsoft submitted a draft of the CIFS specifications to the IETF (Internet Engineering Task Force) in an effort to make CIFS into an official Internet protocol. And though there are a number of CIFS implementations today, none of them — not even Microsoft’s implementation — has ever quite matched the IETF draft specifications. Though the IETF drafts have expired and CIFS has not become the Internet standard that Microsoft envisioned, its name has stuck. Today CIFS means “next version of SMB.”
There are many other players in the CIFS arena including Microsoft’s former networking partner, IBM. Some have implemented CIFS for small embedded systems, others for small-to-high-end dedicated fileservers. There are also about a half-dozen CIFS products for desktop platforms not supported by Microsoft, such as Thursby Software Systems’ Dave for Macintosh.
Even with all of the commercial competition, the Samba team is a formidable force in the CIFS community. At the 1998 CIFS conference, the Samba team’s 12 attendees made up the largest contingent at the event. The breadth and depth of the Samba developers knowledge, coupled with their willingness to share information, has earned them the respect of their corporate peers, even at Microsoft. A few months ago, a Microsoft employee asked to copy some of the Samba Web pages on an internal server, so that Microsoft itself could use it as reference documentation for the SMB/CIFS protocol. Clearly the CIFS specification that Microsoft had made publicly available through the IETF was not exactly cutting the mustard.
Being popular is not without its risks. Though Samba is licensed under the GNU GPL (General Public License), a small company tried selling a compiled copy of Samba as their own code, in violation of the GPL. The Free Software Foundation kindly donated a lawyer, and the issue was settled quietly out of court.
The Keepers of the Code
None of the Samba team members are in it for the money. Around 1993, Samba users began trying to pay Tridgell for the software they were using. In fact, some government users were not allowed to install software without an invoice. So Tridgell began issuing official-looking invoices for $0, which were then dutifully filed away by the bureaucrats. As a joke, Tridgell added a note to the Samba documentation encouraging satisfied Samba customers to pay him with pizza instead of money. But joke or no, the free pizza soon started to roll in. People would send international pizza coupons, or even call local Canberra pizza joints to order for Andrew. The Samba Web site now includes a list of Canberra pizza restaurants to make paying for Samba a little easier.
Though companies like SGI and Linuxcare have full-time Samba developers on payroll, Samba is, at its heart, a product of creativity, rather than a product of industry. SGI’s Jeremy Allison remembers how his Samba work became a full-time obsession years before he was paid for it. “In 1993 I spent a lot of time working on Samba when I should have been working for my employer, because it was fun. It was a lot of fun.” Don’t tell SGI, but developers like Jeremy Allison would work on Samba whether they were paid to or not. Financial support is a bonus; the real reward lies in producing good code that is useful to people.
In a sense, Tridgell started the Samba team when he first published his server code back in 1991. This was an open invitation to the Internet community to test, use, fix, whine about, and generally become involved in Samba. The team roster has changed over time, but for the most part it has simply grown. Allison has acted as Tridgell’s second-in-command for a few years now, stepping up to the task when Tridgell most desperately needed time to finish his Ph.D. thesis.
The team is a lively bunch. There are sometimes differences of opinion, sometimes arguments, but the Samba team works because the members all respect Tridgell and Allison’s leadership. In the end, they get to decide how Samba will grow and what it will become. Because Samba is released under the GPL anyone who wanted to could start a separate project based on the Samba code. In fact Tridgell hopes that a new Samba-like project will spring up to take a new approach on implementing CIFS for UNIX systems. “I hope they won’t use the Samba code,” observes Tridgell, “not because I would mind, but because I think they will produce something more innovative if they start from scratch. There are a couple of new projects on the horizon that may provide some good competition, and I’m trying to encourage them whenever possible”
The Next Samba?
|The Samba team|
One of the “new projects” that Tridgell is talking about is jCIFS, an attempt to implement SMB client and server capabilities in Java. The project got off with a bang, but has stalled of late, and is now looking for new blood. Its first goal is a Java implementation of the NetBIOS Name Server (NBNS), as NBNS is well documented, and one of the few pieces of the Samba suite that will work on its own.
Another interesting project is SMBFS. SMBFS is a network filesystem for Linux that can mount SMB shares (a share is a shared network resource, like a directory or a printer). Imagine, for example, that there is a Windows system sitting around connected to your LAN that contains Doom WAD files. Now, these days you play Doom on Linux, but you’d like to have access to the old Windows WAD files so that you can play them when you’re in a nostalgic mood.
With SMBFS, you can share the directory on the Windows system and then mount it as, say /usr/games/ doom/wad on your Linux box. SMBFS was originated by Volker Lendecke a few years after he had successfully ported Samba to the NeXT operating system.
Finally, several attempts have been made to capture and distill the core functionality of SMB into some form of code library. SMBLIB is a C library written by team member Richard Sharpe, and LIBSMB is a C++ library, written by Nicolas Brodu.
Domain Control: the Next Great Hack
These days, the Samba team is putting a great deal of effort into implementing Windows NT Domain Control features.Samba’s existing Primary Domain Controller code is still in the alpha stage, but there is a lot of interest in this project: Some users are even attempting to run the Samba Primary Domain Controller code in near-production environments. These folks are providing a great deal of very useful feedback.
The NT Domain projects were born out of a friendly disagreement two years ago. Paul Ashton told fellow team member Luke Leighton that he didn’t think it would be hard to figure out the NT Domain Logon procedure. Luke thought otherwise. Fortunately, Paul didn’t listen to Luke and two months later he had the basics worked out. Ashton and Leighton then published a document outlining how the NT Domain Logon protocol works. One programmer, Linus Nordberg, put together an implementation based on the specification, and donated that effort to the cause, leaving the commercial SMB vendors quietly scrambling to catch up.
Cracking the surface of the NT Logon protocols was not really sufficient, though, as Paul and Luke quickly discovered. They could use the Logon protocol to trick a client into thinking that it was talking to an NT Domain Controller, but that would also cause the client to make all sorts of other assumptions about the capabilities of the logon server.
The only solution was to implement all of the NT 4.x Domain Control capabilities in Samba, which is why development of the NT Domain code is still in the alpha stage. As it turns out, much of the NT Domain system is built on top of Microsoft’s own specialized variation of DCE/RPC (Distributed Computing Environment/ Remote Procedure Calls), something that has to be decrypted and studied before it can be implemented.
Taming the Hydra
Lion tamers lock themselves into big cages and play with wild animals that are much larger and more powerful than they are. Samba, by design, is in the same ring as Microsoft. Unlike a lion, Microsoft is a many-headed beast. Folks within the company have a variety of opinions on Samba. On the one hand, Samba helps Microsoft in their goal to make CIFS a de factostandard. On the other hand, Samba is chewing at the edges of Microsoft’s NT server market — so much so that Microsoft recently sponsored a set of benchmarks, run by MindCraft, to demonstrate their server’s superiority.
The fact that Microsoft was not willing to comment on this story, underscores the company’s firmly conflicted relationship with Samba. “Microsoft is a big company,” notes Tridgell. “I think it is a mistake to view any big company as a single entity. We have lots of supporters in Microsoft, including people who have gone to a lot of trouble to help out with Samba development. Others have been less helpful, but luckily, they are the minority.”
Microsoft first began supporting Samba development in December 1993, when a Microsoft developer relations manager named Lee Fisher took an interest in the software, and began feeding the team SMB information. “Lee is one of the unsung heroes of Samba,” remembers Jeremy Allison, “He gave us a bunch of internal documentation about how SMB works, and what the extensions were.” Since then, Fisher has moved on, but there are others within Microsoft who continue to lend a helping hand.
There is still quite a bit of debate over the MindCraft benchmarks, but in the end, Linux kernel developers and the Samba team took them as an opportunity to figure out where their code could be improved. Linus Torvalds has called Microsoft a “Linux user in disguise,” and now says he considers the fact that Microsoft is reporting bugs to the Open Source community a help rather than a hindrance. Benchmarks, like sonar readings, can provide information about the shape and configuration of things that might otherwise be invisible. The Samba team itself has had to become expert at using this kind of indirect information to figure out what Microsoft is doing inside its systems.
Another tool that has been crucial to Samba’s evolution is the packet sniffer. Packet sniffers are programs that examine and dissect the data that networked machines send to and from each other. Much of the Samba team’s knowledge of SMB has been gained by simply watching the wire. Microsoft includes the netmon tool in their Resource Kits for this very purpose. Netmon does an excellent job of sniffing SMB traffic, as it leverages Microsoft’s knowledge of SMB and NetBIOS. Not to be outdone, team member Richard Sharpe has been adding SMB and NetBIOS decode support to the Open Source Ethereal sniffer (http:// www.zing.org), giving Samba developers an open source alternative to netmon.
Though some within Microsoft have helped the team, others are openly hostile. Microsoft has some deep, dark secrets that they would like to keep to themselves. The inner workings of NT Domain control is one of those secrets, and the Samba team have had to rely on their own resourcefulness to figure out how it actually works. They have dug up clues in header files, obscure documentation, and parenthetical references from Microsoft presentations. Often, the team learns as much from what Microsoft does not say as what they reveal. Half secrets are as dangerous to the teller as half truths are to the listener, and Microsoft isn’t sure what to share and what to keep under its collective hat.
At one point, Allison and Tridgell were trying desperately to discover a “magic constant” that Microsoft was using for password encryption. They knew the basic algorithm, but without the key they could neither lock nor unlock the data being transferred. Of course Microsoft didn’t want to share the key, but information about an intermediate value slipped out during a technical discussion. It was a clue, a big clue, and Allison was determined to make use of it. He spent a sleepless night hunched over his computer, working and reworking the problem. By sunrise he had discovered the magic constant. Bleary-eyed but elated, Allison logged on to send an e-mail to the Samba team, but there was already a message waiting for him. Andrew Tridgell had cracked the code a few hours earlier.
Samba exists both because of, and despite, Microsoft. If there were no Windows, there would be no need for Samba, and perhaps that would be a good thing. SMB is not a very elegant protocol, after all, and the Internet might be better off without it. Even Tridgell himself wouldn’t complain if it simply went away. “The SMB protocol is really pretty horrible,” he observes, “I would really like to see it die off in the next five years.”
If SMB is the heart of the file and printer sharing architecture, then what are the epidermis, appendix, and Achilles tendon? There are many protocols that work together, more or less, to form the complete system. Anyone who has ever fiddled with networking on a Windows box will be familiar with such terms as NetBIOS, NetBEUI, Workgroups, and Network Neighborhood. Perhaps the Samba mascot should be a duck-billed platypus. Both are Australian, and both are made up of so many funny pieces.
NetBIOS (Network Basic Input Output System)
In the early days of the PC, IBM and a company named Sytec co-developed the hardware and software for a simple LAN system designed to handle a maximum of 70 to 80 nodes. The software they developed was named NetBIOS, which was an interface that loaded into DOS’ memory so that applications could talk to the network hardware. This was all before Ethernet or Token Ring took hold, and the network cards they designed are museum pieces now. NetBIOS, however, has survived and flourished due simply to the annoying bulk of software written to use it. One key product that is still tied to NetBIOS is the SMB protocol. Microsoft says that Windows 2000 will include support for SMB running native over TCP/IP (see CIFS, below).
NetBEUI (NetBIOS Extended User Interface)
IBM moved to Token Ring but, as mentioned, they still needed to support NetBIOS. NetBEUI was created so that NetBIOS messages could be sent over Token Ring or Ethernet. Like NetBIOS, NetBEUI was designed for a LAN environment and includes no routing information. NetBEUI is still used for filesharing on local LANs, but Samba doesn’t do NetBEUI.
NBT (NetBIOS over TCP/IP)
Unix folks, as well as Microsoft (who once sold a UNIX-like product called Xenix) wanted their share of the SMB thing. So in 1987, two documents — Internet RFC1001 and RFC1002 — were published. These described a mechanism for emulating a NetBIOS LAN over a routed TCP/IP network. This is what made Samba possible. Today, NBT is the preferred way to shove NetBIOS packets around.
NBNS (aka. WINS)
On a local LAN, a computer can send out a broadcast message to find the address of another machine. It’s sort of like yelling “Yo! Fred!” If Fred is within shouting distance he will answer “Waddaya want?” (or words to that effect). This doesn’t work where a router is involved because computers on the far side of the router can’t hear the broadcast messages. RFC1001 and RFC1002 solved the router problem with something called the NetBIOS Name Service (NBNS). The NBNS acts as a sort of address book. NetBIOS nodes send their names and addresses to the NBNS. When one machine needs to find another, it looks up the name in the NBNS. Microsoft called their NBNS implementation WINS, for Windows Internet Name Server. In Samba, both broadcast and NBNS name resolution are handled by the nmbd program.
Workgroups and NT Domains
Workgroups were devised to separate users into logical groups, so that a network administrator would not have to manage user access on an individual basis. The NT Domain system takes this one step further by providing a central server to handle user authentication. The majority of new Samba development is targeted at implementing NT Domain functionality.
Browsing (aka. Network Neighborhood)
The Browsing system is used to keep track of a list of available SMB services, known as the Browse List or Network Neighborhood. Samba’s nmbd handles browse list management.
CIFS (Common Internet FileSystem)
CIFS is the new name for SMB. In 1997 Microsoft submitted draft specifications for CIFS to the IETF, apparently hoping that the SMB filesharing protocol would become an Internet standard. The drafts and revisions have expired.
There are other oddities that make deciphering Microsoft’s protocols difficult, including variations between implementations and outright bugs. It sometimes seems as though Microsoft has a company policy against code reuse, preferring to reinvent the same wheels with every new operating system. One of the factors that worked against Samba in the MindCraft benchmarks was that Samba is tuned to serve NT workstations, while NT Server is tuned to serve Windows 95/98 clients. Working around Microsoft’s bugs is another problem. In general, the Samba team will mimic a Microsoft bug rather than try to fix it within Samba. This is done to prevent the kind of problems that occur when Windows clients do not encounter the bug they expect.
Samba development has traditionally been fueled by pizza, and the Samba Web site has a listing of pizza shops in Canberra (see the documentation page) where you can make a donation to the development effort. You can also purchase a Samba t-shirt; the team is still in the hole for the printing costs.
And then there’s the next version of Windows NT, known as Windows 2000 to Microsoft, and affectionately named W2K by the Samba team. At this point, the team is taking a hands-off approach to W2K. Microsoft has promised a degree of backward compatibility, so it should not be difficult to integrate Windows 2000 into a network of Samba and older Windows systems. Once W2K has stabilized the team will evaluate its market penetration and decide which parts, if any, to attack first. Samba is an on-going and open effort. Visit the Samba Web site (http://www.samba.org), join the mailing lists, and become a part of it. Or if you like, just send pizza.
Christopher Hertel and Luke Leighton are members of the Samba team. They can be reached at firstname.lastname@example.org and email@example.com.