I don't agree that Solaris is the OS of choice for much of the web. First off, there's the ufs filesystem. I don't know that they've fully resolved booting into zfs -- which is a very fast FS -- but ufs is a dog. It makes ext3 look like a screamer, which is hardly a recommendation. zfs easily outperforms ext3 but one major hosting company was down for days earlier this year due to a bug in the zfs filesystem.
Second, testing Solaris against RHEL last year, it was quickly apparent that standard apps like Apache, nginx, etc., performed horribly without some serious tweaking (Sun engineers spent a month and a half to get it to match performance). Most of the binaries at the time were 32-bit as well, and nothing gnu-based performed acceptably (or, in many cases, failed to even compile).
Third, and I doubt they've fixed it as Sun is stubbornly opposed to local filesystems, in order to get Solaris working we had to removed the RAID card physically. Then, we had to remove the fiber ether card because it wasn't supported, either. Neither card was an unknown brand -- a 3Ware RAID and a Tigon card, both of which several varieties of Linux picked up on without a problem.
Fourth, there's Sun even being involved. We purchased an X4600 with the promise of a super-performing machine that they would support Linux on. Their engineers didn't know the first thing about RHEL (and in point of fact I had to help them find crucial logfiles in Solaris when they debugged). The X4600 came with Solaris, crashed completely on first boot and crashed again multiple times. The X4600 was easily outperformed by a 32-bit Slackware box and even got beaten in one filesystem test against an NFS mount. Sun talks the talk about open source and Web 2.0 but doesn't have the first idea about where to step without breaking eggs.
I did try the FUSE-based zfs port to RHEL and it was an unqualified disaster.
It's almost an aside but these experiences have given me the impression that the "Enterprise class" solutions are all about turning Linux into Windows for organizations without the technical skill to use the OS. RHEL, for example, relies on the 32-bit-ish ext3 and offers no options where xfs/jfs easily outperform it; a 64-bit compile of Slackware (Slamd64) is a very unforgiving Linux which outperforms RHEL 4/5 in some cases 3:1. IMHO the community could do wonders for itself by honoring standard kernels and supporting compiles like that which offer lots of power with minimal overhead.
-d »
I don't know about recently, but lighttpd suffered some leakiness a while back (which is in part why nginx became so popular). I've tested nginx for static content and even on smaller servers concurrency+requests I can max out the ethernet without the server hiccuping (this is a dual quad-core Xeon X5355). It simply doesn't run out of power and easily overpowers Apache for anything static. It's clumsy on PHP, but Apache is awful on Rails, so it's a choice of poison. One factor not mentioned here is nginx has Flash streaming available in the binary and some pretty slick stats are possible using rrdtool, and of course gzip compression is available. It's extremely small, extremely simple to configure, and I've never been able to crash it in comparable tests against Apache where Apache ran through all memory and then began to fail. If you're running Rails, you'll probably want to run nginx (the dev version 0.6.30 is just as stable in my tests as the "stable" version FYI). I noticed even CNN is using it for their politics blog. I can't recommend it highly enough, and I've been configuring webservers from scratch for almost 15yrs. »