Gentoo Optimizations Benchmarked – Part 2

Summary

The results of the tests show that the biggest jump in performance is from i486 to i686 and that there is not much extra to gain from then on. One would have expected to have seen a greater advantage for the more optimised binaries when it came to encoding, however this also comes down to the program itself and whether it can make use of extensions such as MMX and SSE (something which an i486 binary can still do).

The biggest surprise was in Ogg encoding, where the Core2 system was twice as slow as others, including Pentium3! This does not seem right, however the test was repeated several times, all with the same results. This may be an issue with the Ogg encoder itself, when used with the highly optimized Core2 binary.

Yes, we are not comparing even systems here, but it’s interesting to note that Ubuntu faired well in some tests, but fell behind in others. Generally, it performed at or below the Gentoo i486 level. The main benefit that Gentoo has over Ubuntu is the ability to fine tune the system, leaving out what you don’t want and including those components you do. This means the system can be more lean, which results in better performance and less overhead. The systems used in these tests were already very lean, with just X.Org and Xfce4.

So does it pay to optimize your operating system specifically for your CPU? It sure does, but only just. Performance of the Core2 binaries over i686 was marginal and certainly there was a much larger performance gain from i486 to i686. Nevertheless, combining a CPU optimized binary with a specific GCC optimization (such as -Os) to fit your specific requirements may well be a very attractive proposition. Of course, the question will be whether the trade-off between compile time and performance gain is worthwhile, but then, that’s not what Gentoo’s really about.

Conclusion

At its core, Gentoo is about flexibility, not about optimizing code to run the fastest. It’s about being able to make your system whatever you want it to be, through the use of USE flags. While primarily referring to extra GCC optimizations (and not the basic CPU architecture and optimization level), the Gentoo Optimization Guide says the following:

“While CFLAGS and CXXFLAGS can be very effective means of getting source code to produce smaller and/or faster binaries, they can also impair the function of your code, bloat its size, slow down its execution time, or even cause compilation failures.

CFLAGS are not a magic bullet; they will not automatically make your system run any faster or your binaries to take up less space on disk. Adding more and more flags in an attempt to optimize your system is a sure recipe for failure. There is a point at which you will reach diminishing returns.

Despite the bragging you’ll find on the internet, aggressive CFLAGS and CXXFLAGS are far more likely to harm your programs than do them any good. Keep in mind that the reason the flags exist in the first place is because they are designed to be used at specific places for specific purposes. Just because one particular CFLAG is good for one bit of code doesn’t mean that it is suited to compiling everything you will ever install on your machine!”

So there you have it, straight from the horse’s mouth. Despite the reputation it has gathered over time due to a small minority, Gentoo is not about highly optimizing your system to get that “extra 1% performance.” Rather, it’s strength lies in the flexibility provided by its package management system.

Nevertheless, these tests do show that as a result of compiling from source, Gentoo offers some ever-so-slight additional benefits in performance (from i686 to Core2) which can be utilised to the user’s advantage. In reality however, these benefits are so tiny that they simply might not justify it (and will also make the system less portable). The combination of a specific GCC optimization level such as -Os is probably a better reason for building from source.

It’s fair to say that optimizing for a specific CPU architecture over i686 does not result in a sizeable performance increase. It was however, common to see a 20% performance increase over the i486 systems. What does make a huge increase (primarily in encoding and cryptography) is in the use of CPU instructions such as MMX and SSE. This really comes down to the coding of a particular application and whether it includes the ability to use them or not. An application built for i686 can still make use of these, thus being able to take advantage of both worlds – being a low common denominator and still making use of faster, more modern CPUs.

We must also remember that benchmarks are just benchmarks and are not necessarily a true representation of real world environments. There are lots of other factors which come into play, including the amount of other processes running and resources used on the system. This is usually where Gentoo has a major advantage, because users can easily strip out all the extras that their system doesn’t need.

So, while optimizing your computer for a specific CPU may not be everything, if you’re building from source anyway then applying some specific optimizations for your system generally isn’t going to hurt (especially if you can take advantage of CPU instruction sets). And besides, it’s a whole lot of fun!