Gentoo: Ten Years Emerge

Gentoo, the most popular source based distribution, has turned ten years old. What benefits does a such a system provide over its binary distros? As Linux becomes more and more popular, is there still a place for source based distros?

When Linus first released the kernel onto the Internet, it wasn’t a pre-compiled binary that he provided, but rather the source code. Software has its origins in the source and it’s what free software is all about!

In fact, many free software projects still provide their primary releases in source code form, leaving it up to distribution package managers to compile and ship the binaries with their operating systems.

Building applications from source code is actually not as foreign as it might seem, indeed it is integral to a free desktop. So why then do so many users balk at the idea of a source based distribution?

The Way Things Are

Certainly the majority of the top Linux distributions are binary based – that is to say that the package manager delivers applications to the machine pre-compiled from source code, in binary form. All of these distributions have the ability for users to build applications from source and, to varying degrees of ease, permit the re-building of installed applications to the user’s requirements.

Debian based distros can pull down the source code of any package available in the repository with a single command. Along side its binary package manager, Arch Linux has a complete ports style system that allows a user to easily re-build any package and install it onto the computer.

The major benefit of a binary based distro is speed of installation. All of the packages available are able to be installed as quickly as they can be downloaded and extracted to the hard drive.

The Secret is in the Source

When using a binary distribution however, the package managers have made several core decisions for you. They’ve decided the architecture that the package is compiled for, which is usually i386 or generic x86_64. They have also decided what optimizations are applied to the code when compiled via the GNU Compiler Collection (GCC).

They have decided which features will be built into the program, such as Bluetooth support, OpenGL or zero configuration networking. When you install one of these binaries, it will pull in whatever libraries the developer has decided will be included. Generally speaking, these tend to be very broad as the binary must work on as many computers and provide as many features as possible.

Imagine if you were able to make these choices for yourself! Well, you can. DistroWatch lists over half a dozen source based distributions, including Linux From Scratch and of course Gentoo itself, which is perhaps the most popular.

Recently Gentoo celebrated its tenth anniversary, with a special live DVD release. So what has made this distribution stand the test of time? Obviously it must offer some benefit to a large number of people or it would have died out.

You’ve Gotta Have a System

Gentoo’s package management system is ports based and consists of build scripts for each available program, called Ebuilds. The Ebuilds, patches and other scripts sit inside what is called the Portage tree.

The installation of a package under Gentoo is completely automated and generally as simple as any other distro. Installing an application like Vim is as easy as:

emerge vim

These Portage tools, such as emerge, are very powerful and automate the entire build and installation process. An added benefit is the ability to install a particular version of a package you require. The Portage tree often contains several versions of a package and although a specific one would be recommended, a user can install whichever they please. Creating your own Portage tree is also as simple as creating or modifying an existing Ebuild to suit your needs.

Portage isn’t just about the source however, it also supports the installation of binary packages. A user could connect to an online repository and install applications from there. Or they could create binary packages from the applications they have already installed and use them on other machines, all via Portage.

What this means is that a user can easily build their own custom system and distribute that onto as many other computers as they like, from where it would do so at the speed of a regular binary distro.

Benefits

That’s the main benefit of a source based distribution, flexibility. The user can turn the system into whatever they want it to be, making the decisions themselves rather than the package managers, as with other distros.

Don’t want support for Bluetooth, zero configuration networking, OpenGL, Pulseaudio, or even GTK? It can all be done, and Gentoo makes it easy.

When building a system from scratch for a specific machine, one needn’t be concerned with the lowest common denominator. The system can be tuned precisely for that system. PowerPC systems aside, most 32-bit Linux distributions build their code for an Intel 386 processor. They do this because computers since then are a subset of that original and are therefore backwards compatible. Even if you have a more modern processor such as an Intel Pentium 4 or even an AMD Athlon, they are all x86 CPUs.

Newer x86 CPUs include more advanced instruction sets such as mmx, sse and 3dnow. A system built for an i386 processor will not be able to take advantage of these, but if it were optimized for a specific later processor then it could.

Code is often optimized for a specific processor when compiled, so when building from source the possibility opens up to optimize for something more modern. The GNU C compiler can, of course, optimize code for any such processor, including Intel Core 2 and AMD64 processors in 64-bit. If you are building Linux for a low-end CPU such as Atom, C3, or Geode, then compiling each and every binary on your system to take advantage of that specific CPU can be highly beneficial.

Pick and Choose Features

The settings for building packages on the Gentoo system revolves around a central make configuration file called, make.conf. Here the user can specify what processor they want to build the system for, what level of optimization they want GCC to use and what features they want included in the system. For example, one sets the GCC optimization level such as -O2 or -Os, along with the CPU such as -march=core2.

When compiling an application, these settings will automatically be used and therefore optimize the entire operating system for that processor. The resulting binaries may not be compatible with older machines however, as they enable all instruction sets for the specified CPU.

The features each package will support is handled via the “USE” variable, the values for which are called “use flags.” If the system requires support for Bluetooth, then adding “bluetooth” to the USE flag variable will ensure that for any applications where bluetooth support is available, they will get it built in.

Likewise, the “-bluetooth” value will remove support for Bluetooth from any package which could have made use of it, such as GNOME.

The result is a more lean system, which only includes support for the features wanted on the machine.

The Downside?

Comments on "Gentoo: Ten Years Emerge"

ntwoo

The great thing about Gentoo is that it turns a pig slow machine into something useful again. This is most probably the handiest aspect of Gentoo imo. My previous machine, an Athlon 2000+ was at the point of retirement and got an extra year of life due to Gentoo.

My Acer Aspire one Laptop with Gentoo onboard is nice and snappy! The latest versions of X are always available and certainly improve the battery life. The latest kernels have more stable wifi drivers. All in all an absolute must for such a system.

There is another major advantage of source based distro\’s. You never ever have to upgrade your distro again! In Gentoo, new libs? emerge libs and revdep-rebuild. Now all the applications that used the old libraries are rebuilt with the new libraries. This system is really the beauty of Gentoo. One install for the entire lifecycle of a machine. Always the latest and greatest software. With binary distro\’s you always have some killer app after a number of months that you want to run. Get an rpm, oops requires an update of some lib. Get the latest lib, oops requires some other update of a lib. This lib now requires an update of libc!! Upgrade… Thank goodness those times are gone!

Reply
dragonwisard

ntwoo, but you pay a price for that level of configurability. How long did it take to emerge X? How long to revdep-rebuild after upgrading some major libs (like libstdc++)?

That\’s why I run Slackware on most of my systems. You start with a fast and stable binary distro and only recompile the packages you have reason to customize.

Running the bleeding-edge versions of every package might be exciting, but it doesn\’t make for a very reliable system.

I have great respect for Gentoo, but I think it\’s overkill for most practical use cases.

Reply
typhoidmary

This article leaves out one of the biggest pains in compiling your own code. This is dealing with complex interdependencies. Want and example, try compiling ffmpeg with all the options. If you get far enough into it, you realize that some of the libraries needed to do this have cross dependencies that interfere with your compilation.

One of the great benefits of a distro is that someone else has resolved this complexity for you. This can mean the difference between getting something to work and not.

So what I want to know about Gentoo, is how is this dealt with? Is it possible to compile a package like ffmpeg with all the options, or will it bog down part way through? (Like I did.)

Reply
lescoke

I use gentoo for several reasons:

1. I\’m not running the latest hardware, so compiling for the specific processor gives me the best performance.
2. I use encryption on all my hard drives. I installed into fully encrypted partitions; most other distro\’s require installing first then moving to encrypted partitions. If I screw-up the kernel build and it doesn\’t boot, I can always boot the minimal live cd and decrypt / mount everything and try the build again.
3. The incremental updates over time, I always have a system that is as up to date as I want it to be.

Hey Typhoid; There have been some recent package blockages that seemed like circular dependencies, but what happed was a package was moved from one category to another: dev-misc to dev-lang. realizing that, the blocking package could be unmerged and then the emerge would proceed. The only other circular dependency I know of is when building some of the libraries for a package like ffmpeg, but if you follow the guide it recommends emerging a couple of the libs with restrictive use flags first, then again later with them the way you want. A lot of times this is spelled out in ewarn messages during the emerge.

My only complaint is that sometimes a package can have 100\’s of dependencies, and the scroll back on my terminal is limited, so I miss some of the ewarn messages. Then there is the case where something fails in the middle, so you have to change something and resume; I feel like I miss some important ewarn messages, so I have to go back through the emerge logs to make certain.

Reply
csmart

@dragonwisard: True, however you can build the system on a faster machine and use those optimised binaries on your low end machines. This way you can make use of older hardware for longer without the downside of compiling on them.

@typhoidmary: Gentoo\’s use flags should take care of that for you. When you emerge ffmpeg it lists all the possible flags which will provide all the features. Specifying them all should then pull in all required deps and build them in order. It should work correctly every time.

-c

Reply
ntwoo

@dragonwisard: Don\’t know, I don\’t time all my builds. Usually I start big updates in the night. Modular X builds much faster than the old versions. Firefox and Ooffice are installed as binaries, since these have their compiler flags masked in any case. I don\’t run all the stuff bleeding edge, only the components I need bleeding edge. (such as the kernel, X, inkscape, some libs for kde4)

Reply
bapun007

gentoo boots faster then ubuntu , fedora , arch ,debian and others and thats the benefit of making software from source .

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>