I have a large amount of data taken from a number of different databases/mailing lists. I want to pull all of this data into a common format so that I can read it into a spreadsheet. What kinds of tools does Linux have to help me sort through all this data? I know Perl is good at text processing, but I don’t want to write a whole program.
I have a large amount of data taken from a number of different databases/mailing lists. I want to pull all of this data into a common format so that I can read it into a spreadsheet. What kinds of tools does Linux have to help me sort through all this data? I know Perl is good at text processing, but I don’t want to write a whole program.
When it comes to text processing, two of the most basic and powerful tools are built into the shell; they are available right from the command line — redirection and pipes. These let you string together a number of smaller commands, which allow you to create very flexible and powerful text processing systems. To shed some light on your question, let’s look at redirection and pipes and the Linux/Unix utilities sed, awk, grep, and cut.
To understand how redirection and pipes work, it is necessary to have a very basic understanding of the Unix and Linux concept of a “file.” Under Unix (and Linux), everything is considered to be a “file.” There are really only two basic activities associated with files: you can read from them and you can write to them. Executable programs are files, and so are text files. The interesting thing is that devices (such as your hard disk, keyboard, and monitor) are also considered files. Viewing all of these things as files means that even a small program can contain code that allows it…
Please log in to view this content.
Not Yet a Member?
Register with LinuxMagazine.com and get free access to the entire archive, including:
