Transfer Tips, Part II

Jerry Peek

Tuesday, April 15th, 2003

Share This:

Last month’s column (available online at http://www.linux-mag.com/2003-03/power_01.html) presented a number of ways to transfer files by hand with ssh and scp and the power of the shell. This month’s column looks at ways to transfer and synchronize sets of files automatically with ftp, sftp, and wget. Along the way, you’ll also see tips on compressed ssh and detached processes. Let’s dig in!

Transfer Tips Part I, Continued

First, a note about last month’s column, where we used gzip and bzip2 to compress data transferred through an ssh connection. Separate compression and decompression steps are often unnecessary because many ssh servers have compression built-in.

For example, if the ssh server on host foo supports compression, the command scp -C afile foo: compresses afile, transfers the compressed (intermediate) file to foo, and then uncompresses it into your home directory on foo. The -C option enables compression.

In the same way, the command ssh -C foo compresses all traffic traversing the interactive connection to foo. In fact, compression even works with X forwarding (tunneling X Window System connections through ssh), boosting the performance of tunneled X applications.

ssh compression is most useful over slow network connections like dialup modems. However, if you’re transferring a lot of small files over a slow link, network latency can defeat the time savings from scp -C. That’s because each file has to be negotiated and sent separately — and that “overhead” time can build up. Instead, pipeline the transfer…

1. Got Security? You're in Denial
2. KDE 4.4: Does It Work Yet?
3. Writing Custom Nagios Plugins with Python
4. Power Up Linux GUI Apps
5. Tweeting from the Command Line with Twyt