My preferred way of transferring a file or a directory over the network is
Receiver: nc -l 3452 | bunzip2 -vvv | tar -x
Sender: tar -c folder/ | bzip2 -vv9 | nc 127.123.0.1 3452
The benefit is that I do not need to decompress the file on the receiving file, it decompresses by itself, I do not have to set a name or a folder name, tar will create those for me. I can just as easily omit the unzip and untag and save it.
I know tar -j does compression too, but I prefer to see the terminal „doing something“ instead of staring at a blank screen.
I have seen savings of 40% of transfer time using this method and I like it a lot, even more so when moving VMs from one server to another.
Nice post.
]]>tar cz SRC_DIR | ssh root@DEST_SVR ‚cd DEST_DIR && tar xz‘
Over a crossover cable between servers it hovered at around 930 mbps.
]]>tar c directory | nc -l localhost_or_ip 7878
found on: http://superuser.com/questions/115553/netcat-on-mac-os-x
]]>scp -c arcfour256 -C -4 user@remotehost:Downloads/bigfile.iso .
These options can be made implicit by using alias or ssh config file, too
another scenario where this is a lot useful is transferring file over a VPN connection.
this way you avoid the overhead – both in bandwidth usage and in CPU time – of double encypher/decypher and you can go as fast as you can. This can make a significant difference for slow connections and/or high load of data to transfer.
decompressing the tarfile on the fly on the destination machine is even better.
]]>