SCP vs RSync vs SMB vs FTP

Blog — Dillon @ 11:55 pm

Update: as many have commented, my testing with /dev/zero might be skewing the results. I’ll be adding a random DVD iso test soon. I have a newborn so give me a sec. I’ll even add the NFS test. :)

@fearthepenguin made an rsync comment that made me curious. He said that rsync in cygwin is faster than native SMB in Windows. Ok I haven’t done this test in a while, let’s get a reminder about how fat SMB is.

Test Setup

SOHO gigabit switch. Ubuntu 9.10 and octo-core 2008 Mac Pro running 10.6. Both pretty fast boxes. Regular SATA drives in each, not very fast I/O. Whatever, network should be the bottleneck.

Get my test file generated.
$ dd if=/dev/zero of=1gb_file.zeros bs=1G count=1
1073741824 bytes (1.1 GB) copied, 133.577 s, 8.0 MB/s

Ignore that 8.0 MB/s. You can do a blocksize trick to make it output a file faster, I just didn’t feel like looking up the switches. Now you can see Mr. 1GB Zero File in all it’s empty and big glory.

$ ls -lh
total 1.2G
-rw-r--r-- 1 dude herd 1.0G 2010-02-11 12:29 1gb_file.zeros

The Tests

RSync
Copy from ubuntu box to Mac Pro:
$ time rsync -t /tmp/1gb_file.zeros dude@mac:~/tmp
real 0m17.694s
user 0m11.577s
sys 0m3.056s

SMB
Mount Mac share from Ubuntu box. Copy same file from ubuntu box to Mac Pro over SMB mount. I never do this. It’s stupid, slow, strips permissions and requires a mount. In the name of science!
# time cp /tmp/1gb_file.zeros /mnt/tmp
real 0m32.649s
user 0m0.008s
sys 0m0.568s

FTP
Ok let’s goddamn turn on OSX FTP and test that too. FTP is stupid.
The remote file "1gb_file.zeros" already exists.
Local: 1073741824 bytes, dated Thu 11 Feb 2010 12:29:06 PM EST.
(Files are identical, skipped)

Hey at least FTP is showing some smarts about it. Or maybe ncftp just rules. I dunno. Deleted it and got FTP time.

$ time ncftpput -u dude -p whoa mac /dest/tmp /tmp/1gb_file.zeros
/tmp/1gb_file.zeros: 1.00 GB 39.09 MB/s
real 0m26.507s
user 0m0.036s
sys 0m0.828s

SCP
Is SCP any different than rsync?
$ time scp /tmp/1gb_file.zeros dude@mac:~/tmp
1gb_file.zeros 100% 1024MB 42.7MB/s 00:24
real 0m24.303s
user 0m9.641s
sys 0m2.212s

Weird. It is. I wonder if rsync has compression flags whereas ssh does not without the -C magic switches. You can get SCP to be pretty quick with blowfish or arcfour:

$ time scp -c arcfour /tmp/1gb_file.zeros dude@mac:~/tmp
1gb_file.zeros 100% 1024MB 46.6MB/s 00:22
real 0m21.653s
user 0m5.452s
sys 0m2.032s

Conclusion

rsync SMB FTP SCP SCP arcfour
time 17.694 32.649 26.507 24.303 21.653
MB/sec 57.87 31.36 38.63 42.13 47.29

So RSync is pretty quick and SMB is pretty slow. @fearthepenguin was right.

7 Comments »

  1. Very good article. I sometimes compare rsync and scp. The time varies to me, depending on file. If it is text files, rsync is faster. If it is images or movies that I want to copy, scp is the winner. But is first time that I see the arcfour flag. I will check it. Thanks.

    Comment by Endless Nameless — October 19, 2010 @ 4:11 am
  2. I think rsync uses more time when on some files when it is trying to figure out if to use delta encoding or not. Which is the main pro over scp.

    Comment by Endless Nameless — December 28, 2010 @ 1:09 am
  3. Your hint with the -c arcfour saved my day, i was struggling to copy a 75GB file (which was already a LUKS-encrypted container) within my network. estimated time was something around 80 Minutes. I could not believe that transfer time was reduced to 15 Minutes due to the -c arcfour switch!

    Thanks for sharing this, you are my hero!

    Comment by James — December 6, 2011 @ 12:08 pm
  4. There is no way SCP beat FTP!!

    Something has to be wrong with your testing methods.

    FTP can saturate the wire easily, and without the additional burden of compression or encryption that comes with SCP/SFTP.

    rsync has even more computational complexities and burdens than even SCP as it is hashing the whole files, and performing a rolling-hash of the file contents, deallign with the file on two sides, etc.

    Also I noticed you did not compare NFS.

    I would suggest you create a file of random data instead of all zero. A good idea would be to use a DVD iso image you can download online, and other people can reproduce the results. You might also like to purge your filesystem buffer cache before or after each successive test run to ensure the file is not sitting in memory instead of on the disk.

    Comment by anonymous_coward — January 7, 2012 @ 3:03 am
  5. Test results using a test file created from /dev/zero will be completely bogus due to the almost perfect 100% compression ratio. (This is why SCP beat FTP.)
    Try it again with a file created from /dev/random and the results will be completely different, (and far more applicable to the real world).

    Comment by M — January 30, 2012 @ 2:52 pm
  6. I did similar tests with an MS-office CD tarball.

    Results on my network (fastest first):

    1. FTP
    2. SMB
    3. rsync -avz
    4 scp -Cv

    Comment by Tej — March 28, 2012 @ 5:44 am
  7. BTW, ‘scp -Cv -c arcfour’ edged out SMB but was still slower than simple FTP.

    Comment by Tej — March 28, 2012 @ 5:53 am

RSS feed for comments on this post. TrackBack URI

Leave a comment

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
(c) 2012 SQUARISM | powered by WordPress with Barecity