Resume recursive scp transfer (with rsync?)

Posted by vgm64 on Stack Overflow See other posts from Stack Overflow or by vgm64
Published on 2010-06-01T15:52:16Z Indexed on 2010/06/01 15:53 UTC
Read the original article Hit count: 450

Filed under:

rsync

|

scp

|

recursive

I was transferring several thousand files each ~1MB via scp and my connection was broken after the first 2k files or so. I wanted to know if there was a way to resume the recursive transfer w/o starting over. Something like

$ scp -r [email protected]:/datafiles/ ./
... Happy Transfer ...
...     BREAK!     ...
$ rsync -P [email protected]:/datafiles/ ./
... Continue transf...

The problem is I can't seem to get the syntax correct if it is possible. Can anyone shed some light on if/how it can be done?

PS. If you specify the slash after "datafiles" in the rsync line, does that transfer the directory or its contents? I saw conflicting comments when I googled.

© Stack Overflow or respective owner

Related posts about rsync

Rsync: how to mount truecrypt on-the-fly on the receiving side?

as seen on Super User - Search for 'Super User'
The short version: how can I keep an rsync backup on a truecrypt volume? The hard part is to mount/unmount this volume on the fly when it is needed for rsync. Details This is my current backup configuration (which works fairly well for the most part): backup source is on Win7 64 bit, destination… >>> More
Two Questions on for Rsync - rsync by date and by file name

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I have two questions with respect to rsync: 1: I have a bunch of files which are incremented by day of the year. Ex: file.txt.81, file.txt.82, etc. Now, these files are in different directories: data1/file.txt.81 data1/file.txt.82 data2/file2.txt.81 data2/file2.txt.82 How can I have rsync… >>> More
Cygwin rsync broken

as seen on Super User - Search for 'Super User'
I get an error with cygwin rsync trying to transfer files between local - remote host. Any ideas? C:\>rsync user@host:~/file newfile Password: rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender= 3… >>> More
macport selfupdate not working

as seen on Super User - Search for 'Super User'
macbookpro:~ eistrati$ port -v MacPorts 2.1.2 macbookpro:~ eistrati$ xcodebuild -version Xcode 4.5.2 Build version 4G2008a macbookpro:~ eistrati$ sudo port -d selfupdate DEBUG: Copying /Users/eistrati/Library/Preferences/com.apple.dt.Xcode.plist to /opt/local/var/macports/home/Library/Preferences DEBUG:… >>> More
Rsync from godaddy to OS X

as seen on Super User - Search for 'Super User'
I would like to use rsync to backup my website to my local computer (OS X). I started of with this guide and got pretty far. I use the following rsync-line: rsync -PzrlptgD --del --delete-excluded -r --rsync-path=~/bin/rsync user@server:~/ /local/backup/folder/ I wanted to use the -a option (same… >>> More

Related posts about scp

Variable directory names over SCP

as seen on Server Fault - Search for 'Server Fault'
We have a backup routine that previously ran from one disk to another on the same server, but have recently moved the source data to a remote server and are trying to replicate the job via scp. We need to run the script on the target server, and we've set up key-based scp (no username/password… >>> More
Does SCP lock the file it is transfering?

as seen on Server Fault - Search for 'Server Fault'
Have a situation where we have an application log sitting out on an AIX server. The log is being continually written to from the application, and we have users on Windows who want to view the file. What they've been doing is using WinSCP to transfer the file to their desktop and they open it using… >>> More
Problems with SCP stalling during file copy over VPN

as seen on Server Fault - Search for 'Server Fault'
I have a series of files I need to copy via SCP over a VPN to a remote linux server each night. The files are not large, we're talking about tens of megabytes here, but the file copy almost always stalls after a few seconds. Running the SCP command with -vvv, I see the following over and over throughout… >>> More
scp hangs at 100%, before finishing

as seen on Super User - Search for 'Super User'
Hi, my scp commands do hang at 100% before the finish (some seconds), The output I get using -vvv is this: http://paste.pocoo.org/show/137504/ Any ideas? >>> More
Prevent scp from copying local files?

as seen on Super User - Search for 'Super User'
I just read the Linux scp command issue question and it reminded me that I regularily forget to specify the colon in the host part of a scp command, and thus copying a file locally instead of copying to a remote host, e.g. I do scp foo host instead of scp foo host: But I never use scp to copy… >>> More