linux to linux, 10TB transfer?

Posted by lostincode on Server Fault See other posts from Server Fault or by lostincode
Published on 2010-06-07T21:59:52Z Indexed on 2010/06/07 22:02 UTC
Read the original article Hit count: 536

Filed under:

linux

|

networking

|

rsync

|

file-transfer

|

copy

I've looked at all the previous similar questions, but the answers seemed to be all over the place and no one was moving a lot of data (100GB != 10TB).

I've got about 10TB that I need to move from one raid to another, gigabit net, XFS file systems. My biggest concern is having the transfer die midway and not being able to resume easily. Speed would be nice, but ensuring transfer is much more important.

Normally I'd just tar & netcat, but the raid I'm moving from has been super flaky as of late and I need to be able to recover and resume if it drops mid process. Should I be looking at rsync?

© Server Fault or respective owner

Related posts about linux

apt-get install and update fail

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've got a problem with apt-get update and apt-get install ... commands . every time update or installing fails and errors are : Get:1 http://dl.google.com stable Release.gpg [198B] Ign http://dl.google.com/linux/chrome/deb/ stable/main Translation-en_US Get:2 http://dl… >>> More
kernel module compiling error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
sh@ubuntu:/home/ccpp/helloworld$ make gcc-4.6 -O2 -DMODULE -D_KERNEL_ -W -Wall -Wstrict-prototypes -Wmissing-prototypes -isystem /lib/modules/`uname -r`/build/include -c -o hello-1.o hello-1.c hello-1.c:4:0: warning: "MODULE" redefined [enabled by default] <command-line>:0:0: note: this is… >>> More
Build-Essentials installation failing

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am having trouble accessing the several critical header files that show to be a part of the build process. The "Ubuntu Software Center" shows "Build Essentials" as installed: Next I did the following two commands, which did not improve the problem: ~$ sudo apt-get install build-essential [sudo]… >>> More
Updating Debian kernel

as seen on Super User - Search for 'Super User'
I'm trying to update my Debian machine to 2.6.32-46 (which is the new stable). However, after doing apt-get update my apt-cache search linux-image shows me: linux-headers-2.6.32-5-486 - Header files for Linux 2.6.32-5-486 linux-headers-2.6.32-5-686-bigmem - Header files for Linux 2.6.32-5-686-bigmem linux-headers-2… >>> More
Serial connection over a single USB cable (Windows to linux, or linux to linux)

as seen on Server Fault - Search for 'Server Fault'
I'm helping out with a project for an embedded device that only has USB and no serial. This device is running Linux. These days, when we need to connect to a serial port on a device we typically use a USB to serial adapter (on something like a phone system or a load balancing device, etc). I would… >>> More

Related posts about networking

Networking stopped working on Ubuntu

as seen on Super User - Search for 'Super User'
I installed Ubuntu 10.04 through the Wubi installer (Funny, I installed it today and thought I would have gotten 10.10). I had a network connection and everything was working fine. I rebooted my coumputer a couple of times and then suddenly, I could not connect to the network and when I click the… >>> More
Troubles with start up defenition of networking service in Ubuntu

as seen on Super User - Search for 'Super User'
I use Ubuntu 12.04.1. I put attention that networking script starting in runlevel 0: user@comp:/etc/rc0.d$ chkconfig -l networking networking 0:on 1:off 2:off 3:off 4:off 5:off 6:off When I try to move it working to appropriate run levels I get error: user@comp:/etc/rc0… >>> More
Installing the Elgg Social Networking CMS

as seen on Internet.com - Search for 'Internet.com'
One Open Source CMS that stands out above the rest for small businesses and personal networks is the Elgg open source "social engine," which can be used to create social applications. Scott Clark shows you how to install and get started with this social networking CMS. >>> More
OAuth: Token-Based Authorization for the Social Networking Age

as seen on Internet.com - Search for 'Internet.com'
The OAuth token-based authorization system allows users to grant third-party access to their resources without sharing their usernames and passwords. It's perfect for the age of data scattered across many social networking sites. >>> More
OAuth: Token-Based Authorization for the Social Networking Age

as seen on Internet.com - Search for 'Internet.com'
The OAuth token-based authorization system allows users to grant third-party access to their resources without sharing their usernames and passwords. It's perfect for the age of data scattered across many social networking sites. >>> More