Copy large files to multiple machines on a LAN

Posted by Jonathan Callen on Server Fault See other posts from Server Fault or by Jonathan Callen
Published on 2011-11-18T13:23:02Z Indexed on 2011/11/18 17:54 UTC
Read the original article Hit count: 230

Filed under:

linux

|

file-sharing

I have a few large files that I need to copy from one Linux machine to about 20 other Linux machines, all on the same LAN as quickly as is feasible. What tools/methods would be best for copying these files, noting that this is not going to be a one-time copy. These machines will never be connected to the Internet, and security is not an issue.

Update:

The reason for my asking this is because (as I understand it) we are currently using scp in serial to copy the files to each of the machines and I have been informed that this is "too slow" and a faster alternative is being sought. According to what I have been told, attempting to parallelize the scp calls simply slows it down further due to hard drive seeks.

© Server Fault or respective owner

Related posts about linux

apt-get install and update fail

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've got a problem with apt-get update and apt-get install ... commands . every time update or installing fails and errors are : Get:1 http://dl.google.com stable Release.gpg [198B] Ign http://dl.google.com/linux/chrome/deb/ stable/main Translation-en_US Get:2 http://dl… >>> More
kernel module compiling error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
sh@ubuntu:/home/ccpp/helloworld$ make gcc-4.6 -O2 -DMODULE -D_KERNEL_ -W -Wall -Wstrict-prototypes -Wmissing-prototypes -isystem /lib/modules/`uname -r`/build/include -c -o hello-1.o hello-1.c hello-1.c:4:0: warning: "MODULE" redefined [enabled by default] <command-line>:0:0: note: this is… >>> More
Build-Essentials installation failing

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am having trouble accessing the several critical header files that show to be a part of the build process. The "Ubuntu Software Center" shows "Build Essentials" as installed: Next I did the following two commands, which did not improve the problem: ~$ sudo apt-get install build-essential [sudo]… >>> More
Updating Debian kernel

as seen on Super User - Search for 'Super User'
I'm trying to update my Debian machine to 2.6.32-46 (which is the new stable). However, after doing apt-get update my apt-cache search linux-image shows me: linux-headers-2.6.32-5-486 - Header files for Linux 2.6.32-5-486 linux-headers-2.6.32-5-686-bigmem - Header files for Linux 2.6.32-5-686-bigmem linux-headers-2… >>> More
Serial connection over a single USB cable (Windows to linux, or linux to linux)

as seen on Server Fault - Search for 'Server Fault'
I'm helping out with a project for an embedded device that only has USB and no serial. This device is running Linux. These days, when we need to connect to a serial port on a device we typically use a USB to serial adapter (on something like a phone system or a load balancing device, etc). I would… >>> More

Related posts about file-sharing

Windows 7 file sharing password protecting or making stuff available to just me

as seen on Super User - Search for 'Super User'
Even with the new Homegroup feature I'm still finding the way Windows deals with folder sharing utterly baffling. Here's what I want to do. I have two computers, a PC Desktop and a laptop. I also live in a shared flat with other computer users. I have set up a Homegroup and a Workgroup on the… >>> More
File sharing service with an API?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello! Is there a file sharing service akin to Dropbox or SugarSync that would offer an API? I would like to add a file sharing service to a website, but the full Dropbox interface is still too complicated for my users, so that I’d like an API to build a dead-simple list of files integrated to the… >>> More
How to Configure Windows Machine to Allow File Sharing with DNS Alias

as seen on Server Fault - Search for 'Server Fault'
I have not seen a single article posted anywhere online that brings together all the settings one would need to do to make this work properly on Windows, so I thought I would post it here. To facilitate failover schemes, a common technique is to use DNS CNAME records (DNS Aliases) for different machine… >>> More
How to securely enable file sharing over PPTP on Windows 2008 Server

as seen on Server Fault - Search for 'Server Fault'
I have set up a Windows SBS 2003 domain (LAN) and a stand-alone Windows 2008 Server (web server) at another location (workgroup). I established PPTP VPN connection (SBS dials web server) over which users from LAN should be able to access web server. On web server I enabled TCP/IPv4 and File and Printer… >>> More
Enable SMB file sharing on OS X - "Incorrect Password"

as seen on Super User - Search for 'Super User'
I have a Mac running Snow Leopard connected to an Active Directory domain. I can share folders on the Mac and view files from Windows without problems. When I try to enable my Mac account for write access through System Preferences, I'm prompted for my account's password. Even though I'm entering… >>> More