Copying large number of files from one directory to another in linux

Posted by Ritesh Sharma on Super User See other posts from Super User or by Ritesh Sharma
Published on 2010-02-10T14:10:26Z Indexed on 2010/06/14 23:33 UTC
Read the original article Hit count: 288

Filed under:

linux

|

copy

|

command

|

move

Hi all,

I've a directory containing around 2.8 lacs of files. I want to move them to another directory.

If I use 'cp' or 'mv' then I get an error 'argument list too long'.

If I write a script like

for file in ls *; do
   cp {source} to {destination} 
done

then because of 'ls' command , its performance degrades.

How can I do this?

© Super User or respective owner

Related posts about linux

apt-get install and update fail

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've got a problem with apt-get update and apt-get install ... commands . every time update or installing fails and errors are : Get:1 http://dl.google.com stable Release.gpg [198B] Ign http://dl.google.com/linux/chrome/deb/ stable/main Translation-en_US Get:2 http://dl… >>> More
kernel module compiling error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
sh@ubuntu:/home/ccpp/helloworld$ make gcc-4.6 -O2 -DMODULE -D_KERNEL_ -W -Wall -Wstrict-prototypes -Wmissing-prototypes -isystem /lib/modules/`uname -r`/build/include -c -o hello-1.o hello-1.c hello-1.c:4:0: warning: "MODULE" redefined [enabled by default] <command-line>:0:0: note: this is… >>> More
Build-Essentials installation failing

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am having trouble accessing the several critical header files that show to be a part of the build process. The "Ubuntu Software Center" shows "Build Essentials" as installed: Next I did the following two commands, which did not improve the problem: ~$ sudo apt-get install build-essential [sudo]… >>> More
Updating Debian kernel

as seen on Super User - Search for 'Super User'
I'm trying to update my Debian machine to 2.6.32-46 (which is the new stable). However, after doing apt-get update my apt-cache search linux-image shows me: linux-headers-2.6.32-5-486 - Header files for Linux 2.6.32-5-486 linux-headers-2.6.32-5-686-bigmem - Header files for Linux 2.6.32-5-686-bigmem linux-headers-2… >>> More
Serial connection over a single USB cable (Windows to linux, or linux to linux)

as seen on Server Fault - Search for 'Server Fault'
I'm helping out with a project for an embedded device that only has USB and no serial. This device is running Linux. These days, when we need to connect to a serial port on a device we typically use a USB to serial adapter (on something like a phone system or a load balancing device, etc). I would… >>> More

Related posts about copy

encfs error while decoding the data

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I have installed encfs and started using it to secure all my personal & office data and it was working absolutely fine until 2 hours back. The setup is like this. I have a folder in Copy folder called OfficeData which gets synchronized with my Copy folder When I login into the system I use… >>> More
XSL How to copy identical + add copy with attributes changes

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello I need to copy a node and its sub-nodes: first: one identical copy followed by a modified copy with some attributes values changed Here is the extract to change: <Configuration Name="Debug|Win32" OutputDirectory=".\Debug" IntermediateDirectory=".\Debug" ATLMinimizesCRunTimeLibraryUsage="FALSE" … >>> More
Problems with wireless file copy - hangs during copy

as seen on Super User - Search for 'Super User'
I have this problem on my home network where when I try to copy files via wireless connection, at some point the copy fails and the machine doing the copy loses connection to the network. I'm pretty much lost as to figuring out the source for the problem... Some more details that might help: It's… >>> More
Copy files in Linux, avoid the copy if files do exist in destination

as seen on Super User - Search for 'Super User'
Hi, I need to copy a /home/user folder from one hardisk to another one. It has 100000 files and around 10G size. I use cp -r /origin /destination sometines I get some errors due to broken links, permissions and so on. So I fix the error, and need to start again the copy. I wonder how could… >>> More
AMD 24 core server memory bandwidth

as seen on Server Fault - Search for 'Server Fault'
I need some help to determine whether the memory bandwidth I'm seeing under Linux on my server is normal or not. Here's the server spec: HP ProLiant DL165 G7 2x AMD Opteron 6164 HE 12-Core 40 GB RAM (10 x 4GB DDR1333) Debian 6.0 Using mbw on this server I get the following numbers: foo1:~# mbw… >>> More