Compressing and copying large files on Windows Server?
- by Aaron
I've been having a hard time copying large database backups from the database server to a test box at another site. I'm open to any ideas that would help me get this database moved without having to resort to a USB hard drive and the mail.
The database server is running Windows Server 2003 R2 Enterprise, 16 GB of RAM and two quad-core 3.0 GHz Xeon X5450s. Files are SQL Server 2005 backup files between 100 GB and 250 GB.
The pipe is not the fastest and SQL Server backup files typically compress down to 10-40% of the original, so it made sense to me to compress the files first. I've tried a number of methods, including:
gzip 1.2.4 (UnxUtils) and 1.3.12 (GnuWin)
bzip2 1.0.1 (UnxUtils) and 1.0.5 (Cygwin)
WinRAR 3.90
7-Zip 4.65 (7za.exe)
I've attempted to use WinRAR and 7-Zip options for splitting into multiple segments. 7za.exe has worked well for me for database backups on another server, which has ~50 GB backups.
I've also tried splitting the .BAK file first with various utilities and compressing the resulting segments. No joy with that approach either- no matter the tool I've tried, it ends up butting against the size of the file.
Especially frustrating is that I've transferred files of similar size on Unix boxes without problems using rsync+ssh. Installing an SSH server is not an option for the situation I'm in, unfortunately.
For example, this is how 7-Zip dies:
H:\dbatmp>7za.exe a -t7z -v250m -mx3 h:\dbatmp\zip\db-20100419_1228.7z h:\dbatmp\db-20100419_1228.bak
7-Zip (A) 4.65 Copyright (c) 1999-2009 Igor Pavlov 2009-02-03
Scanning
Creating archive h:\dbatmp\zip\db-20100419_1228.7z
Compressing db-20100419_1228.bak
System error:
Unspecified error
