Compressing and copying large files on Windows Server?

Posted by Aaron on Server Fault See other posts from Server Fault or by Aaron
Published on 2010-05-05T18:02:28Z Indexed on 2012/06/06 10:43 UTC
Read the original article Hit count: 235

I've been having a hard time copying large database backups from the database server to a test box at another site. I'm open to any ideas that would help me get this database moved without having to resort to a USB hard drive and the mail.

The database server is running Windows Server 2003 R2 Enterprise, 16 GB of RAM and two quad-core 3.0 GHz Xeon X5450s. Files are SQL Server 2005 backup files between 100 GB and 250 GB.

The pipe is not the fastest and SQL Server backup files typically compress down to 10-40% of the original, so it made sense to me to compress the files first. I've tried a number of methods, including:

  • gzip 1.2.4 (UnxUtils) and 1.3.12 (GnuWin)
  • bzip2 1.0.1 (UnxUtils) and 1.0.5 (Cygwin)
  • WinRAR 3.90
  • 7-Zip 4.65 (7za.exe)

I've attempted to use WinRAR and 7-Zip options for splitting into multiple segments. 7za.exe has worked well for me for database backups on another server, which has ~50 GB backups.

I've also tried splitting the .BAK file first with various utilities and compressing the resulting segments. No joy with that approach either- no matter the tool I've tried, it ends up butting against the size of the file.

Especially frustrating is that I've transferred files of similar size on Unix boxes without problems using rsync+ssh. Installing an SSH server is not an option for the situation I'm in, unfortunately.

For example, this is how 7-Zip dies:

H:\dbatmp>7za.exe a -t7z -v250m -mx3 h:\dbatmp\zip\db-20100419_1228.7z h:\dbatmp\db-20100419_1228.bak

7-Zip (A) 4.65  Copyright (c) 1999-2009 Igor Pavlov  2009-02-03
Scanning

Creating archive h:\dbatmp\zip\db-20100419_1228.7z

Compressing  db-20100419_1228.bak

System error:
Unspecified error

© Server Fault or respective owner

Related posts about windows-server-2003

Related posts about sql-server