Faster, secure, protocol/code required for long-distance transfer.

Posted by Chopper3 on Server Fault See other posts from Server Fault or by Chopper3
Published on 2010-05-24T10:50:47Z Indexed on 2010/05/24 11:02 UTC
Read the original article Hit count: 218

I've ran into a problem and I'm looking for a new secure protocol/client/server that's faster over a 1Gb/s fibre link - let me tell you the story...

  • I have a pair of redundant, diversely-routed, 1Gb/s links over a distance of around 250 miles or so (not dark fibre but a dedicated point to point link, not a mesh).
  • At the 'client' end I have a HP DL380 G5 (2 x dual-core 2.66Ghz Xeon's, 4GB, Windows 2003EE 32-bit), at the 'server' end I have a HP BL460c G6 (2 x quad-core 2.53Ghz Xeons, 48GB, Oracle Linux 5.3 64-bit).
  • I need to transfer around 500 x 2GB files per week from the client to the server machines per week - but the transfer NEEDS to be secure.
  • Using both iPerf or regular FTP I can get ~80MB/s of transfer pretty consistently, which is great.
  • Using WinSCP or Windows SFTP I can't seem to get more that ~3-4MB/s, at this point the server's CPU is >3% busy while CPU0 of the client goes to ~30% utilised. We've tried editing various TCP window sizes with little success.
  • Both ends are connected to quite low-usage Cisco Cat6509's with Sup720's.

  • I can replace the client machine with a newer machine and/or move it to Linux - but this will take time.

Clearly these single-threaded secure Windows clients are introducing too much latency doing their encryption.

So a few questions/thoughts;

  • Are there any higher performing secure protocols or client software for Windows that I could try? I'm pretty protocol-gnostic so long as it'll work between Windows and Linux.
  • Should I be using hardware to do the encryption, either in the client or the network parts? If so what would you recommend?
  • I'm not convinced that just swapping the server would be that much faster, the CPU was only at 30% but then again that's higher than I'd have expected given the load - moving to Linux at the client end may be a better idea but would be quite disruptive.
  • Am I missing a trick?

Thanks in advance.

© Server Fault or respective owner

Related posts about file-transfer

Related posts about secure