Why would HTTP transfer via wget be faster than lftp/pget?

Posted by jondahl on Stack Overflow See other posts from Stack Overflow or by jondahl
Published on 2010-05-03T21:38:41Z Indexed on 2010/05/03 22:18 UTC
Read the original article Hit count: 601

Filed under:
|
|
|

I'm building software that needs to do massive amounts of file transfer via both HTTP and FTP. Often times, I get faster HTTP download with a multi-connection download accelerator like axel or lftp with pget. In some cases, I've seen 2x-3x faster file transfer using something like:

axel http://example.com/somefile

or

lftp -e 'pget -n 5 http://example.com/somefile;quit'

vs. just using wget:

wget http://example.com/somefile

But other times, wget is significantly faster than lftp. Strangly, this is even true even when I do lftp with get, like so:

lftp -e 'pget -n 1 http://example.com/somefile;quit'

I understand that downloading a file via multiple connections won't always result in a speedup, depending on how bandwidth is constrained. But: why would it be slower? Especially when calling lftp/pget with -n 1?

© Stack Overflow or respective owner

Related posts about http

Related posts about download