Multithreaded FTP upload. Is it possible?

Posted by Arty on Stack Overflow See other posts from Stack Overflow or by Arty
Published on 2010-04-03T08:07:09Z Indexed on 2010/04/03 8:13 UTC
Read the original article Hit count: 309

Filed under:

threading

|

ftp

|

python

|

php

|

curl

I need to upload multiple files from directory to the server via FTP and SFTP. I've solved this task for SFTP with python, paramiko and threading. But I have problem with doing it for FTP. I tried to use ftplib for python, but it seems that it doesn't support threading and I upload all files one by one, which is very slow.

I'm wondering is it even possible to do multithreading uploads with FTP protocol without creating separate connections/authorizations (it takes too long)?

Solution can be on Python or PHP. Maybe CURL? Would be grateful for any ideas.

© Stack Overflow or respective owner

Related posts about threading

WebClient error when using a thread in .NET

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm having a very weird error using the WebClient class in .NET 4. The app simply downloads some files off the internet and provides output on a textbox (the GUI is WPF). The method that does this is the following: void DownloadFiles(object files) { fileL = (List<string>) files; foreach… >>> More
Silverlight Threading and its usage

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello Experts, Scenario : I am working on LOB application, as in silverlight every call to service is Async so automatically UI is not blocked when the request is processed at server side. Silverlight also supports threading as per my understanding if you are developing LOB application threads are… >>> More
Partial Trust in WPF 4

as seen on Stack Overflow - Search for 'Stack Overflow'
I've started a new project in WPF 4 (.NET 4) and trying to see if I can run it in xbap mode. I need to run the application in Full Trust with the new mode made available in .NET 4 which asks the end user if the full trust application should be run. I've set the "Security" mode to "Full Trust" application… >>> More
TypeInitializeException on MVVM pattern

as seen on Stack Overflow - Search for 'Stack Overflow'
System.TypeInitializationException was unhandled Message=The type initializer for 'SmartHomeworkOrganizer.ViewModels.MainViewModel' threw an exception. Source=SmartHomeworkOrganizer TypeName=SmartHomeworkOrganizer.ViewModels.MainViewModel StackTrace: at SmartHomeworkOrganizer.ViewModels… >>> More
WebClient error when using a thread in .NET 4

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm having a very weird error using the WebClient class in .NET 4. The app simply downloads some files off the internet and provides output on a textbox (the GUI is WPF). The method that does this is the following: void DownloadFiles(object files) { fileL = (List<string>) files; foreach… >>> More

Related posts about ftp

sudo apt-get update problem

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I have a problem with sudo apt-get update that I can't seem to fix and It's causing problems with alot of installations I want to do. I run Ubuntu 12.04. Ign http://ftp.availo.se precise InRelease Ign http://ftp.availo.se precise-updates InRelease Ign http://ftp.availo… >>> More
Why does DEP kill IE when accessing Microsoft FTP?

as seen on Super User - Search for 'Super User'
I start up IE (9.0.8112.16421) with about:blank and I go to ftp://ftp.microsoft.com/ I press Alt, click View and then Open FTP Site in Windows Explorer. At this point IE stops responding and eventually crashes (though the window is still active, sometimes) and I get the usual Windows dialog box saying… >>> More
Shrinking TCP Window Size to 0 on Cisco ASA

as seen on Server Fault - Search for 'Server Fault'
Having an issue with any large file transfer that crosses our Cisco ASA unit come to an eventual pause. Setup Test1: Server A, FileZilla Client <- 1GBPS - Cisco ASA <- 1 GBPS - Server B, FileZilla Server TCP Window size on large transfers will drop to 0 after around 30 seconds of a large… >>> More
Shrinking Windows Size to 0 on Cisco ASA

as seen on Server Fault - Search for 'Server Fault'
Having an issue with any large file transfer that crosses our Cisco ASA unit come to an eventual pause. Setup Test1: Server A, FileZilla Client <- 1GBPS - Cisco ASA <- 1 GBPS - Server B, FileZilla Server TCP Window size on large transfers will drop to 0 after around 30 seconds of a large… >>> More
Weird FTP issue between Unity Express and Windows Server 2008 FTP

as seen on Server Fault - Search for 'Server Fault'
My VOIP specialist complained about not being able to run backups of the Unity Express onto our FTP server (Microsoft FTP on Server 2008). I did a packet trace and observed some weird behavior that I think is even kind of funny in a way. The Unity FTP client is able to initiate both control and data… >>> More