Where would an S3 upload speed cap originate?

Posted by CoreyH on Server Fault See other posts from Server Fault or by CoreyH
Published on 2010-05-12T17:27:03Z Indexed on 2010/05/12 17:34 UTC
Read the original article Hit count: 225

Filed under:

amazon-s3

|

Windows

|

networking

|

throughput

I do a ton of uploading to S3 and am experiencing capped speeds and I can't quite figure out how to address it.

The setup: Windows Server 2008 R2 x64, external HD, using a Java based upload tool called Jsh3ll and custom VBS scripts to kick the jobs off.

Running one process at a time, I am always limited to about 4mbps. I have FiOS at 35/35mbps speeds, so it isn't an outright limit. AND, I can run parallel instances and can go all the way up to 35mbps, so I know the problem isn't gateway/nic/machine/amazon related. Running parallel instances works to a degree as a solution, but increases the complexity of my workflow greatly. Solving this would make my life dramatically easier.

When I was first doing this I was playing around with a bunch of Windows TCP parameters and was able to briefly get unconstrained bandwidth, but it wasn't repeatable.

Thoughts?

© Server Fault or respective owner

Related posts about amazon-s3

Amazon S3 Tips: Quickly Add/Modify HTTP Headers To All Files Recursively

as seen on Tech Dreams - Search for 'Tech Dreams'
Amazon S3 is an dead cheap cloud storage service that offers unlimited storage in pay as you use model. Recently we moved all the images and other static files(scripts & css) of Tech Dreams to Amazon S3 to reduce load on VPS server. Amazon S3 is cheap, but monthly bill will shoot up if images/static… >>> More
Alternative to Amazon's S3 service?

as seen on Super User - Search for 'Super User'
Just wondering if there is good alternative to Amazon's S3 service? I like S3 but the bandwidth cost is high. I looked at CouldFiles from Rackspace but the cost is even higher. I don't mind prepaying or having monthly payment in order to reduce the bandwidth cost greatly. Thank you for any help >>> More
Alternative to Amazon’s S3 service?

as seen on Stack Overflow - Search for 'Stack Overflow'
Just wondering if there is good alternative to Amazon's S3 service? I like S3 but the bandwidth cost is high. I looked at CouldFiles from Rackspace but the cost is even higher. I don't mind prepaying or having monthly payment in order to reduce the bandwidth cost greatly. Thank you for any help >>> More
Saving Python Complex Data Types to Amazon S3

as seen on Stack Overflow - Search for 'Stack Overflow'
Can Python class data be saved to S3 without marshalling? I am trying to cut down of I/O operations until necessary. >>> More
Filesize with SWFUpload and Amazon S3

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello all, I'm currently using SWFUpload to upload files to my S3 bucket. And it's working great. I'm using the script from a website here: http://www.anedix.com/news/article/50 Again, the upload to my S3 works fine, however, I've been running into an issue when attempting to upload larger… >>> More

Related posts about Windows

Server 2012 DFS New Member Issue

as seen on Server Fault - Search for 'Server Fault'
I am trying to add a new member to our DFS topology. We have 3 DCs (VMs - VMware) running Windows server 2012, two servers are located in or Primary site and the third at our DR site. Currently the two servers at our primary site are currently replicating DFS (full mesh) and are working fine. I have… >>> More
error in IIS7 but not on IIS6

as seen on Server Fault - Search for 'Server Fault'
I have a website that is we are now deploying to windows 2008 servers that has worked in the past on IIS6 without a problem. It is using .net 2 framework. Most of the website works. Just when we create a screen report over a certain size on the server we get this error. Event code: 3005 Event… >>> More
error in IIS7 but not on IIS6

as seen on Server Fault - Search for 'Server Fault'
I have a website that is we are now deploying to windows 2008 servers that has worked in the past on IIS6 without a problem. It is using .net 2 framework. Most of the website works. Just when we create a screen report over a certain size on the server we get this error. Event code: 3005 Event… >>> More
Microsoft Management Console stops working when I add snap-in to it

as seen on Super User - Search for 'Super User'
I have Windows 7 Ultimate OS. I'm opening mmc.exe as administrator and trying add Certificates or any other snap-in, then while loading that snap-in MMC breaks and displays following message and after that it closes automatically once I click on close button on that message. What could be the problem… >>> More
Handling keyboard and mouse input (Win API)

as seen on Game Development - Search for 'Game Development'
There is a number of ways to catch mouse or keyboard under Windows. So I tried some of them, but every of them has some advantages and drawbacks. I want to ask you: Which method do use? I've tried these: WM_KEYDOWN/WM_KEYUP - Main disadvantage is that, I can't distinguish between left and right-handed… >>> More