Selecting keys based on metadata, possible with Amazon S3?

Posted by nbv4 on Stack Overflow See other posts from Stack Overflow or by nbv4
Published on 2010-05-06T07:21:08Z Indexed on 2010/05/06 7:28 UTC
Read the original article Hit count: 249

Filed under:

amazon-s3

|

boto

|

aws

I'm sending files to my S3 bucket that are basically gzipped database dumps. They keys are a human readable date ("2010-05-04.dump"), and along with that, I'm setting a metadata field to the UNIX time of the dump.

I want to write a script that retrieve the latest dump from the bucket. That is to say I want the the key with the largest unix time metadata value. Is this possible with Amazon S3, or is this not how S3 is meant to work?

I'm using both the command line tool aws, and the python library boto

© Stack Overflow or respective owner

Related posts about amazon-s3

Amazon S3 Tips: Quickly Add/Modify HTTP Headers To All Files Recursively

as seen on Tech Dreams - Search for 'Tech Dreams'
Amazon S3 is an dead cheap cloud storage service that offers unlimited storage in pay as you use model. Recently we moved all the images and other static files(scripts & css) of Tech Dreams to Amazon S3 to reduce load on VPS server. Amazon S3 is cheap, but monthly bill will shoot up if images/static… >>> More
Alternative to Amazon's S3 service?

as seen on Super User - Search for 'Super User'
Just wondering if there is good alternative to Amazon's S3 service? I like S3 but the bandwidth cost is high. I looked at CouldFiles from Rackspace but the cost is even higher. I don't mind prepaying or having monthly payment in order to reduce the bandwidth cost greatly. Thank you for any help >>> More
Alternative to Amazon’s S3 service?

as seen on Stack Overflow - Search for 'Stack Overflow'
Just wondering if there is good alternative to Amazon's S3 service? I like S3 but the bandwidth cost is high. I looked at CouldFiles from Rackspace but the cost is even higher. I don't mind prepaying or having monthly payment in order to reduce the bandwidth cost greatly. Thank you for any help >>> More
Saving Python Complex Data Types to Amazon S3

as seen on Stack Overflow - Search for 'Stack Overflow'
Can Python class data be saved to S3 without marshalling? I am trying to cut down of I/O operations until necessary. >>> More
Filesize with SWFUpload and Amazon S3

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello all, I'm currently using SWFUpload to upload files to my S3 bucket. And it's working great. I'm using the script from a website here: http://www.anedix.com/news/article/50 Again, the upload to my S3 works fine, however, I've been running into an issue when attempting to upload larger… >>> More

Related posts about boto

Import boto from local library

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to use boto as a downloaded library, rather than installing it globally on my machine. I'm able to import boto, but when I run boto.connect_dynamodb() I get an error: ImportError: No module named dynamodb.layer2 Here's my file structure: project/ project/ __init__.py … >>> More
Problem importing moduls from boto

as seen on Stack Overflow - Search for 'Stack Overflow'
I have installed boto like so: python setup.py install; and then when I launch my python script (that imports moduls from boto) on shell, an error like this shows up: ImportError: No module named boto.s3.connection How to settle the matter? >>> More
Problem importing modul2s from boto

as seen on Stack Overflow - Search for 'Stack Overflow'
I have installed boto like so: python setup.py install; and then when I launch my python script (that imports moduls from boto) on shell, an error like this shows up: ImportError: No module named boto.s3.connection How to settle the matter? >>> More
finding last snapshot using boto

as seen on Stack Overflow - Search for 'Stack Overflow'
I have read the explanation about "describe_cluster_snapshots" from ... http://docs.pythonboto.org/en/latest/ref/redshift.html#boto.redshift.layer1.RedshiftConnection.create_cluster It has an option start_time and end_time but there is no way to sort it. How do I get the id of the latest snapshot… >>> More
Eclipse and python: library will import in interprer, but not in IDE

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm running Windows 7, Python 2.6.4 and the latest version of Eclipse. I downloaded the boto library (http://code.google.com/p/boto/) and ran python setup.py install, which created boto-1.9b-py2.6.egg in C:\Python26\Lib\site-packages. Importing a class - say, by doing 'from boto.sqs.connection import… >>> More