dictionary interface for large data sets

Posted by Richard on Stack Overflow See other posts from Stack Overflow or by Richard
Published on 2010-03-31T06:51:59Z Indexed on 2010/03/31 7:13 UTC
Read the original article Hit count: 369

Filed under:

python

|

dictionary

|

dataset

|

large-files

I have a set of key/values (all text) that is too large to load in memory at once. I would like to interact with this data via a Python dictionary-like interface.

Does such a module already exist?

Reading key values should be efficient and values compressed on disk to save space.

© Stack Overflow or respective owner

Related posts about python

unmet dependencies in Ubuntu 12.04

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I tried today to install a dvb-card on my Ubuntu 12.04 (Linux blauhai-linux 3.2.0-25-generic #40-Ubuntu SMP Wed May 23 20:30:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux ). The installation failed with an error. After that, i tried to install python (it was already installed but i got this error): linux:~$… >>> More
How can I get sikuli-ide to work?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I installed sikuli-ide with sudo apt-get install sikuli-ide Everything was fine until I tried to start it from the terminal. I typed sikuli-ide But the only response I got was [info] locale: en_US The application was not started, furthermore there is no desktop file and sikuli-ide does not… >>> More
Getting PATH right for python after MacPorts install

as seen on Super User - Search for 'Super User'
I can't import some python libraries (PIL, psycopg2) that I just installed with MacPorts. I looked through these forums, and tried to adjust my PATH variable in $HOME/.bash_profile in order to fix this but it did not work. I added the location of PIL and psycopg2 to PATH. I know that Terminal is… >>> More
call python with system() in R to run a python script emulating the python console

as seen on Stack Overflow - Search for 'Stack Overflow'
I want to pass a chunk of Python code to Python in R with something like system('python ...'), and I'm wondering if there is an easy way to emulate the python console in this case. For example, suppose the code is "print 'hello world'", how can I get the output like this in R? >>> print… >>> More
Python - Calling a non python program from python?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am currently struggling to call a non python program from a python script. I have a ~1000 files that when passed through this C++ program will generate ~1000 outputs. Each output file must have a distinct name. The command I wish to run is of the form: program_name -input -output -o1 -o2… >>> More

Related posts about dictionary

Reflective Generic Detection

as seen on Stack Overflow - Search for 'Stack Overflow'
Trying to find out if a provided Type is of a given generic type (with any generic types inside) Let me Explain: bool IsOfGenericType(Type baseType, Type sampleType) { /// ... } Such that: IsOfGenericType(typeof(Dictionary<,>), typeof(Dictionary<string, int>)); // True IsOfGenericType(typeof(IDictionary<… >>> More
delta-dictionary/dictionary with revision awareness in python?

as seen on Stack Overflow - Search for 'Stack Overflow'
I am looking to create a dictionary with 'roll-back' capabilities in python. The dictionary would start with a revision number of 0, and the revision would be bumped up only by explicit method call. I do not need to delete keys, only add and update key,value pairs, and then roll back. I will never… >>> More
Dictionary<string,string> to Dictionary<Control,object> using IEnumerable<T>.Select()

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a System.Collections.Generic.Dictionary<string, string> containing control ID and appropriate data column to data bind: var dic = new Dictionary<string, string> { { "Label1", "FooCount" }, { "Label2", "BarCount" } }; I use it that way: var row = ((DataRowView)FormView1… >>> More
Windows Phone 7: Building a simple dictionary web client

as seen on Geeks with Blogs - Search for 'Geeks with Blogs'
Like I mentioned in this post a while back, I came across a dictionary web service called Aonaware that serves up word definitions from various dictionaries and is really easy to use. The services page on their website, http://services.aonaware.com/DictService/DictService.asmx, lists all the operations… >>> More
Java: If vs. Switch

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a piece of code with a) which I replaced with b) purely for legibility ... a) if ( WORD[ INDEX ] == 'A' ) branch = BRANCH.A; /* B through to Y */ if ( WORD[ INDEX ] == 'Z' ) branch = BRANCH.Z; b) switch ( WORD[ INDEX ] ) { case 'A' : branch = BRANCH.A; break; /* B through to Y… >>> More