find the top K most frequent numbers in a data stream

Posted by Jin on Stack Overflow See other posts from Stack Overflow or by Jin
Published on 2013-06-30T04:16:22Z Indexed on 2013/06/30 4:21 UTC
Read the original article Hit count: 139

Filed under:

data-structures

|

data-stream

This is more of a data structure question rather than a coding question.

If I am fetching a data stream, i.e, I keep receiving float numbers once at a time, how should I keep track of the top K frequent numbers? Here my memory is 4G and I prefer to have less communication with hard drive unless necessary.

I think heap is good for updating the max and min. How should I design the data structure? Thanks

© Stack Overflow or respective owner

Related posts about data-structures

Clever ways of implementing different data structures in C & data structures that should be used mor

as seen on Stack Overflow - Search for 'Stack Overflow'
What are some clever (not ordinary) ways of implementing data structures in C, and what are some data structures that should be used more often? For example, what is the most effective way (generating minimal overhead) to implement a directed and cyclic graph with weighted edges in C? I know that… >>> More
Is there a way to track data structure dependencies from the database, through the tiers, all the way out to a web page?

as seen on Programmers - Search for 'Programmers'
When we design applications, we generally end up with the same tiered sets of data structures: A persistent data structure that is described using DDL and implemented as RDBMS tables and columns. A set of domain objects that consist primarily of data structures, usually combined with business-rule… >>> More
Why are data structures so important in interviews?

as seen on Programmers - Search for 'Programmers'
I am a newbie into the corporate world recently graduated in computers. I am a java/groovy developer. I am a quick learner and I can learn new frameworks, APIs or even programming languages within considerably short amount of time. Albeit that, I must confess that I was not so strong in data structures… >>> More
Thread-safe data structures

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, I have to design a data structure that is to be used in a multi-threaded environment. The basic API is simple: insert element, remove element, retrieve element, check that element exists. The structure's implementation uses implicit locking to guarantee the atomicity of a single API call.… >>> More
Data Structures

as seen on Stack Overflow - Search for 'Stack Overflow'
There is a large stream of numbers coming in such as 5 6 7 2 3 1 2 3 .. What kind of data structure is suitable for this problem given the constraints that elements must be inserted in descending order and duplicates should be eliminated. I am not looking for any code just ideas? I was thinking… >>> More

Related posts about data-stream

C# Process flow - Datastream, XML and datagrid

as seen on Stack Overflow - Search for 'Stack Overflow'
Im looking for some advice/suggestions on how I should setup the work flow of a small application Im building. When the application is launched the datagrid will be populated via the XML file. Once running the application will receive a datastream that I hope to update the file and datagrid. So… >>> More
Elegant Algorithm for Parsing Data Stream Into Record

as seen on Stack Overflow - Search for 'Stack Overflow'
I am interfacing with a hardware device that streams data to my app over Wifi. The data is streaming in just fine. The data contains a character header (DATA:) that indicates a new record has begun. The issues is that the data I receive doesn't necessarily fall on the header boundary, so I have to… >>> More
IStream interface of the data stream to write DVD

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I want t write a directory to DVD. How to say this directory as IStream. HRESULT Write( [in] IStream *data); >>> More
Detecting periodic repetitions in the data stream

as seen on Stack Overflow - Search for 'Stack Overflow'
Let's say I have an array of zeros: a = numpy.zeros(1000) I then introduce some repetitive 'events': a[range(0, 1000, 30)] = 1 Question is, how do I detect the 'signal' there? Because it's far from the ideal signal if I do the 'regular' FFT I don't get a clear indication of where my 'true' signal… >>> More
How to receive a datastream from a device on your computer, in C#

as seen on Stack Overflow - Search for 'Stack Overflow'
I plan to build a small audio-recorder app in C#. My laptop has a built in Microphone that's always active, so I want to use that as an early-stage test. I would simply start recording, save the file as a .wav or even use the LAME dll to make it into an MP3. The problem is, I don't know how to contact… >>> More