Waiting for thread to finish Python
        Posted  
        
            by 
                lunchtime
            
        on Stack Overflow
        
        See other posts from Stack Overflow
        
            or by lunchtime
        
        
        
        Published on 2014-06-08T02:54:08Z
        Indexed on 
            2014/06/08
            3:25 UTC
        
        
        Read the original article
        Hit count: 213
        
Alright, here's my problem.
I have a thread that creates another thread in a pool, applies async so I can work with the returned data, which is working GREAT. But I need the current thread to WAIT until the result is returned.
Here is the simplified code, as the current script is over 300 lines. I'm sure i've included everything for you to make sense of what I'm attempting:
from multiprocessing.pool import ThreadPool
import threading
pool = ThreadPool(processes=1)
class MyStreamer(TwythonStreamer):
#[...]
    def on_success(self, data): #### Everytime data comes in, this is called
    #[...]
#<Pseudocode> 
    if score >= limit
        if list exists: Do stuff
        elif list does not exist:              
#</Pseudocode>
            dic = []
            dic.append([k1, v1])
            did = dict(dic)
            async_result = pool.apply_async(self.list_step, args=(did))
            return_val = async_result.get()
            slug = return_val[0]
            idd = return_val[1]
#[...]
    def list_step(self, *args):
            ## CREATE LIST
            ## RETURN 2 VALUES
class threadStream (threading.Thread):
    def __init__(self, auth):
        threading.Thread.__init__(self)
        self.auth = auth
    def run(self):
        stream = MyStreamer(auth = auth[0], *auth[0])
        stream.statuses.filter(track=auth[1])
t = threadStream(auth=AuthMe)
t.start()
I receive the results as intended, which is great, but how do I make it so this thread t waits for the async_result to come in?? My problem is everytime new data comes in, it seems that the ## CREATE LIST function is called multiple times if similar data comes in quickly enough. So I'm ending up with many lists of the same name when I have code in place to ensure that a list will never be created if the name already exists. 
So to reiterate: How do I make this thread wait on the function to complete before accepting new data / continuing. I don't think time.sleep() works because on_success is called when data enters the stream. I don't think Thread.Join() will work either since I have to use a ThreadPool.apply_async to receive the data I need. Is there a hack I can make in the MyStreamer class somehow? I'm kind of at a loss here. Am I over complicating things and can this be simplified to do what I want? 
© Stack Overflow or respective owner