Using httplib2 in python 3 properly? (Timeout problems)
        Posted  
        
            by Sho Minamimoto
        on Stack Overflow
        
        See other posts from Stack Overflow
        
            or by Sho Minamimoto
        
        
        
        Published on 2010-01-30T06:52:54Z
        Indexed on 
            2010/03/24
            1:03 UTC
        
        
        Read the original article
        Hit count: 529
        
Hey, first time post, I'm really stuck on httplib2. I've been reading up on it from diveintopython3.org, but it mentions nothing about a timeout function. I look up the documentation, but the only thing I see is an ability to put a timeout int but there are no units specified (seconds? milliseconds? What's the default if None?) This is what I have (I also have code to check what the response is and try again, but it's never tried more than once)
h = httplib2.Http('.cache', timeout=None)
for url in list:
    response, content = h.request(url)
    more stuff...
So the Http object stays around until some arbitrary time, but I'm downloading a ton of pages from the same server, and after a while, it hangs on getting a page. No errors are thrown, the thing just hangs at a page. So then I try:
h = httplib2.Http('.cache', timeout=None)
for url in list:
    try:
        response, content = h.request(url)
    except:
        h = httplib2.Http('.cache', timeout=None)
    more stuff...
But then it recreates another Http object every time (goes down the 'except' path)...I dont understand how to keep getting with the same object, until it expires and I make another. Also, is there a way to set a timeout on an individual request?
Thanks for the help!
© Stack Overflow or respective owner