Error while trying to parse a website url using python . how to debug it ?

Posted by mekasperasky on Stack Overflow See other posts from Stack Overflow or by mekasperasky
Published on 2010-12-21T06:51:13Z Indexed on 2010/12/21 6:54 UTC
Read the original article Hit count: 260

Filed under:
#!/usr/bin/python
import json
import urllib
from BeautifulSoup import BeautifulSoup
from BeautifulSoup import BeautifulStoneSoup
import BeautifulSoup  


def showsome(searchfor):
  query = urllib.urlencode({'q': searchfor})
  url = 'http://ajax.googleapis.com/ajax/services/search/web?v=1.0&%s' % query
  search_response = urllib.urlopen(url)
  search_results = search_response.read()
  results = json.loads(search_results)
  data = results['responseData']
  print 'Total results: %s' % data['cursor']['estimatedResultCount']
  hits = data['results']
  print 'Top %d hits:' % len(hits)
  for h in hits: 
    print ' ', h['url']
    resp = urllib.urlopen(h['url'])
        res = resp.read()

    soup = BeautifulSoup(res)
        print soup.prettify()
  print 'For more results, see %s' % data['cursor']['moreResultsUrl']

showsome('sachin')

What is the wrong in this code ?

Note all the 4 links that I am getting out of the search , I am feeding it back to extract the contents out of it , and then use BeautifulSoup to parse it . How should I go about it ?

© Stack Overflow or respective owner

Related posts about python