Scraping Google docs (can't use API)

Posted by Andy Waite on Stack Overflow See other posts from Stack Overflow or by Andy Waite
Published on 2010-05-29T14:49:18Z Indexed on 2010/05/29 14:52 UTC
Read the original article Hit count: 238

Filed under:

screen-scraping

|

google-docs

|

google-data-api

I'm building an iPhone app which needs a peice of meta data from a user's Google Spreadsheet. Unfortunately the meta data I need is not exposed by the API, so I will need to scrape it from the document's HTML source (it would not be present in any of the exported variants).

Is there anyway to include authentication parameters in a call such as:

http://spreadsheets.google.com/ccc?key=abc123&username=...&password=...

© Stack Overflow or respective owner

Related posts about screen-scraping

PHP Screen Scraping Class

as seen on Bradino - Search for 'Bradino'
After some positive feedback I have decided to continue to develop the PHP Screen Scraping class. This post will server as the permanent home for the class. Download PHP Screen Scraping Class Updates 20009-07-30 Added setHeader() function >>> More
Screen scraping over SSL with .NET

as seen on Stack Overflow - Search for 'Stack Overflow'
What solutions exist for screen scraping a site over SSL for use with .NET? My use case is that I need to login to a partner website (https), navigate through a dynamic hierarchy, and download a zipped file of reports. I certainly could use other screen scrapers if there are no good viable options… >>> More
looking for alternative to Webzinc .NET , screen scraping, web automation library for .net

as seen on Stack Overflow - Search for 'Stack Overflow'
i came across this .net library http://www.webzinc.com/online/faq.aspx however, i was wondering if there was a free alternative out there ? >>> More
Screen-scraping of a secure page of any site on https:// with asp.net in C#

as seen on Stack Overflow - Search for 'Stack Overflow'
I've done site scraping of secure page of any site on http:// but when I am trying to scrap any site on https:// then i always scrape the login page not secure page. Please advice what should i do for scraping a secure page of any site on https://. >>> More
How different is mashup from screenscraping and consuming webservices

as seen on Stack Overflow - Search for 'Stack Overflow'
From what I understand, Mashup is aggregating data from separate sources and providing a single view. How different is mashup when compared to screenscraping or using webservices to get data from external sources? >>> More

Related posts about google-docs

Remove 'Google Chrome' folder on Google Docs

as seen on Super User - Search for 'Super User'
I used to sync my Chrome bookmark, and Chrome create a folder named Google Chrome in my Google Docs Now I want to delete it. Is there any way to do? >>> More
Alternative to google docs?

as seen on Super User - Search for 'Super User'
Any alternative that you know? Because google docs really sucks, it wont let you upload powerpoint or any document file which is higher than 3 mb. Please, thanks in advance. >>> More
Direct URL to a Google docs revision

as seen on Super User - Search for 'Super User'
I have a Google doc document. Using Edit-File-See Revision History I can see older revisions and go to them. I want to give some-one a URL which points directly to a specific revision of my document which is not the current one. How do I do this? (Note: The link from the revision list does not work… >>> More
Map a GoogleDocs Spreadsheet to a SQL table

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm looking for a generalized plan of attack at this problem: Upon saving a spreadsheet in my Google Docs account a SQL 08 table is updated. Or perhaps via spreadsheet's new 'Form' input method. But the overarching goal is letting google docs function as a front-end editor to sql. thx --steve... >>> More
Google Docs as a Drive - Maximum number of images on 200GB account

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi all, have you upgraded to a Google Docs Premium Account? I'm wondering if that can be used as I'm using S3 at the moment. I'd like to upload images. Do you know if there is there any limit on the number of images I can upload on my 200GB Google Docs account? Thank you ;) >>> More