Groovy htmlunit getFirstByXPath returning null
- by StartingGroovy
I have had a few issues with HtmlUnit returning nulls lately and am looking for guidance. each of my results for grabbing the first row of a website have returned null. I am wondering if someone can
A) explain why they might be returning null
B) explain better ways (if there are some) to go about getting the information
Here is my current code (URL is in the source):
client = new WebClient(BrowserVersion.FIREFOX_3)
client.javaScriptEnabled = false
def url = "http://www.hidemyass.com/proxy-list/"
page = client.getPage(url)
IpAddress = page.getFirstByXPath("//html/body/div/div/form/table/tbody/tr/td[2]").getValue()
println "IP Address is: $data" //returns null
//Port_Number is an Image
Country = page.getFirstByXPath("//html/body/div/div/form/table/tbody/tr/td[4][@class='country']/@rel").getValue()
println "Country abbreviation is: $Country"
//differentiate speed and connection by name of gif?
Type = page.getFirstByXPath("//html/body/div/div/form/table/tbody/tr/td[7]").getValue()
println "Proxy type is: $Type"
Anonymity = page.getFirstByXPath("//html/body/div/div/form/table/tbody/tr/td[8]").getValue()
println "Anonymity Level is: $Anonymity"
client.closeAllWindows()
Right now all of my XPaths return null and .getValue() obviously doesn't work on null.
I also have questions as to what I should do about the PORT since it is an image? Is there a better alternative than downloading it and attempting to solve it by OCR?
Side Note
There is no significance in this site, I was just looking for a site that I could practice scraping on (the last one I ran into issues of fragment identities and couldn't get an answer to: HtmlUnit getByXpath returns null and HtmlUnit and Fragment Identities )
