I am having an issue with navigation. I get a list of rows from an html table. I iterate over the rows and scrape information from them. But there is also a link on the row that I click to go to more information related to the row to scrape. Then I navigate back to the page with the original table. This works for the first row, but for the subsequent rows, it throws an exception.
I look at my row collection after the first time the link inside a row is clicked, and none of them have the correct values like they did before I clicked the link. I believe that there is something going on when I navigate to a different URL that I'm not getting.
My code is below. How do I get this working so I can iterate over the parent table, click the links in each row, navigate to the child table, but still continue iterating over the rows in the parent table?
private List<Document> getResults()
{
var documents = new List<Document>();
//Results
IWebElement docsTable = this.webDriver.FindElements(By.TagName("table"))
.Where(table => table.Text.Contains("Document List"))
.FirstOrDefault();
var validDocRowRegex = new Regex(@"^(\d{3}\s+)");
var docRows = docsTable.FindElements(By.TagName("tr"))
.Where(row =>
//It throws an exception with .FindElement() when there isn't one.
row.FindElements(By.TagName("td")).FirstOrDefault() != null &&
//Yeah, I don't get this one either. I negate the match and so it works??
!validDocRowRegex.IsMatch(
row.FindElement(By.TagName("td")).Text))
.ToList();
foreach (var docRow in docRows)
{
//Todo: find out why this is crashing on some documents.
var cells = docRow.FindElements(By.TagName("td"));
var document = new Document
{
DocID = Convert.ToInt32(cells.First().Text),
PNum = Convert.ToInt32(cells[1].Text),
AuthNum = Convert.ToInt32(cells[2].Text)
};
//Go to history for the current document.
cells.Where(cell =>
cell.FindElements(By.TagName("a")).FirstOrDefault() != null)
.FirstOrDefault().Click();
//Todo: scrape child table.
this.webDriver.Navigate().Back();
}
return documents;
}
