html agility pack parsing error?

c# html-agility-pack

Question

I'm using hap to sort through several hundred pages of Amazon search results for some data analysis.

hap.DocumentNode.SelectNodes("//ul[@id='s-results-list-atf']/li")

However, if there are more results, this just returns the top four li's. I've checked and checked again, and I'm certain that this is correct. Am I doing something really wrong? I don't understand why there are just four results available. Example of typical page content: https://www.amazon.com/s/?url=search-alias%3Daps&field-keywords=100+percent+barstow

Using other search results pages results in the same issue and consistently yields results that are much inferior to those on the page.

1
0
1/17/2017 5:36:04 AM

Accepted Answer

Try this out:

hap.DocumentNode.SelectNodes("//div[contains(@id,'tfResults')]//li[contains(@class,'s-result-item')]");

No promises, however, since I'm simply relying on the URL you provided.

1
1/17/2017 6:03:10 AM


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow