Question
I'm using hap to sort through several hundred pages of Amazon search results for some data analysis.
hap.DocumentNode.SelectNodes("//ul[@id='s-results-list-atf']/li")
However, if there are more results, this just returns the top four li's. I've checked and checked again, and I'm certain that this is correct. Am I doing something really wrong? I don't understand why there are just four results available. Example of typical page content: https://www.amazon.com/s/?url=search-alias%3Daps&field-keywords=100+percent+barstow
Using other search results pages results in the same issue and consistently yields results that are much inferior to those on the page.
Accepted Answer
Try this out:
hap.DocumentNode.SelectNodes("//div[contains(@id,'tfResults')]//li[contains(@class,'s-result-item')]");
No promises, however, since I'm simply relying on the URL you provided.
Related Questions
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow