How to scrape data in a page with jquery button click using HtmlAgility pack

html-agility-pack web-crawler web-scraping

Question

Using the HTMLAgility module, I'm attempting to scrape information from a page with related content (a shopping website).

There is a button to load more tag-designed goods. On clicking, additional content loads on the same page.

If it was constructed using tags, I would be able to access the subsequent things using the href attribute URL in the tags and also load a new page for the subsequent items, so there would be no issue.

However, everything on this page is fetched from the same URL.

Is there a method to provide this feature then? How can I press the button that says "load more" to acquire additional items?

1
0
6/17/2016 9:18:56 AM

Accepted Answer

HtmlAgilityPack is a standalone html parser that can only parse static html documents. Zzz-6-Zzz may help you achieve your goals.

Another option is to manually finish loading the items if there are enough actions required, store the final HTML locally, and then utilize it.HtmlAgiliyPack to parse the locally stored static HTML (instead of parsing the http response).

Please provide the website's URL so I can add some code examples to illustrate.

0
6/21/2016 1:46:49 PM


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow