HtmlAgilityPack download webpage which loaded asynchronously by javascript
c# html-agility-pack javascript web-scraping
Question
i am using HtmlAgilityPack and trying to load some webpages. some webpages are javascript based and loads asynchoronously. is there any way to load web page after x seconds or after making sure page is completely loaded
Popular Answer
Html Agility Pack is not mimicking the client side calls to dynamically load content into the DOM. It is a headless scraper that is downloading the static page given by the server; if you want that content, you will have to mimic the calls made by the client browser. If you do not want to try to emulate the calls a browser would make, instead of using a headless scraper, you can use something like Selenium to do this for you, the down side being, the browser will be opened on the host machine.
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow