How can I use HTMLAgilityPack to download a CSV file?

asp.net c# html-agility-pack vb.net

Question

I have the code to go to a website, login with the hidden fields and cookies and include a browser header so that I appear as a normal user.

Now that I am in the protected content I need to download a csv file that I have found within the document using HTMLAgilityPack.

I would like to grab the csv with HTMLAgilityPack so that I can continue to use the cookies and browser user-agent string already setup.

From what I have read HTMLAgilityPack parses the dom. I would expect a csv file to cause an error and return null. But I have seen vague references of being able to grab the raw data of the page/file requested before it is parsed. If so, that would be the solution but I cannot find how to do that.

Accepted Answer

You don't need to use HtmlAgilityPack at all, assuming the HTML form you're submitting is constant. Just craft the HTTP request manually and submit it, then download the corresponding CSV file using a HttpWebRequest.

HtmlAgilityPack is only used for working with HTML you already have in your possession. It does include an ability to make basic HTTP requests, but that's a convenience feature. Generally you should use HttpWebRequest where possible.




Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why