This pertains to cookies set inside a script (maybe inside a script tag).
System.Windows.Forms.HtmlDocument
executes those scripts and the cookies set (like document.cookie=etc...
) can be retrieved through its Cookies property.
I assume HtmlAgilityPack.HtmlDocument
doesn't do this (execution). I wonder if there is an easy way to emulate the System.Windows.Forms.HtmlDocument
capabilities (the cookies part).
Anyone?
When I need to use Cookies and HtmlAgilityPack together, or just create custom requests (for example, set the User-Agent
property, etc), here is what I do:
WebQuery
...
public HtmlAgilityPack.HtmlDocument GetSource(string url);
What do we need to do inside this method?
Well, using HttpWebRequest and HttpWebResponse, generate the http request manually (there are several examples of how to do this on Internet), create an instance of a HtmlDocument
class using the constructor that receives an stream.
What stream do we have to use? Well, the one returned by:
httpResponse.GetResponseStream();
If you use HttpWebRequest to make the query, you can easily set the CookieContainer
property of it to the variable you declared before everytime you access a new page, and that way all cookies set by the sites you access will be properly stored in the CookieContainer
variable you declared in your WebQuery
class, taking in count you're using only one instance of the WebQuery
class.
Hope you find useful this explanation. Take in count that using this, you can do whatever you want, no matter if HtmlAgilityPack supports it or not.