HTML Agility Pack - Can only load xml document from file system, not from web

.net c# html-agility-pack scrape

Question

I've used HAP successfully before, downloading xhtml pages from web. However, now I'm trying to load and parse xml documents. HAP will only load xml documents that are located on my file system, "C:\xml\MyXml.xml" for instance. It will not load it from web (http://www.web.com/doc.xml). Using Fiddler, I can see that HAP is actually requesting the xml documents over the web, and the server also responds with the xml document. However, it stops there, nothing get parsed. The HtmlDocument is empty, no ChildNodes or anything. When loading from file system, it get parsed successfully to a HtmlDocument.

Any ideas?

Accepted Answer

If you are using just the XML (and not (X)HTML) then you don't need to use HAP as .Net has comprehensive XML processing built in:

String PostUrl = "http://www.web.com/doc.xml"; 
WebResponse webResponse = WebRequest.Create(PostUrl).GetResponse();
StreamReader sr = new StreamReader(webResponse.GetResponseStream());

String Result = sr.ReadToEnd().Trim();

XmlDocument xdoc = new XmlDocument(); xdoc.LoadXml(Result);

Popular Answer

I assume you are using HAP because the XML you are trying to parse specifies a XSL stylesheet to transform it to (X)HTML which you then want to manipulate in some way?

If this isn't the case and you are just interested in the raw XML structure then use .Net's built in XmlDocument and System.Xml namespaces as Sebastian's answer suggests.

If you actually need to manipulate the HTML structure of such a document you will need to download the XML yourself, apply the XSLT using System.Xml to generate the resulting HTML before then attempting to parse this with HAP.




Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why