HtmlAgilityPack and HtmlDecode

html-agility-pack

Question

I am currently using HtmlAgilityPack with a console application to scrape a website. Since the html is encoded (it returns encoded characters like ') I have to decode before I save the content to my database.

Is there a way to decode the returned html using HtmlAgilityPack without having to use HttpUtility.HtmlDecode? I want to avoid adding System.Web to my console application if possible.

Accepted Answer

The Html Agility Pack is equiped with a utility class called HtmlEntity. It has a static method with the following signature:

    /// <summary>
    /// Replace known entities by characters.
    /// </summary>
    /// <param name="text">The source text.</param>
    /// <returns>The result text.</returns>
    public static string DeEntitize(string text)

It supports well-known entities (like &nbsp;) and encoded characters such as &#039; as well.


Popular Answer

Just adding my 2 cents: I've ran some performance tests using StopWatch class and found that HttpUtility.HtmlDecode is about 15-20% faster than the DeEntitize method. Also DeEntitize has some bugs (see comments above).

So maybe referencing System.Web is not that bad after all.

If you're writing an app that already targets ".NET full" (opposed to ".NET Client Profile" - which is a lightweight version) - I'd go for referencing System.Web.




Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why