is it possible to fix the problem in HtmlAgilityPack when there is a not closed html tag?

c# html-agility-pack xpath

Question

well i have the following problem.
the html i have is malformed and i have problems with selecting nodes using html agility pack when this is the case.
the code is below:

string strHtml = @"
<html>
  <div>
    <p><strong>Elem_A</strong>String_A1_2 String_A1_2</p>
    <p><strong>Elem_B</strong>String_B1_2 String_B1_2</p>
  </div>
  <div>
    <p><strong>Elem_A</strong>String_A2_2 <String_A2_2> asdas</p>
    <p><strong>Elem_B</strong>String_B2_2 String_B2_2</p>
  </div>
</html>";
HtmlAgilityPack.HtmlDocument objHtmlDocument = new HtmlAgilityPack.HtmlDocument();
objHtmlDocument.LoadHtml(strHtml);
HtmlAgilityPack.HtmlNodeCollection colnodePs = objHtmlDocument.DocumentNode.SelectNodes("//p");
List<string> lststrText = new List<string>();
foreach (HtmlAgilityPack.HtmlNode nodeP in colnodePs)
{
  lststrText.Add(nodeP.InnerHtml);
}

the problem is that String_A2_2 is enclosed in brackets.
so htmlagility pack returns 5 strings instead of 4 in the lststrText.
so is it possible to let htmlagility pack return element 3 as "<strong>Elem_A</strong>String_A2_2 <String_A2_2> asdas"?
or maybe i can do some preprocessing to close the element?
the current content of lststrText is

lststrText[0] = "<strong>Elem_A</strong>String_A1_2 String_A1_2"  
lststrText[1] = "<strong>Elem_B</strong>String_B1_2 String_B1_2"  
lststrText[2] = ""  
lststrText[3] = ""  
lststrText[4] = "<strong>Elem_B</strong>String_B2_2 String_B2_2"

Accepted Answer

Most html parsers try to build a working DOM, meaning dangling tags are not accepted. They will be converted, or closed in some way.

If only selecting the nodes is of importance to you, and speed and huge amounts of data is not an issue, you could grab all your <p> tags with a regular expression instead:

Regex reMatchP = new Regex(@"<(p)>.*?</\1>");
foreach (Match m in reMatchP.Matches(strHtml))
{
   Console.WriteLine(m.Value);
}

This regular expression assumes the <p> tags are well formed and closed.

If you are to run this Regex a lot in your program you should declare it as:

static Regex reMatchP = new Regex(@"<(p)>.*?</\1>", RegexOptions.Compiled);

[Edit: Agility pack change]

If you want to use HtmlAgility pack you can modify the PushNodeEnd function in HtmlDocument.cs:

if (HtmlNode.IsCDataElement(CurrentNodeName()))
{
   _state = ParseState.PcData;
   return true;
}

// new code start
if ( !AllowedTags.Contains(_currentnode.Name) )
{
    close = true;
}
// new code end

where AllowedTags would be a list of all known tags: b, p, br, span, div, etc.

the output is not 100% what you want, but maybe close enough?

<strong>Elem_A</strong>String_A1_2 String_A1_2
<strong>Elem_B</strong>String_B1_2 String_B1_2
<strong>Elem_A</strong>String_A2_2 <ignorestring_a2_2></ignorestring_a2_2> asdas
<strong>Elem_B</strong>String_B2_2 String_B2_2

Popular Answer

You could use TidyNet to do the pre/postprocessing you allude to. Can you edit your answer to explain why that wouldnt be applicable in your case?




Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why