Parsing HTML with LINQ

Question

Parsing HTML with LINQ

c# html html-agility-pack linq

Question

I'm trying to take all cells from a HTML table using Html Agility Pack and LINQ. I have loaded the HTML source in a HtmlAgilityPack.HtmlDocument and selected the tags with LINQ. However after using foreach to iterate the result, it crashes in the second record.

This is a fragment of the HTML source:

<tr>
    <td class='city'>New York</td>
    <td>Card 1</td>
</tr>
<tr>
    <td class='city'>London</td>
    <td>Card 2</td>
</tr>
<tr>
    <td class='city'>Tokyo</td>
    <td>Card 3</td>
</tr>
<tr>
    <td class='city'>Berlin</td>
    <td>Card 4</td>
</tr>

And this is what I made:

htmlDoc.LoadHtml(await msgRecived.Content.ReadAsStringAsync());

var tds=
    from td in htmlDoc.DocumentNode.Descendants("td")
    where td.Attributes["class"].Value == "city"
    select td.InnerText;

foreach (var td in tds)
{
    citiesText = citiesText + " " + td;
}

It only return the first element, e.g. if instead of using foreach I do:

citiesText = tds.ElementAt(0);

It returns New York, but if I try ElementAt(1) it crashes with Object reference not set to an instance of an object.

Any help? Thanks

Nezz

Popular Answer

Just a guess but you are probably only looking at the td on the first element. Maybe you need

htmlDoc.DocumentNode.Descendants("table") instead.

Vilhelm

View more on Stack Overflow

Andrew Whitaker · Accepted Answer · 8/14/2014 3:22:50 PM

You need to make sure that Attributes["class"] is not null:

var tds =
    from td in doc.DocumentNode.Descendants("td")
    where td.Attributes["class"] != null && td.Attributes["class"].Value == "city"
    select td.InnerText;

The second <td> retrieved has no class attribute, so when you access Attributes["class"] in that case, you're getting null. Calling .Value on null is causing the exception.

Alternatively you could use GetAttributeValue:

var tds =
    from td in doc.DocumentNode.Descendants("td")
    where td.GetAttributeValue("class", null) == "city"
    select td.InnerText;

Parsing HTML with LINQ

Question

Fastest Entity Framework Extensions

Accepted Answer

Popular Answer

Performance

Expression Evaluator

Related