htmlagilitypack xpath not working

c# html-agility-pack xpath


I have a problem that my xpath is not working.

I am trying to get the url of's next link at the bottom.

But i am unable to reach on url using Xpath.

Please help me in correcting my xpath. Also tell me what should be on the place of ??

HtmlWeb hw = new HtmlWeb();

HtmlAgilityPack.HtmlDocument doc = hw.Load("");
HtmlNodeCollection linkNodes = doc.DocumentNode.SelectNodes("//*[@id='pnnext']");

foreach (HtmlNode linkNode in linkNodes)
    HtmlAttribute link = linkNode.Attributes["href"];
    MessageBox.Show(link.Value );
1/31/2013 9:56:53 PM

Accepted Answer

The weird thing here is that somehow HtmlAgilityPack does not recognize the id attribute of the "Next" link.

This could be a bug in HtmlAgilityPack; you can post it in the HAP Issue Tracker.

However, in the meantime I found this workaround:

  • find the table that contains the paging elements (the table with id="nav"). For this element the id is correctly recognized
  • take the first (and only tr) in the table and the last td of it (using the XPath last() function)
  • take the a element inside the td we obtained at the previous step.

Long story short, here's the code:

var doc = new HtmlWeb().Load("");

var nextLink = doc.DocumentNode

Console.WriteLine(nextLink.GetAttribute("href", "err"));


After Simon's comment I checked this again and the conclusion is that this is not a bug in HTML Agility Pack; the id="pnnext" attribute is only present when the request is made by a browser (perhaps depending on the UserAgent header value). When doing an HttpWebRequest from code, this is how the "Next" link appears in the output:

<a href="/search?q=seo&amp;hl=en&amp;ie=UTF-8&amp[...]" style="text-align:left">
2/5/2013 1:12:24 PM

Related Questions


Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow