Why is my xpath null? Why is it not finding the nodes

.net c# html-agility-pack xml xpath

Question

I am just getting into traversing through XML documents to learn how to use xpath.

I have stumbled on to a issue. Everytime I try to execute my xpath it returns null as if it didnt find anything. I've tried the xpath out in XMLQuire and it worked there.

class Program
    {
        private static string URL = "https://www.kijiji.ca/b-renovation-contracting-handyman/ontario/home-renovations/k0c753l9004";
        private static HtmlWeb client = new HtmlWeb();
        static void Main(string[] args)
        {
            var DOM = client.Load(URL); // //table/tbody/tr/td[@class = 'description']/p
            var Featured = DOM.DocumentNode.SelectNodes("//table[contains(@class,'top-feature')]/tbody/tr/td/a");
            foreach (var Listing in Featured)
            {

            }
        }
    }

I commented out the other xpath I tried, I've tried those two and both are returning null why is that?

Here is a image showing the part of the DOM I want to access. enter image description here

<table class="top-feature          js-hover" data-ad-id="1299717863" data-vip-url="/v-renovation-contracting-handyman/sudbury/c-l-contracting-any-job-big-or-small/1299717863">
    <tbody><tr>
        <td class="watchlist">
            <div class="watch js-hover p-vap-lnk-actn-addwtch" data-action="add" data-adid="1299717863" title="Click to add to My Favourites"><div class="icon"></div></div>
                    <input id="watchlistXsrf" name="ca.kijiji.xsrf.token" value="1527418405414.9b71d1309fdd8a315258ea5a3dac1a09e4a99ec7f32041df88307c46e26a5b1b" type="hidden">
</td>

        <td class="image">
            <div class="multiple-images"><img src="https://i.ebayimg.com/00/s/NjAwWDgwMA==/z/fXEAAOSwaZdZxTv~/$_2.JPG" alt="C.L. Contracting. Any job big or small."></div>
                        </td>
        <td class="description">
            <a href="/v-renovation-contracting-handyman/sudbury/c-l-contracting-any-job-big-or-small/1299717863" class="title ">
                C.L. Contracting. Any job big or small.</a>

            <p>
                Contractor handyman home renovations and repairs. Contractor for Dollarama, Rexall, LaSenza and more. Fully licensed and insured. Able to do drywall, decks, framing, plumbing, flooring windows, ...</p>

            <p class="details">
                </p>
        </td>
        <td class="posted">
            </td>
    </tr>
    </tbody></table>

My solution (Need help making my xpath into 1 line instead of traversing through with a bunch of loops.)

private static string URL = "https://www.kijiji.ca/b-renovation-contracting-handyman/ontario/home-renovations/k0c753l9004";
        private static HtmlWeb client = new HtmlWeb();
        static void Main(string[] args)
        {
            var DOM = client.Load(URL); // //table/tbody/tr/td[@class = 'description']/p
            var Featured = DOM.DocumentNode.SelectNodes("//table[contains(@class,'top-feature')]/tbody/tr/td/a");
            foreach (var table in DOM.DocumentNode.SelectNodes("//table[contains(@class, 'top-feature')]"))
            {
                Console.WriteLine($"Found: {table}");
                foreach (var rows in table.SelectNodes("tr"))
                {
                    Console.WriteLine(rows);

                    foreach (var cell in rows.SelectNodes("td[@class='description']/a"))
                    {
                    Console.WriteLine(cell.InnerText.Trim());

                    }
                }
            }

            Console.ReadKey();

I've managed to fix it, however I ams till curious to why this xpath works

//table[contains(@class, 'top-feature')]/tr/td[@class='description']/a

And this one doesnt. //table[contains(@class,'top-feature')]/tbody/tr/td/a

Popular Answer

As mentioned in the comment, the <tbody> element is generated by a browser developer tool.

If you look at your var DOM object during runtime with the debugger, you can see the InnerHtml property.

<table class="regular-ad js-hover" data-ad-id=".." data-vip-url="..">
    <tr>
        <td class="watchlist">
            ...
        </td>    
        <td class="image">
            ...
        </td>
        ...
    </tr>
 </table>

No <tbody> element so your XPath has to look like this:

DOM.DocumentNode.SelectNodes("//table[contains(@class,'top-feature')]/tr/td/a");


Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow