How do I correctly grab the images I scrape with HtmlAgilityPack?

Question

How do I correctly grab the images I scrape with HtmlAgilityPack?

c# html-agility-pack web-scraping xpath

Question

I am currently working on a project and I am learning HAP as I go. I get the basics of it and it seems like it could be very powerful.

I'm having an issue right now, I am trying to scrape a product on this one website and get the links to the images but I dont know how to extract the link from the xpath. I used to do this with Regex which was alot easier but I am moving on this HAP.

This is my current code I dont think it will be very useful to see but i'll out it in either way.

    private static void HAP()
    {
        var url = "https://www.dhgate.com/product/brass-hexagonal-fidget-spinner-hexa-spinner/403294406.html#gw-0-4|ff8080815e03d6df015e9394cc681f8a:ff80808159abe8a5015a3fd78c5b51bb";
        // HtmlWeb - A Utility class to get HTML document from http
        var web = new HtmlWeb();
        //Load() Method download the specified HTML document from an Internet resource.
        var doc = web.Load(url);

        var rootNode = doc.DocumentNode;

        var divs = doc.DocumentNode.SelectNodes(String.Format("//IMG[@src='{0}']", "www.dhresource.com/webp/m/100x100/f2/albu/g5/M00/14/45/rBVaI1kWttaAI1IrAATeirRp-t8793.jpg"));
        Console.WriteLine(divs);
        Console.ReadLine();
    }

This is the link I am scraping from

https://www.dhgate.com/product/2017-led-light-up-hand-spinners-fidget-spinner/398793721.html#s1-0-1b;searl|4175152669

And this should be the xPath of the first image.

//IMG[@src='//www.dhresource.com/webp/m/100x100s/f2-albu-g5-M00-6E-20-rBVaI1kWtmmAF9cmAANMKysq_GY926.jpg/2017-led-light-up-hand-spinners-fidget-spinner.jpg']

Mark Gerb.

View more on Stack Overflow

Mark Gerb. · Accepted Answer · 9/20/2017 12:59:51 PM

I create a helper method for this. I had to get the node, and then the attribute and then cycle through the attribute to get all the links.

private static void HAP()
        {
            //Declare the URL
            var url = "https://www.dhgate.com/product/brass-hexagonal-fidget-spinner-hexa-spinner/403294406.html#gw-0-4|ff8080815e03d6df015e9394cc681f8a:ff80808159abe8a5015a3fd78c5b51bb";
            // HtmlWeb - A Utility class to get HTML document from http
            var web = new HtmlWeb();
            //Load() Method download the specified HTML document from an Internet resource.
            var doc = web.Load(url);

            var rootNode = doc.DocumentNode;

            var nodes = doc.DocumentNode.SelectNodes("//img");
            foreach (var src in nodes)
            {
                var links = src.Attributes["src"].Value;
                Console.WriteLine(links);
            }
            Console.ReadLine();
        }

How do I correctly grab the images I scrape with HtmlAgilityPack?

Question

Fastest Entity Framework Extensions

Popular Answer

Performance

Expression Evaluator

Related