HtmlAgilityPack - Extract links from a specific table

.net c# html-agility-pack

Question

Finding the precise procedure to parse for links on a website is proving to be difficult. The table's precise xPath is: employing firebug.

/html/body/div/form/table/tbody/tr/td/table/tbody/tr/td/table/tbody/tr[1]/td/table/tbody/tr/td/table/tbody/tr[2]/td/table/tbody/tr[1]/td/div/table/tbody/tr[3]/td/div/table/tbody/tr/td/div/table

Additionally, it contains the id "ctl00 cp1 GridView1" (which hasn't exactly been useful).

All I want to do is compile a list of all the links in the first.

Here is my most recent code sample, with assistance from from this article:

protected void btnSubmitURL_Click(object sender, EventArgs e)
{
    try
    {
        List<string> siteList = new List<string>();
        int counter = 1;

        var web = new HtmlWeb();
        var doc = web.Load(txtURL.Text);
        var table = doc.DocumentNode.SelectSingleNode("html/body/div/form/table/tbody/tr/td/table/tbody/tr/td/table/tbody/tr[1]/td/table/tbody/tr/td/table/tbody/tr[2]/td/table/tbody/tr[1]/td/div/table/tbody/tr[3]/td/div/table/tbody/tr/td/div/table[@id='ctl00_cp1_GridView1']/tbody");
        HtmlNodeCollection rows = table.SelectNodes("./tr");
        if (rows != null)
        {
            for (int i = 0; i < rows.Count; i++)
            {
                HtmlNodeCollection cols = rows[i].SelectNodes("./td[1]");
                if (cols != null)
                {
                    for (int j = 0; j < cols.Count; j++)
                    {
                        HtmlNode aTags = cols[i].SelectSingleNode("./a[@id='NormalColoredFont']");
                        if (aTags != null)
                        {
                            siteList.Add(counter + ". " + aTags.InnerHtml + " - " + aTags.Attributes["href"].Value);
                        }
                    }
                }
            }
        }

        lblOutput.Text = siteList.Count.ToString();
    }

    catch (Exception ex)
    {
        MessageBox.Show(ex.ToString());
    }
}

Because it can't locate that exact table, I keep receiving a Null Exception problem right at the HtmlNodeCollection rows. Although I tried searching using the table id, it didn't work either.

I'd appreciate any assistance getting to that table.

1
0
5/23/2017 11:50:51 AM

Accepted Answer

Using the illustration taken from Scott Mitchell, I was eventually able to extract all of the connections. His illustration is as follows:

var linksOnPage = from lnks in document.DocumentNode.Descendants()
              where lnks.Name == "a" && 
                   lnks.Attributes["href"] != null && 
                   lnks.InnerText.Trim().Length > 0
              select new
              {
                 Url = lnks.Attributes["href"].Value,
                 Text = lnks.InnerText
              };

We appreciate the prompt responses from casperOne and jessehouwing!

1
3/3/2015 2:59:51 PM


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow