Get links with specific words from a HTML code in C#

c# html-agility-pack regex

Question

A webpage is being parsed by me. In an HTML document with certain terms, I need certain links. Although I am aware of how to retrieve "href" characteristics, is there a way to do it without using all of them? Can I use regex in HTML Agility Pack, for instance?

HtmlNode links = document.DocumentNode.SelectSingleNode("//*[@id='navigation']/div/ul");

foreach (HtmlNode urls in document.DocumentNode.SelectNodes("//a[@]"))
{
    this.dgvurl.Rows.Add(urls.Attributes["href"].Value);
}   

I'm attempting this to locate every link in the HTML code.

1
1
9/26/2017 2:22:31 PM

Accepted Answer

If your HTML file looks like this:

<div class="a">
    <a href="http://www.website.com/"></a>
    <a href="http://www.website.com/notfound"></a>
    <a href="http://www.website.com/theword"></a>
    <a href="http://www.website.com/sub/theword"></a>
    <a href="http://www.website.com/theword.html"></a>
    <a href="http://www.website.com/other"></a>
</div>

For instance, you may be looking for the words:theword and other To get links with an attribute, write a regular expression and utilize LINQ.href according to your regular expression as follows:

Regex regex = new Regex("(theworld|other)", RegexOptions.IgnoreCase);

HtmlNode node = htmlDoc.DocumentNode.SelectSingleNode("//div[@class='a']");
List<HtmlNode> nodeList = node.SelectNodes(".//a").Where(a => regex.IsMatch(a.Attributes["href"].Value)).ToList<HtmlNode>();

List<string> urls = new List<string>();

foreach (HtmlNode n in nodeList)
{
    urls.Add(n.Attributes["href"].Value);
}

Recall that there is acontains keyword using XPATH, but you'll need to repeat the search criteria for each word, like:

node.SelectNodes(".//a[contains(@href,'theword') or contains(@href,'other')]")

Additionally, amatches XPATH 2.0 is the only version that supports the keyword, whereas HtmlAgilityPack only supports XPATH 1.0. With XPATH 2.0, you could carry out the following:

node.SelectNodes(".//a[matches(@href,'(theword|other)')]")
1
9/26/2017 2:24:06 PM

Popular Answer

This and that, in my opinion, work well.

HtmlNode links = document.DocumentNode.SelectSingleNode("//*[@id='navigation']/div/ul");
    foreach (HtmlNode urls in document.DocumentNode.SelectNodes("//a[@]"))
        {
           var temp = catagory.Attributes["href"].Value;
           if (temp.Contains("some_word"))
              {
                dgv.Rows.Add(temp);
              }
        }


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow