Get links of a html document in order

Question

Get links of a html document in order

c# html html-agility-pack regex webbrowser-control

Question

I want to get all the links of a HTML document. This isn't a problem, but apparently it puts all the links in an alphabetic order before storing them in an array one by one. I want to have the links in original order (not in alphabetic).

So is there any possibility to get the first found link, store it, then the second one,...? I already tried using HtmlAgilityPack and the Webbrowser-Control methods, but both order them alphabetically. The original order is important for later purposes.

I heard that it might be possible with Regex, but I've found enough answers, where they say that you shouldn't use it for HTML parsing. So how can I do it?

Here's the Webbrowser-Control code, I tried to use to get the links and store them into an array:

    private void btnGet_Click(object sender, EventArgs e)
    {
        HtmlWindow mainFrame = webFl.Document.Window.Frames["mainFrame"];
        HtmlElementCollection links = mainFrame.Document.Links;

        foreach (HtmlElement link in links)
        {
            string linkText = link.OuterHtml;
            if (linkText.Contains("puzzle"))
            {
                arr[i] = linkText;
                i++;
            }
        }
    }

Thank you in advance, Opak

Opak

View more on Stack Overflow

noseratio · Accepted Answer · 12/1/2013 5:16:35 AM

You can get the correct order by walking the DOM tree using HTML DOM API. The following code does this. Note, I use dynamic to access DOM API. That's because WebBrowser's HtmlElement.FirstChild/HtmlElement.NextSibling don't work for this purpose, as they return null for DOM text nodes.

private void btnGet_Click(object sender, EventArgs e)
{
    Action<object> walkTheDom = null;
    var links = new List<object>();

    // element.FirstChild / NextSibling don't work as they stop at DOM text nodes

    walkTheDom = (element) =>
    {
        dynamic domElement = element;
        if (domElement.tagName == "A")
            links.Add(domElement);
        for (dynamic child = domElement.firstChild; child != null; child = child.nextSibling)
        {
            if (child.nodeType == 1) // Element node?
                walkTheDom(child);
        }
    };

    walkTheDom(this.webBrowser.Document.Body.DomElement);

    string html = links.Aggregate(String.Empty, (a, b) => a + ((dynamic)b).outerHtml + Environment.NewLine);
    MessageBox.Show(html);
}

[UPDATE] If you really need to get a list of HtmlElement objects for <A> tags, instead of dynamic native elements, that's still possible with a little trick using GetElementById:

private void btnGet_Click(object sender, EventArgs e)
{
    // element.FirstChild / NextSibling don't work because they stop on DOM text nodes

    var links = new List<HtmlElement>();
    var document = this.webBrowser.Document;
    dynamic domDocument = document.DomDocument;
    Action<dynamic> walkTheDom = null;

    walkTheDom = (domElement) =>
    {
        if (domElement.tagName == "A")
        {
            // get HtmlElement for the found <A> tag
            string savedId = domElement.id;
            string uniqueId = domDocument.uniqueID;
            domElement.id = uniqueId;
            links.Add(document.GetElementById(uniqueId));
            if (savedId != null)
                domElement.id = savedId;
            else
                domElement.removeAttribute("id");
        }
        for (var child = domElement.firstChild; child != null; child = child.nextSibling)
        {
            if (child.nodeType == 1) // is an Element node?
                walkTheDom(child);
        }
    };

    // walk the DOM for <A> tags
    walkTheDom(domDocument.body);

    // show the found tags
    string combinedHtml = links.Aggregate(String.Empty, (html, element) => html + element.OuterHtml + Environment.NewLine);
    MessageBox.Show(combinedHtml);
}

Get links of a html document in order

Question

Fastest Entity Framework Extensions

Popular Answer

Performance

Expression Evaluator

Related