How to extract data from a HTML table?



I recently downloaded HtmlAgilityPack but I haven't found any real instructions on how to use it. I have attempted to piece together some code based on some various discussion board posts and other sources. Here is what I have so far:

Private Sub Button3_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)
    Dim document As New HtmlAgilityPack.HtmlDocument

    Dim tabletag = document.DocumentNode.SelectSingleNode("//table[@class='data']/tr[1]/td[2]")
End Sub

As you can see I am working with the HTML from

I am trying to extract the Beta value from this page. This value is currently 1.04.

When I run the code above my immediate window shows this repeated 100 times:

Trading Report for (GOOG). A detailed report, including free correlated market analysis, and updates.
ValuEngine Detailed Valuation Report for GOOG
GOOGLE INC CL A (GOOG)  12-months forecast
GOOGLE INC CL A (GOOG)  2-weeks forecast
Google Inc: Business description, financial summary, 3yr and interim financials, key statistics/ratios and historical ratio analysis.

I only want the first number (1.04) returned. What am I doing wrong? Any suggestions?

1/14/2013 4:32:36 PM

Accepted Answer

You need to use cookies and a proxy. The below works great for me. Let me know your thoughts:

Imports System.Net
Imports System.Web

Public Class Form1

    Public cookies As New CookieContainer

    Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click

        Dim wreq As HttpWebRequest = WebRequest.Create("")

        wreq.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv: Gecko/20091102 Firefox/3.5.5"

        wreq.Method = "get"

        Dim prox As IWebProxy = wreq.Proxy

        prox.Credentials = CredentialCache.DefaultCredentials

        Dim document As New HtmlAgilityPack.HtmlDocument
        Dim web As New HtmlAgilityPack.HtmlWeb

        web.UseCookies = True
        web.PreRequest = New HtmlAgilityPack.HtmlWeb.PreRequestHandler(AddressOf onPreReq)

        wreq.CookieContainer = cookies

        Dim res As HttpWebResponse = wreq.GetResponse()

        document.Load(res.GetResponseStream, True)

        'just for testing:
        '   Dim tabletag = document.DocumentNode.SelectNodes("//table")
        '  MsgBox(tabletag.Nodes.Count.ToString)

        'returns your field
        Dim tabletag2 = document.DocumentNode.SelectSingleNode("//td[@class='data']")

    End Sub

    Private Function onPreReq(req As HttpWebRequest)

        req.CookieContainer = cookies
        Return True

    End Function
End Class
1/14/2013 7:50:07 PM

Related Questions


Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow