Using HtmlAgilityPack in VB.Net to Get Text from a Website

html-agility-pack vb.net

Question

I am writing a program for my girlfriend that allows her to open the program and it will automatically gather her a quote from a Horoscope website and display that line of text in a TextBox.

As of what I have now, it basically displays the entire website in HTML, which is not what I want. This is the HTML line that I need to grab.

<div class="fontdef1" style="padding-right:10px;" id="textline">
"You might have the desire for travel, perhaps to visit a friend who lives far away, Gemini. You may actually set the wheels in motion to make it happen. Social events could take up your time this evening, and you could meet some interesting people. A friend might need a sympathetic ear. Today you're especially sensitive to others, so be prepared to hear a sad story. Otherwise, your day should go well. 
</div>

The code that I have so far is.

Imports System.Net
Imports System.IO
Imports HtmlAgilityPack

Public Class Form1

    Private Function getHTML(ByVal Address As String) As String
        Dim rt As String = ""

        Dim wRequest As WebRequest
        Dim wResponse As WebResponse

        Dim SR As StreamReader

        wRequest = WebRequest.Create(Address)
        wResponse = wRequest.GetResponse

        SR = New StreamReader(wResponse.GetResponseStream)

        rt = SR.ReadToEnd
        SR.Close()

        Return rt
    End Function

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        Label2.Text = Date.Now.ToString("MM/dd/yyyy")
        TextBox1.Text = getHTML("http://my.horoscope.com/astrology/free-daily-horoscope-gemini.html")
    End Sub
End Class

Thank you for any help that I can get. I honestly have no idea where to go with this program now. It's been 3 days with no progress.

Accepted Answer

Learn XPath or LINQ to pull out certain information from an HTML document using HtmlAgilityPack. This is a console application example which using XPath selector :

Imports System
Imports System.Xml
Imports HtmlAgilityPack

Public Module Module1
    Public Sub Main()
        Dim link As String = "http://my.horoscope.com/astrology/free-daily-horoscope-gemini.html"
        'download page from the link into an HtmlDocument'
        Dim doc As HtmlDocument = New HtmlWeb().Load(link)
        'select <div> having class attribute equals fontdef1'
        Dim div As HtmlNode = doc.DocumentNode.SelectSingleNode("//div[@class='fontdef1']")
        'if the div is found, print the inner text'
        If Not div Is Nothing Then
            Console.WriteLine(div.InnerText.Trim())
        End If
    End Sub
End Module

Dotnetfiddle Demo

output :

You might have the desire for travel, perhaps to visit a friend who lives far away, Gemini. You may actually set the wheels in motion to make it happen. Social events could take up your time this evening, and you could meet some interesting people. A friend might need a sympathetic ear. Today you're especially sensitive to others, so be prepared to hear a sad story. Otherwise, your day should go well.



Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why