HTML AGILITY PACK: Using Proxies?

html-agility-pack

Question

How do I use proxies with HAP? This is what I have so far...(with no luck).

IPCHICKEN is just used to test the ip address. it is showing my ip address and not the ip address of my proxy

Function GetPrice(ByVal AmazonURL As String, ByVal Delay As Integer)
    Dim aHtml As New HtmlWeb
        Dim ChromeAgent As String = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11"
    aHtml.UserAgent = ChromeAgent


    Dim proxy As New System.Net.WebProxy
    Dim proxyAddress As New Uri("http://111.111.111/")


    Dim aDoc As HtmlDocument = aHtml.Load("http://www.ipchicken.com", "GET", proxy, System.Net.CredentialCache.DefaultCredentials)

    Dim aNode As HtmlAgilityPack.HtmlNode
    aNode = aDoc.DocumentNode.SelectSingleNode("//div[@id='olpDivId']/span[2]")

    If aNode.InnerText Is Nothing Then

    End If

    Dim UsedPrice1 As String = aNode.InnerText
    Dim i As Integer = UsedPrice1.IndexOf("$")
    Dim UsedPrice As Integer = UsedPrice1.Substring(i + 1)

    System.Threading.Thread.Sleep(Delay)

    Return UsedPrice
End Function
1
0
7/24/2012 12:03:05 AM

Popular Answer

Maybe try something like the following. Depending on the page you mightn't need the UTF8 conversion.

WebProxy proxy = new WebProxy("proxyname", 8080);
proxy.Credentials = CredentialCache.DefaultCredentials;

WebClient client = new WebClient();
client.Proxy = proxy;

string baseHtml = "";

byte[] pageContent = client.DownloadData("your target url");

UTF8Encoding utf = new UTF8Encoding();
baseHtml = utf.GetString(pageContent);

HtmlDocument pageHtml = new HtmlDocument();
pageHtml.LoadHtml(baseHtml);
0
7/27/2012 4:25:08 PM


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow