Calling a web service with that needs cookies c# html-agility-pack web-services


I'm attempting to scrape the list of attorneys at using the HTML Agility Pack. I haven't had much success trying to accomplish it server side since the site utilizes Ajax to generate the names. I tried to use the same web service from my application after seeing the site is calling one at

I'm trying to use the following code, but it keeps giving me an Internal Server Error 500, which I think is because I need a session cookie. (since if I deactivate cookies, the site won't function)

Can anyone assist?

private static string WebServiceCall()

    WebRequest webRequest = WebRequest.Create("");
    HttpWebRequest httpRequest = (HttpWebRequest)webRequest;
    httpRequest.Method = "POST";
    httpRequest.ContentType = "text/xml; charset=utf-8";
    httpRequest.ProtocolVersion = HttpVersion.Version11;
    Stream requestStream = httpRequest.GetRequestStream();
    //Create Stream and Complete Request             
    StreamWriter streamWriter = new StreamWriter(requestStream, Encoding.ASCII);
    string request = "<soap:Envelope xmlns:xsi='' xmlns:xsd='' xmlns:soap=''><soap:Body><FindAttorneys xmlns='FCWSite.FCWSite.DlaPiperWS'><scFirstName></scFirstName><scLastName>A</scLastName><scKeyword></scKeyword><scOfficesGUID></scOfficesGUID><scSpokenLanguagesGUID></scSpokenLanguagesGUID><scServicesGUID></scServicesGUID><scRegionGUID></scRegionGUID><scSchoolGUID></scSchoolGUID><scAdmissionGUID></scAdmissionGUID><scLevelGUID></scLevelGUID><strLanguageGUID>7483b893-e478-44a4-8fed-f49aa917d8cf</strLanguageGUID><strCountry>global</strCountry><returnUntranslated>true</returnUntranslated><sortBy>name</sortBy><page>0</page></FindAttorneys></soap:Body></soap:Envelope>";

    //Get the Response    
    HttpWebResponse wr = (HttpWebResponse)httpRequest.GetResponse();
    StreamReader srd = new StreamReader(wr.GetResponseStream());
    string resulXmlFromWebService = srd.ReadToEnd();
    return resulXmlFromWebService;
3/3/2014 6:21:31 AM

Accepted Answer

I included the Attorney.asmx address as a web reference in my project, as I said in my remark.

When doing so, I permitted the web reference to use the namespace "com.dlapiper.www" as the reference.

The following is written in VB.NET and is an example of a default page. The context of the connection is represented by the class com.dlapiper.www.Attorneys.

Take note that lawyers. A string value is returned by FindAttorney.

Additionally, my buffer and array let you view. If you attempt to execute FindAttorney in the code without adding the cookie container and the subsequent cookies, you will mistakenly get a SOAP error that claims the localizationGUID was not in the right input. This is because I was experimenting with various encodings.

Because I like contravariant programming, I left this pretty pointless section of code in the program.

The cookie name:value is what the function really uses, even though that argument can't be empty.


Public Class _Default
    Inherits System.Web.UI.Page

    Dim AttorneyList As com.dlapiper.www.Attorneys = New com.dlapiper.www.Attorneys()
    Dim attys As String

    Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
        Dim buffer As Byte()
        buffer = Array.ConvertAll(Of Char, Byte)("7483b893e47844a48fedf49aa917d8cf".ToUpper.ToCharArray(), New Converter(Of Char, Byte)(AddressOf CharToBuffer))

        Dim cookieLanguage As System.Net.Cookie = New System.Net.Cookie
        cookieLanguage.Domain = ""
        cookieLanguage.Name = "Language"
        cookieLanguage.Value = UTF8Encoding.ASCII.GetString(buffer)
        cookieLanguage.Path = "/"

        Dim cookieLocalization As System.Net.Cookie = New System.Net.Cookie
        cookieLocalization.Domain = ""
        cookieLocalization.Name = "Localization"
        cookieLocalization.Value = "TimeZone=0&UsesDaylightSavings=False&TimeZoneAbbrev=IDLW&Persists=True"
        cookieLocalization.Path = "/"

        Dim cookieCulture As System.Net.Cookie = New System.Net.Cookie
        cookieCulture.Domain = ""
        cookieCulture.Name = "DefaultCulture"
        cookieCulture.Value = "en-US"
        cookieCulture.Path = "/"

        Dim cookieHideNotice As System.Net.Cookie = New System.Net.Cookie
        cookieHideNotice.Domain = ""
        cookieHideNotice.Name = "hide-cookie-notice"
        cookieHideNotice.Value = "1"
        cookieHideNotice.Path = "/"

        Dim cookieMode As System.Net.Cookie = New System.Net.Cookie
        cookieMode.Domain = ""
        cookieMode.Name = "Mode"
        cookieMode.Value = "1"
        cookieMode.Path = "/"

        Dim cookieNavId As System.Net.Cookie = New System.Net.Cookie
        cookieNavId.Domain = ""
        cookieNavId.Name = "NavId"
        cookieNavId.Value = "1074"
        cookieNavId.Path = "/"

        Dim cookiePortletId As System.Net.Cookie = New System.Net.Cookie
        cookiePortletId.Domain = ""
        cookiePortletId.Name = "PortletId"
        cookiePortletId.Value = "12601"
        cookiePortletId.Path = "/"

        Dim cookieSERVER_PORT As System.Net.Cookie = New System.Net.Cookie
        cookieSERVER_PORT.Domain = ""
        cookieSERVER_PORT.Name = "SERVER_PORT"
        cookieSERVER_PORT.Value = "80"
        cookieSERVER_PORT.Path = "/"

        Dim cookieSiteId As System.Net.Cookie = New System.Net.Cookie
        cookieSiteId.Domain = ""
        cookieSiteId.Name = "SiteId"
        cookieSiteId.Value = "1039"
        cookieSiteId.Path = "/"

        Dim cookieZoneId As System.Net.Cookie = New System.Net.Cookie
        cookieZoneId.Domain = ""
        cookieZoneId.Name = "ZoneId"
        cookieZoneId.Value = "8"
        cookieZoneId.Path = "/"

        Dim cookieEventingStatus As System.Net.Cookie = New System.Net.Cookie
        cookieEventingStatus.Domain = ""
        cookieEventingStatus.Name = "EventingStatus"
        cookieEventingStatus.Value = "1"
        cookieEventingStatus.Path = "/"

        AttorneyList.CookieContainer = New System.Net.CookieContainer()


        attys = AttorneyList.FindAttorneys("B", "", "", "", "", "", "", "", "", "", UTF8Encoding.ASCII.GetString(buffer), "global", False, "name", 0)
    End Sub

    Function CharToBuffer(ByVal character As Char) As Byte
        Return Convert.ToByte(character)
    End Function
End Class
3/3/2014 8:56:49 AM

Related Questions


Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow