How to use Html Agility Pack to timeout a request

.net asp.net-mvc-3 c# html-agility-pack timeout

Question

I'm sending a request to an unavailable remote web server (on purpose).

I want to determine the most effective approach to timing the request. In essence, if the request takes longer than "X" milliseconds, it will be terminated and returned.null response.

The web request is now merely waiting for a response.

How do I go about solving this issue?

Here is a sample of recent code.

    public JsonpResult About(string HomePageUrl)
    {
        Models.Pocos.About about = null;
        if (HomePageUrl.RemoteFileExists())
        {
            // Using the Html Agility Pack, we want to extract only the
            // appropriate data from the remote page.
            HtmlWeb hw = new HtmlWeb();
            HtmlDocument doc = hw.Load(HomePageUrl);
            HtmlNode node = doc.DocumentNode.SelectSingleNode("//div[@class='wrapper1-border']");

            if (node != null)
            { 
                about = new Models.Pocos.About { html = node.InnerHtml };
            }
                //todo: look into whether this else statement is necessary
            else 
            {
                about = null;
            }
        }

        return this.Jsonp(about);
    }
1
6
7/4/2011 6:15:22 PM

Accepted Answer

I had to slightly modify the code I had first provided.

    public JsonpResult About(string HomePageUrl)
    {
        Models.Pocos.About about = null;
        // ************* CHANGE HERE - added "timeout in milliseconds" to RemoteFileExists extension method.
        if (HomePageUrl.RemoteFileExists(1000))
        {
            // Using the Html Agility Pack, we want to extract only the
            // appropriate data from the remote page.
            HtmlWeb hw = new HtmlWeb();
            HtmlDocument doc = hw.Load(HomePageUrl);
            HtmlNode node = doc.DocumentNode.SelectSingleNode("//div[@class='wrapper1-border']");

            if (node != null)
            { 
                about = new Models.Pocos.About { html = node.InnerHtml };
            }
                //todo: look into whether this else statement is necessary
            else 
            {
                about = null;
            }
        }

        return this.Jsonp(about);
    }

Then, I changed myRemoteFileExists a technique for having a timeout extension

    public static bool RemoteFileExists(this string url, int timeout)
    {
        try
        {
            //Creating the HttpWebRequest
            HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;

            // ************ ADDED HERE
            // timeout the request after x milliseconds
            request.Timeout = timeout;
            // ************

            //Setting the Request method HEAD, you can also use GET too.
            request.Method = "HEAD";
            //Getting the Web Response.
            HttpWebResponse response = request.GetResponse() as HttpWebResponse;
            //Returns TRUE if the Status code == 200
            return (response.StatusCode == HttpStatusCode.OK);
        }
        catch
        {
            //Any exception will returns false.
            return false;
        }
    }

If my timeout activates in this manner beforeRemoteFileExists can identify the response from the header, then mybool will provide false

2
7/4/2011 6:01:49 PM

Popular Answer

HTML Agility Pack is an accessible source. You may thus change the source yourself. First, add the following code to class zzzz-5 zzzz:

private int _timeout = 20000;

public int Timeout 
    { 
        get { return _timeout; } 
        set
        {
            if (_timeout < 1) 
                throw new ArgumentException("Timeout must be greater then zero.");
            _timeout = value;
        }
    }

then use this strategy.

private HttpStatusCode Get(Uri uri, string method, string path, HtmlDocument doc, IWebProxy proxy, ICredentials creds)

and change it:

req = WebRequest.Create(uri) as HttpWebRequest;
req.Method = method;
req.UserAgent = UserAgent;
req.Timeout = Timeout; //add this

Or something like

htmlWeb.PreRequest = request =>
            {
                request.Timeout = 15000;
                return true;
            };


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow