How can i add a webRequest to set timeout when loading a url to document in HtmlAgilityPack?

Question

How can i add a webRequest to set timeout when loading a url to document in HtmlAgilityPack?

c# html-agility-pack

Question

I have this code when loading the url:

private List<string> test(string url, int levels,DoWorkEventArgs eve)
        {
            HtmlWeb hw = new HtmlWeb();
            List<string> webSites;
            try
            {
                this.Invoke(new MethodInvoker(delegate { Texts(richTextBox1, "Loading The Url: " + url + "..." , Color.Red); }));
                doc = hw.Load(url);
                this.Invoke(new MethodInvoker(delegate { Texts(richTextBox1, "Done " + Environment.NewLine, Color.Red); }));

Sometimes when its Loading the url its taking a lot fo time since the website in the url variable is not responding. I want to add a timeout so lets say after X seconds it will throw a message like " there was a timeout".

Now HtmlAgilityPack does not have any timeout property or class. So i thought to create a new function in my Form1 that will use webrequest and webresponde and set a timeout in this new function then calling this function before loading the url.

Can someone show me how to make the new function to work with my code ? Also with a timeout.

Thanks.

Daniel Lip

Popular Answer

Source: http://blog.jongallant.com/2012/07/htmlagilitypack-set-timeout.html#.VBY-_fmSz3Q

var web = new HtmlWeb();
web.PreRequest = delegate(HttpWebRequest webRequest)
{
     webRequest.Timeout = 4;
     return true;
};
var doc = web.Load("http://www.msn.com/");

Daniel

View more on Stack Overflow

Accepted Answer

I don't know anything about html agility's timeout. but I'm using it like this. Maybe it's helpful to you. Goodluck.

    String Data = GetURLData(url);
    HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
    doc.LoadHtml(Data);




    public static string GetURLData(string URL)
    {
        try
        {
            HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(URL);
            request.UserAgent = "Omurcek";
            request.Timeout = 4000;
            WebResponse response = request.GetResponse();
            Stream stream = response.GetResponseStream();
            StreamReader reader = new StreamReader(stream);
            return reader.ReadToEnd();
        }   

        catch (Exception ex )
        {
            LogYaz("Receive DATA Error : " + URL   + ex.ToString());
            return "";
        }

    }

How can i add a webRequest to set timeout when loading a url to document in HtmlAgilityPack?

Question

Fastest Entity Framework Extensions

Accepted Answer

Popular Answer

Performance

Expression Evaluator

Related