Multiple HttpWebRequest in loop?

Question

Multiple HttpWebRequest in loop?

.net c# html-agility-pack httpwebrequest

Question

I have to call web-service multiple time (in loop) the problem is that my code always return empty object (Image Description) and doesn't run properly when I tested it separately (out of loop) it worked normally

here is my portion of code

HttpWebRequest httpReq = (HttpWebRequest)HttpWebRequest.Create(new Uri(imageCollection[i].ImageTag));

            httpReq.BeginGetResponse(new AsyncCallback((iar) =>
             {
                 try
                 {
                     string strResponse = "";
                     var response = (HttpWebResponse)((HttpWebRequest)iar.AsyncState).EndGetResponse(iar);

                     Stream stream = response.GetResponseStream();
                     StreamReader reader = new StreamReader(stream);
                     strResponse = reader.ReadToEnd();

                     HtmlDocument htmlDocument = new HtmlDocument();
                     htmlDocument.OptionFixNestedTags = true;
                     htmlDocument.LoadHtml(strResponse);
                     HtmlAgilityPack.HtmlNode titleNode = htmlDocument.DocumentNode.SelectSingleNode("//meta[@property='og:description']");

                     if (titleNode != null)
                     {
                         string desc = titleNode.GetAttributeValue("content", "");
                         imageCollection[i].ImageDescription = desc;
                     }
                 }
                 catch (Exception ex)
                 {
                     throw ex;
                 }

             }), httpReq);

            httpReq.Abort();

Mohab

View more on Stack Overflow

Accepted Answer

I got the answer from another post in stack-overflow modified to adapt my solution here Getting the Response of a Asynchronous HttpWebRequest

I made specific class called Request to transform my logic to newly async and await here it's

       public Task<string> MakeAsyncRequest()
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL);

            Task<WebResponse> task = Task.Factory.FromAsync(
                request.BeginGetResponse,
                asyncResult => request.EndGetResponse(asyncResult),
                (object)null);

            return task.ContinueWith(t => ReadStreamFromResponse(t.Result));
        }

        private string ReadStreamFromResponse(WebResponse response)
        {
            string desc = "";
            try
            {
           using (Stream responseStream = response.GetResponseStream())
                using (StreamReader sr = new StreamReader(responseStream))
                {
                    //Need to return this response 
                    string strContent = sr.ReadToEnd();
                    HtmlDocument htmlDocument = new HtmlDocument();
                    htmlDocument.OptionFixNestedTags = true;
                    htmlDocument.LoadHtml(strContent);
                    HtmlAgilityPack.HtmlNode titleNode = htmlDocument.DocumentNode.SelectSingleNode("//meta[@property='og:description']");

                    if (titleNode != null)
                    {
                        desc = titleNode.GetAttributeValue("content", "");

                    }
                    imageDesc = desc;
                    return desc;
                }
            }
            catch (Exception ex)
            { return desc; }
        }

        public string imageDesc { get; private set; }

    }

Then I made a Queue of Request

        queueWebRequest = new Queue<Request>();

        for (int i = 0; i < imageCollection.Count; i++)
        {
            queueWebRequest.Enqueue(new Request(imageCollection[i].ImageTag));

        }

        for (int i = 0; i < imageCollection.Count; i++)
        {
            if (queueWebRequest.Count > 0)
            {
                Request currentRequest = queueWebRequest.Dequeue();
                await currentRequest.MakeAsyncRequest();
                imageCollection[i].ImageDescription = currentRequest.imageDesc;

            }
            else
                break;
        }

Multiple HttpWebRequest in loop?

Question

Accepted Answer

Performance

Expression Evaluator