I am developing an application that pulls data from web and parses it using HtmlAgilityPack. However, its not that straight-forward, login is required to access the data.
I am using Restsharp to authenticate like this:
//class definitions
request = new RestRequest("http://allpoetry.com/login", Method.POST);
request.AddParameter("utf8", "%E2%9C%93");
request.AddParameter("authenticity_token", "Lr+JtLv7TkZrS73u5scRPbWhuUYDaVF7vd6lkKFF3FKYqNKHircT9v5WCT9EkneTJRsKXaA6nf6fiWopDB0INw==");
request.AddParameter("referer", url); // url to the page I want to go
request.AddParameter("user[name]", "username");
request.AddParameter("user[password]", "password");
request.AddParameter("commit", "Log in");
request.AddHeader("content-type", "application/x-www-form-urlencoded");
response = client.Execute(request);
This works perfectly but the problem is that I can't Login permanently, i can't make another request using this instance of RestRequest class, it gives the error that I am not logged in. I have found a work-around but it doesn't work with some pages, basically I put the url
into the referer
field but I have to make request every time I navigate to another page and this is inconvenient.
How do I login permanently, so I can make new request from the same instance?
Thank you!
After some research, I found that the answer is cookies. Each time a request is made, the response has cookies. I used the cookies like this.
After response = client.Execute(request)
I added this:
request.Resource = url;
request.Method = Method.GET;
foreach (var cookie in response.Cookies)
{
request.AddCookie(cookie.Name , cookie.Value); //this adds every cookie in the previous response.
}
response = client.Execute(request);
//use the response as required
This doesn't create a new instance but uses the same request
to make further requests and I am logged in through it all.
Thank you everyone for your help!