I try to scrap offers from http://olx.pl/ site, I'm using HttpClient, problem is that site retrived from client is way diffrient and doesn't contain offers list like it is in source code accessed directly from browser. Any idea? Here's my code:
string url = "http://olx.pl/oferty/q-diablo/?search%5Bdescription%5D=1";
HttpClient client = new HttpClient();
string result = await client.GetStringAsync(url);
HttpClient
wont load content that is generated from javascript. Instead you can use WebView that will run js. I ran both, HttpClient
result had length of 235507 and WebView
result lenght of 464476.
WebView wv = new WebView();
wv.NavigationCompleted += Wv_NavigationCompleted;
wv.Navigate(new Uri(url));
private async void Wv_NavigationCompleted(WebView sender, WebViewNavigationCompletedEventArgs args)
{
string wvresult = await sender.InvokeScriptAsync("eval", new string[] { "document.documentElement.outerHTML;" });
}