Pass string of HTML page and scrape with HtmlAgilityPack

c# html-agility-pack vb.net

Question

Why do I get this error?

"Illegal characters in path" at htmlDoc.Load(pageSource)

pageSource is a string variable of the HTML page. I need to pass the page source as a string, not as a file and not as a URL. How do I do this?

Dim ids As New List(Of String)()
Dim pageSource = getHtml(url)

Dim htmlDoc As HtmlDocument = New HtmlDocument()

htmlDoc.OptionFixNestedTags = True


htmlDoc.Load(pageSource)


Dim s As HtmlNodeCollection = htmlDoc.DocumentNode.SelectNodes("//div/@id")

For Each div As HtmlNode In s
    ids.Add(div.Id)
Next
1
1
3/12/2013 3:42:07 PM

Fastest Entity Framework Extensions

Bulk Insert
Bulk Delete
Bulk Update
Bulk Merge

Accepted Answer

Use LoadHtml instead of Load:

htmlDoc.LoadHtml(pageSource)

See also the source.

9
7/21/2012 4:51:33 AM

Related Questions

Using HtmlAgilityPack to pick Divs by class from an HTML page

You can use XPATH like this...//div[@class='bar'] ...or...//*/div[@class='bar'] ...You also may be able to do && ...foo.Class == "bar"....
c# html html-agility-pack
asked by Rodney S. Foley

Scrape data from a wiki page in C# (screen-scraping)

What's wrong with the code? To be blunt, everything. :P...The page is not formatted in the way you are reading it. You can't hope to get the desired contents that way....The contents of the page (the part we're interested in) looks something like this:...<h2> <span id="Deployments" class="mw-headline">Deployments</span> </h2> <p> <!-- ... --...
c# html-agility-pack screen screen-scraping
asked by Jeff Mercado

How to comprehend page redirection and load redirected pages with HtmlAgilityPack

For ...this case..., parse the HTML is the best way. ...var page = "..."; var doc = new HtmlDocument(); doc.Load(page); var root = doc.DocumentNode; var select = root.SelectNodes("//meta[contains(@content, 'URL')]"); try { Console.WriteLine("has redirect.."); Console.WriteLine(select[0].Attributes["content"].Value.Split('=')[1]); } catch { ...
c# html-agility-pack redirect
asked by The Mask

HTML Agility Pack for HTML Scraping

I wouldn't use the Html Agility Pack for this because I don't know how to make it feed back to the originating website. Instead, I'd use ...WatiN.... WatiN is built for driving a browser for testing purposes, but I've found it extremely useful when I have to scrape websites that are outside my control (such as Facebook or Wal-Mart). Downside is tha...
ajax c# html-agility-pack web-scraping
asked by Jacob Proffitt

HtmlAgilityPack reading data from an html page

You made wrong assumptions: ...//a[@id='UserLink']/text()... and ...//a[@id='PointLink']/text()... searches in the entire document. That's why you get the first ...tr... node. Just use:...string UserName = row.SelectSingleNode("td[@class='tbl_col1']/a[@id='UserLink']/text()").InnerText; string Points = row.SelectSingleNode("td[@class='tbl_col2']/a[...
c# html-agility-pack
asked by Alex

Parsing HTML page with HtmlAgilityPack using LINQ

the document you have to parse is not the most well formed for parsing many elements are missing the class or at least id attribute but what you want to get is a second p tag content in it ...you can try this ...HtmlDocument htmlDocument = new HtmlDocument(); htmlDocument.LoadHtml(html); var pNodes = htmlDocument.DocumentNode.SelectNodes("//p") ...
c# html-agility-pack linq
asked by COLD TOLD

html agility pack url scraping-- getting full html link

You can check the HREF value if it's relative URL or absolute. Load the link into a ...Uri... and test whether it is relative If it relative convert it to absolute will be the way to go....static void Main(string[] args) { List<string> linksToVisit = ParseLinks("https://www.facebook.com"); } public static List<string> ParseLinks(st...
c# html-agility-pack url web-crawler web-scraping
asked by Vikash Rathee

How to check if it is 404 error page(page does not exist) using HtmlAgilityPack

You'll need to register a ...PostRequestHandler... event on the ...HtmlWeb... instance, it will be raised after each downloaded document and you'll get access to the ...HttpWebResponse... object. It has a property for the ...StatusCode....... HtmlWeb web = new HtmlWeb(); HttpStatusCode statusCode = HttpStatusCode.OK; web.PostRequestHandler += (re...
c# html-agility-pack
asked by jessehouwing

How to scrape values from a web page using Html Agility Pack

I would do it like...var list = doc.DocumentNode.SelectSingleNode("//table[@class='table-main js-tablebanner-t js-tablebanner-ntb']") .Descendants("tr") .Select(x => new { Val1 = x.SelectSingleNode("td[@class='h-text-left']")?.InnerText, Val2 = x.SelectSingleNo...
c# html html-agility-pack web-scraping
asked by L.B

HtmlAgilityPack scraping - extracting specific nodes from html document

Here is how to get the HTML for one day of matches using Selenium. Rest is HtmlAgilityPack. The site uses self signed certificates so I had to configure the driver to accept self signed certificates. Have fun.... var ffOptions = new FirefoxOptions(); ffOptions.BrowserExecutableLocation = @"C:\Program Files (x86)\Mozilla Firefox\firef...
c# html html-agility-pack web-scraping xpath
asked by Kent Kostelac

Fastest Entity Framework Extensions

Bulk Insert
Bulk Delete
Bulk Update
Bulk Merge
View more on Stack Overflow

Prime Library

Performance

Expression Evaluator

More Projects...

Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow