Question
i have the following vb.net code which works fine and in the message box i see the exact number of elements which have an id attribute.
Dim hreftext = htmldoc.DocumentNode.SelectNodes("//*[@id]")
MsgBox(hreftext.Count)
now the problem is when i use the following on the message box it gives me Object reference not set to an instance of an object although there are 6 elemnts with the id rso
Dim hreftext = htmldoc.DocumentNode.SelectNodes("//*[@id='rso']")
MsgBox(hreftext.Count)
is there any thing wrong with the second snippet?
Accepted Answer
Having read another one of your SO questions, it seems you are trying to scrape google shopping, and have neglected to check the downloaded html source, rather than the generated dom.
The id "rso" does not appear in the html source, which is why you are getting the error. Google is not keen on being scraped, and certainly make it difficult.
To see what i mean, you can add a multiline textbox to your form and add this instead of your current xpath code:
TextBox1.Text = htmldoc.DocumentNode.OuterHtml
Pretty isnt it!!
Popular Answer
To elaborate on checking the case:
Try:
Dim hreftext = htmldoc.DocumentNode.SelectNodes("//*[translate(@id,'ABCDEFGHIJKLMNOPQRSTUVWXYZ','abcdefghijklmnopqrstuvwxyz')='rso']")
MsgBox(hreftext.Count)
To grab any node that is equal to any case combination of 'rso'