Using htmlAgilityPack scraping all inner text from tag

.net c# html-agility-pack web-scraping xpath

Question

I want to scrap all the word from the link http://search.freefind.com/siteindex.html?id=59478474&ltr=10240&fwr=0&pid=i&ics=1 I tried something like this:

HtmlWeb web = new HtmlWeb();
HtmlAgilityPack.HtmlDocument doc = web.Load("http://search.freefind.com/siteindex.html?id=59478474&ltr=10240&fwr=0&pid=i&ics=1");
HtmlNodeCollection nodes = doc.DocumentNode.SelectNodes("//font[@class='search-index-font']//a");

if (nodes != null)
{

    foreach (HtmlNode n in nodes)
    {
         link = n.InnerText;
        my_link.Add(link);
        MessageBox.Show(link);
    }

}
else
    MessageBox.Show("no wordfound ");

My expexted output should like 

a    
aa
aachhe
aagrashi
aagun
aaj
aam
aanka
aankhi
aar
aashman
abāddhō
abāddhōtā
abadh
..
..

But it didn't work??It shows "no word found" Means it returns null.How can i get all text from < a > tag in that case??? Can anyone tell me What should be in SelectNodes("")???

1
0
2/22/2013 2:21:34 PM

Fastest Entity Framework Extensions

Bulk Insert
Bulk Delete
Bulk Update
Bulk Merge

Accepted Answer

You need to search for the next text node after <script> tag(not <a> tag as you said), inside <font class='search-index-font'>. This xpath expression will do the trick:

//font[@class='search-index-font']/script/following-sibling::text()[1]

And this code:

HtmlWeb web = new HtmlWeb();
HtmlAgilityPack.HtmlDocument doc = web.Load("http://search.freefind.com/siteindex.html?id=59478474&ltr=10240&fwr=0&pid=i&ics=1");
HtmlNodeCollection nodes = doc.DocumentNode.SelectNodes("//font[@class='search-index-font']/script/following-sibling::text()[1]");

will return text nodes you need:

a
aa
aachhe
aagrashi
aagun
aaj
aam
aanka
aankhi
aar
...
0
2/22/2013 2:55:08 PM

Popular Answer

HtmlWeb web = new HtmlWeb();
HtmlAgilityPack.HtmlDocument doc =
    web.Load("http://search.freefind.com/siteindex.html?id=59478474&ltr=10240&fwr=0&pid=i&ics=1");
HtmlNodeCollection nodes = doc.DocumentNode.SelectNodes("//font[@class='search-index-font']");
string link = string.Empty;
if (nodes != null)
{
    foreach (var item in nodes)
    {
        var value =
        item.Elements("script").ToList();
        foreach (var items in value)
        {
            link += items.NextSibling.InnerText+ "\n";
        }
    }
    MessageBox.Show(link);
}
else
    MessageBox.Show("no wordfound ");
user2099420

Related Questions

htmlagilitypack may be used to get p tags that come after a specified tag.

I'm crawling a website using htmlagilitypack c#:...i have in the source code of an html page .... <p>this a p that come before h3</p> .... .... <h3>this h3 </h3> <p>first p after h3</p> .... <p>seconde p after h3</p> ...i want to to all get all Ps that come after ...is there a way to use a where to filter Ps using position....where (position(p)...
c# html-agility-pack web-crawler
asked by user594166

choose a> tag with given inner text with htmlagilitypack

One way (it matches case sensitive):...//a[contains(text(), 'wake')]
.net html-agility-pack xpath
asked by Birei

Selecting Inner Text Using HtmlAgilityPack

I am trying to extract some inner text using HtmlAgilityPack. Here is the HTML of interest:...<select name="Archives" onchange="javascript:setTimeout(&#39;__doPostBack(\&#39;Archives\&#39;,\&#39;\&#39;)&#39;, 0)" id="Archives" style="width:200px;"> <option selected="selected" value="Dashboard_Jul-2012">Dashboard_Jul-2012</option> <option v...
c# html html-agility-pack
asked by Kevin

Removing Html tags while preserving inner text and <br> tags

I'm trying to remove span,font, b, s, strike (and other inner element) tags from HTML content while preserving text and ...<br>... tags inside it. For this purpose I'm using HTML agility pack. I managed to preserve text but ...<br>... tags are still a problem. Any ideas?...Here is the code:... private void removeTagsButPreserveText2(HtmlNode nodeTo...
c# html-agility-pack html-parsing
asked by Rade Milovic

Inner text of Node ignoring inner text of children

The ...div... could possibly have multiple text nodes if there is text before and after its children. As I similarly indicated ...here..., I think the best way to get all the direct text content of a node is to do something like:...HtmlDocument page = new HtmlWeb().Load(url); var nodes = page.DocumentNode.SelectNodes("//div[@id='div1']//div[@class...
c# html-agility-pack xpath
asked by JLRishe

Using HTMLAgilityPack Extract text, which is not between tags and comes after specific node

HTML code:... <b> CAR </b> <br></br> Car is something you can drive. <br></br> <br></br> ...C# code:... HtmlAgilityPack.HtmlDocument doc = new HtmlWeb().Load("http://website.com/x.html"); if (doc != null) { HtmlNode link = doc.DocumentNode.SelectSingleNode("//b[contains(text(), 'CAR')]"); ...
c# html html-agility-pack web-scraping xpath
asked by GiedriusK

how to replace <br> tag with <br/> tag using HtmlAgilityPack?

document.OptionWriteEmptyNodes = true; ...Will do the trick for you!
c# html-agility-pack
asked by ragmn

Append a node to inner text using HTMLAgilityPack

Problem : I Need to remove the style property of all the ...<p>... tags and if it contains ...font-weight:bold... property, then add ...<b>... to it. ...eg :... if the html is ...<p style="margin-top:0pt; margin-bottom:0pt;font-weight:bold; font-weight:bold;font-size:10pt; font-family:ARIAL" align="center"> SOME TEXT HERE</p>.......Output should b...
c# html-agility-pack
asked by user2729272

Get Inner text between two tags and get the output into two labels - VB.NET - HtmlAgilityPack

The short answer is that HAP is not well suited to accomplish your task. My notes below: ...Imports HtmlAgilityPack Public Class Form1 Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click Dim mystring As String = "<BR>Terms of Service<BR></BR>Developers<BR>" Dim myDoc As H...
html html-agility-pack innertext vb.net
asked by user2990854

How to get the inner text for a single node using HtmlAgilityPack

My HTML looks like this:... <div id="footer"> <div id="footertext"> <p> Copyright &copy; FUCHS Online Ltd, 2013. All Rights Reserved. </p> </div> </div> ...I would like to obtain this text from the markup and store it as a string in my C# code: "Copyright ©...
c# html-agility-pack
asked by MD.Amjad Hossain

Fastest Entity Framework Extensions

Bulk Insert
Bulk Delete
Bulk Update
Bulk Merge
View more on Stack Overflow

Prime Library

Performance

Expression Evaluator

More Projects...

Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow