Looping through node created by HtmlAgilityPack

c#-4.0 html-agility-pack xpath

Question

I need to parse this html code using HtmlAgilityPack and C#. I can get the div class="patent_bibdata" node, but I don'know how to loop thru the child nodes.

In this sample there are 6 hrefs, but I need to separate them into two groups; Inventors, Classification. I'm not interested in the last two. There can be any number of hrefs in this div.

As you can see there is a text before the two groups that says what the hrefs are.

code snippet

HtmlWeb hw = new HtmlWeb();
HtmlDocument doc = m_hw.Load("http://www.google.com/patents/US3748943");
string xpath = "/html/body/table[@id='viewport_table']/tr/td[@id='viewport_td']/div[@class='vertical_module_list_row'][1]/div[@id='overview']/div[@id='overview_v']/table[@id='summarytable']/tr/td/div[@class='patent_bibdata']";
HtmlNode node = m_doc.DocumentNode.SelectSingleNode(xpath);

So how would you do this?

<div class="patent_bibdata">
    <b>Inventors</b>:&nbsp;
    <a href="http://www.google.com/search?tbo=p&amp;tbm=pts&amp;hl=en&amp;q=ininventor:%22Ronald+T.+Lashley%22">
    Ronald T. Lashley
    </a>, 
    <a href="http://www.google.com/search?tbo=p&amp;tbm=pts&amp;hl=en&amp;q=ininventor:%22Ronald+T.+Lashley%22">
    Ronald T. Lashley
    </a><br>
    <b>Current U.S. Classification</b>:&nbsp;
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&amp;q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&amp;usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200P">84/312.00P</a>;
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&amp;q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&amp;usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200R">84/312.00R</a><br>
    <br>
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://patft.uspto.gov/netacgi/nph-Parser%3FSect2%3DPTO1%26Sect2%3DHITOFF%26p%3D1%26u%3D/netahtml/PTO/search-bool.html%26r%3D1%26f%3DG%26l%3D50%26d%3DPALL%26RefSrch%3Dyes%26Query%3DPN/3748943&usg=AFQjCNGKUic_9BaMHWdCZtCghtG5SYog-A">
    View patent at USPTO</a><br>
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://assignments.uspto.gov/assignments/q%3Fdb%3Dpat%26pat%3D3748943&usg=AFQjCNGbD7fvsJjOib3GgdU1gCXKiVjQsw">
    Search USPTO Assignment Database
    </a><br>
</div>

Wanted result InventorGroup =

<a href="http://www.google.com/search?tbo=p&amp;tbm=pts&amp;hl=en&amp;q=ininventor:%22Ronald+T.+Lashley%22">
    Ronald T. Lashley
    </a>
    <a href="http://www.google.com/search?tbo=p&amp;tbm=pts&amp;hl=en&amp;q=ininventor:%22Ronald+T.+Lashley%22">
    Thomas R. Lashley
    </a>

ClassificationGroup

<a href="http://www.google.com/url?id=3eF8AAAAEBAJ&amp;q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&amp;usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200P">84/312.00P</a>;
    <a href="http://www.google.com/url?id=3eF8AAAAEBAJ&amp;q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&amp;usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200R">84/312.00R</a>

The page I'm trying to scrape: http://www.google.com/patents/US3748943

// Anders

PS! I know that in this page the names of the inventors are the same, but in most of them they are different!

Accepted Answer

XPATH is your friend! Something like this will get you the inventors name:

HtmlWeb w = new HtmlWeb();
HtmlDocument doc = w.Load("http://www.google.com/patents/US3748943");
foreach (HtmlNode node in doc.DocumentNode.SelectNodes("//div[@class='patent_bibdata']/br[1]/preceding-sibling::a"))
{
    Console.WriteLine(node.InnerHtml);
}

Popular Answer

So it's obvious that I don't understand XPath (yet). So I came up with this solution. Maybe not the smartest solution, but it works!

// Anders

List<string> inventorList = new List<string>();
List<string> classificationList = new List<string>();

string xpath = "/html/body/table[@id='viewport_table']/tr/td[@id='viewport_td']/div[@class='vertical_module_list_row'][1]/div[@id='overview']/div[@id='overview_v']/table[@id='summarytable']/tr/td/div[@class='patent_bibdata']";
HtmlNode nodes = m_doc.DocumentNode.SelectSingleNode(xpath);
bool bInventors = false;
bool bClassification = false;
for (int i = 0; i < nodes.ChildNodes.Count; i++)
{
    HtmlNode node = nodes.ChildNodes[i];
    string txt = node.InnerText;
    if (txt.IndexOf("Inventor") > -1)
    {
        bClassification = false;
        bInventors = true;
    }
    if (txt.IndexOf("Classification") > -1)
    {
        bClassification = true;
        bInventors = false;
    }
    if (txt.IndexOf("USPTO") > -1)
    {
        bClassification = false;
        bInventors = false;
    }
    string name = node.Name;
    if (name.IndexOf("a") > -1)
    {
        if (bInventors)
        {
            string inventor = node.InnerText;
            inventorList.Add(inventor);
        }
        if (bClassification)
        {
            string classification = node.InnerText;
            classificationList.Add(classification);
        }
    }



Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Is this KB legal? Yes, learn why