題
我是Webscraping的新手，並嘗試使用ASP.NET C＃從HTMLAgilityPack網站獲取數據。我想解析的HTML結構是：
<li class='subsubnav' id='new-women-clothing'>
<span class='cat-name'>CLOTHING</span>
<ul>
<li><a href="/womenswear/womens-just-in" id="just-in">Just In</a></li>
<li><a href="/womenswear/new-season-exclusives" id="exclusives">Exclusives</a></li>
<li><a href="/womenswear/new-season-dresses" id="dresses-&-gowns">Dresses & Gowns</a></li>
<li><a href="/womenswear/new-season-coats" id="coats">Coats</a></li>
<li><a href="/womenswear/new-season-jackets" id="jackets">Jackets</a></li>
<li><a href="/womenswear/new-season-shirts-and-blouses" id="shirts-&-blouses">Shirts & Blouses</a></li>
<li><a href="/womenswear/new-season-tops" id="tops">Tops</a></li>
<li><a href="/womenswear/new-season-knitwear" id="knitwear">Knitwear</a></li>
<li><a href="/womenswear/new-season-sweatshirts" id="sweatshirts">Sweatshirts</a></li>
<li><a href="/womenswear/new-season-skirts-and-shorts" id="skirts-&-shorts">Skirts & Shorts</a></li>
<li><a href="/womenswear/new-season-trousers" id="trousers">Trousers</a></li>
<li><a href="/womenswear/new-season-jumpsuits" id="jumpsuits">Jumpsuits</a></li>
<li><a href="/womenswear/new-season-jeans" id="jeans">Jeans</a></li>
<li><a href="/womenswear/new-season-swimwear" id="swimwear">Swimwear</a></li>
<li><a href="/womenswear/new-season-lingerie" id="lingerie">Lingerie</a></li>
<li><a href="/womenswear/new-season-nightwear" id="nightwear">Nightwear</a></li>
<li><a href="/womenswear/sportswear" id="sportswear">Sportswear</a></li>
<li><a href="/womenswear/ski-wear" id="ski-wear">Ski Wear</a></li>
</ul>
</li>
我得到的父類別在這種情況下是完美的衣服，但我無法獲得ul內的元素。
這是我的c＃代碼：
var html = new HtmlDocument();
html.LoadHtml(new WebClient().DownloadString("http://www.harrods.com/men/t-shirts?icid=megamenu_MW_clothing_t_shirts"));
var root = html.DocumentNode;
var nodes = root.Descendants();
var totalNodes = nodes.Count();
var dt = root.Descendants().Where(n => n.GetAttributeValue("class", "").Equals("cat-name"));
foreach(var x in dt)
{
foreach (var element in x.Descendants("ul"))
{
child_data.Add(new cat_childs(element.InnerText));
}
data.Add(new Categories(x.InnerText,child_data));
}
test.DataSource = data;
test.DataBind();
那麼如何在<ul>獲取錨標籤的鏈接和文本？
一般承認的答案
如果您希望使用class='cat-name'將迭代基於span ，那麼與span的目標ul關係將遵循兄弟而不是後代 。您可以使用SelectNodes()從當前span獲取以下兄弟元素，如下所示：
foreach (var x in dt)
{
foreach (var element in x.SelectNodes("following-sibling::ul/li/a"))
{
child_data.Add(new cat_childs(element.InnerText));
}
data.Add(new Categories(x.InnerText,child_data));
}
更新：
似乎實際問題是在外部循環外聲明的child_data變量。這意味著您不斷向同一個child_data實例添加項目。嘗試在foreach (var x in dt){之後的外部循環中聲明它。或者，您可以將整個代碼編寫為LINQ表達式，如下所示：
var data = (from d in dt
let child_data = x.SelectNodes("following-sibling::ul/li/a")
.Select(o => new cat_childs(o.InnerText))
.ToList()
select new Categories(x.InnerText, child_data)
).ToList();
熱門答案
使用此xpath。它將獲得包含具有class ='cat-name'的<span>的所有<li>。之後它會選擇<li>所包含的所有<a>。
//If the span has no influence on what you want you can simply use:
//HtmlNodeCollection hNC = htmlDoc.DocumentNode.SelectNodes("//ul/li/a");
HtmlNodeCollection hNC = htmlDoc.DocumentNode.SelectNodes("//li/span[@class='cat-name']/parent::*/ul/li");
foreach (HtmlNode h in hNC)
{
Console.Write(h.InnerText+" ");
Console.WriteLine(h.GetAttributeValue("href", ""));
}