使用HTMLAgilityPack库检索属性和跨度

.net html html-agility-pack html-parsing vb.net

在这段HTML代码中:

<div class="item">

    <div class="thumb">
        <a href="http://www.mp3crank.com/wolf-eyes/lower-demos-121866" rel="bookmark" lang="en" title="Wolf Eyes - Lower Demos album downloads">
        <img width="100" height="100" alt="Mp3 downloads Wolf Eyes - Lower Demos" title="Free mp3 downloads Wolf Eyes - Lower Demos" src="http://www.mp3crank.com/cover-album/Wolf-Eyes-–-Lower-Demos.jpg" /></a>
    </div>

    <div class="release">
        <h3>Wolf Eyes</h3>
        <h4>
        <a href="http://www.mp3crank.com/wolf-eyes/lower-demos-121866" title="Wolf Eyes - Lower Demos">Lower Demos</a>
        </h4>
        <script src="/ads/button.js"></script>
    </div>

    <div class="release-year">
        <p>Year</p>
        <span>2013</span>
    </div>

    <div class="genre">
        <p>Genre</p>
        <a href="http://www.mp3crank.com/genre/rock" rel="tag">Rock</a>
        <a href="http://www.mp3crank.com/genre/pop" rel="tag">Pop</a>
    </div>

</div>

我知道如何以其他方式解析它,但我想使用HTMLAgilityPack库检索此信息:

Title : Wolf Eyes - Lower Demos
Cover : http://www.mp3crank.com/cover-album/Wolf-Eyes-–-Lower-Demos.jpg
Year  : 2013
Genres: Rock, Pop
URL   : http://www.mp3crank.com/wolf-eyes/lower-demos-121866

这些html行是哪些:

Title : title="Wolf Eyes - Lower Demos"
Cover : src="http://www.mp3crank.com/cover-album/Wolf-Eyes-–-Lower-Demos.jpg"
Year  : <span>2013</span>
Genre1: <a href="http://www.mp3crank.com/genre/rock" rel="tag">Rock</a>
Genre2: <a href="http://www.mp3crank.com/genre/pop" rel="tag">Pop</a>
URL   : href="http://www.mp3crank.com/wolf-eyes/lower-demos-121866" 

这就是我正在尝试的,但我总是在尝试选择单个节点时得到一个object reference not set异常,抱歉但我是HTML的新手,我试图按照这个问题的步骤HtmlAgilityPack基本如何获得标题和链接?

Public Class Form1

    Private htmldoc As HtmlAgilityPack.HtmlDocument = New HtmlAgilityPack.HtmlDocument
    Private htmlnodes As HtmlAgilityPack.HtmlNodeCollection = Nothing

    Private Title As String = String.Empty
    Private Cover As String = String.Empty
    Private Genres As String() = {String.Empty}
    Private Year As Integer = -0
    Private URL as String = String.Empty

    Private Sub Test() Handles MyBase.Shown

        ' Load the html document.
        htmldoc.LoadHtml(IO.File.ReadAllText("C:\source.html"))

        ' Select the (10 items) nodes.
        htmlnodes = htmldoc.DocumentNode.SelectNodes("//div[@class='item']")

        ' Loop trough the nodes.
        For Each node As HtmlAgilityPack.HtmlNode In htmlnodes

            Title = node.SelectSingleNode("//div[@class='release']").Attributes("title").Value
            Cover = node.SelectSingleNode("//div[@class='thumb']").Attributes("src").Value
            Year = CInt(node.SelectSingleNode("//div[@class='release-year']").Attributes("span").Value)
            Genres = ¿select multiple nodes?
            URL = node.SelectSingleNode("//div[@class='release']").Attributes("href").Value

        Next

    End Sub

End Class

一般承认的答案

你在这里的错误是试图从你找到的那个中访问一个childnode的属性。

当你调用node.SelectSingleNode("//div[@class='release']")你得到了正确的div返回,但调用.Attributes只返回div标签本身的属性,而不是任何内部HTML元素。

可以编写选择子节点的XPATH查询,例如//div[@class='release']/a - 有关XPATH的更多信息,请参阅http://www.w3schools.com/xpath/xpath_syntax.asp 。虽然示例适用于XML,但大多数原则应适用于HTML文档。

另一种方法是在您找到的节点上使用进一步的XPATH调用。我修改了你的代码,使它能够使用这种方法:

' Load the html document.
htmldoc.LoadHtml(IO.File.ReadAllText("C:\source.html"))

' Select the (10 items) nodes.
htmlnodes = htmldoc.DocumentNode.SelectNodes("//div[@class='item']")

' Loop through the nodes.
For Each node As HtmlAgilityPack.HtmlNode In htmlnodes

    Dim releaseNode = node.SelectSingleNode(".//div[@class='release']")
    'Assumes we find the node and it has a a-tag
    Title = releaseNode.SelectSingleNode(".//a").Attributes("title").Value
    URL = releaseNode.SelectSingleNode(".//a").Attributes("href").Value

    Dim thumbNode = node.SelectSingleNode(".//div[@class='thumb']")
    Cover = thumbNode.SelectSingleNode(".//img").Attributes("src").Value

    Dim releaseYearNode = node.SelectSingleNode(".//div[@class='release-year']")
    Year = CInt(releaseYearNode.SelectSingleNode(".//span").InnerText)

    Dim genreNode = node.SelectSingleNode(".//div[@class='genre']")
    Dim genreLinks = genreNode.SelectNodes(".//a")
    Genres = (From n In genreLinks Select n.InnerText).ToArray()

    Console.WriteLine("Title : {0}", Title)
    Console.WriteLine("Cover : {0}", Cover)
    Console.WriteLine("Year  : {0}", Year)
    Console.WriteLine("Genres: {0}", String.Join(",", Genres))
    Console.WriteLine("URL   : {0}", URL)

Next

请注意,在此代码中,我们假设文档已正确形成,并且每个节点/元素/属性都存在且正确。您可能想要为此添加大量错误检查,例如, If someNode Is Nothing Then ....

编辑:我稍微修改了上面的代码,以确保每个.SelectSingleNode使用“.//”前缀 - 这确保它有效,如果有几个“项目”节点,否则它选择文件中的第一个匹配不是当前节点。

如果您想要更短的XPATH解决方案,请使用以下方法使用相同的代码:

' Load the html document.
htmldoc.LoadHtml(IO.File.ReadAllText("C:\source.html"))

' Select the (10 items) nodes.
htmlnodes = htmldoc.DocumentNode.SelectNodes("//div[@class='item']")

' Loop through the nodes.
For Each node As HtmlAgilityPack.HtmlNode In htmlnodes

    Title = node.SelectSingleNode(".//div[@class='release']/h4/a[@title]").Attributes("title").Value
    URL = node.SelectSingleNode(".//div[@class='release']/h4/a[@href]").Attributes("href").Value

    Cover = node.SelectSingleNode(".//div[@class='thumb']/a/img[@src]").Attributes("src").Value

    Year = CInt(node.SelectSingleNode(".//div[@class='release-year']/span").InnerText)

    Dim genreLinks = node.SelectNodes(".//div[@class='genre']/a")
    Genres = (From n In genreLinks Select n.InnerText).ToArray()

    Console.WriteLine("Title : {0}", Title)
    Console.WriteLine("Cover : {0}", Cover)
    Console.WriteLine("Year  : {0}", Year)
    Console.WriteLine("Genres: {0}", String.Join(",", Genres))
    Console.WriteLine("URL   : {0}", URL)
    Console.WriteLine()

Next

热门答案

你离解决方案的距离不远。两个重要说明:

  • //是一个递归调用。它可能会产生一些严重的性能影响,并且它可能会选择您不想要的节点,因此我建议您仅在层次结构很深或复杂或可变时使用它,并且您不希望指定整个路径。
  • XmlNode上有一个名为GetAttributeValue的有用辅助方法,即使它不存在也会获得一个属性(您需要指定默认值)。

这是一个似乎有用的示例:

' select the base/parent DIV (here we use a discriminant CLASS attribute)
' all select calls below will use this DIV element as a starting point
Dim node As HtmlNode = htmldoc.DocumentNode.SelectNodes("//div[@class='item']")

' get to the A tag which is a child or grand child (//) of a 'release' DIV
Console.WriteLine(("Title :" & node.SelectSingleNode("div[@class='release']//a").GetAttributeValue("title", CStr(Nothing))))

' get to the IMG tag which is a child or grand child (//) of a 'thumb' DIV
Console.WriteLine(("Cover :" & node.SelectSingleNode("div[@class='thumb']//img").GetAttributeValue("src", CStr(Nothing))))

' get to the SPAN tag which is a child or grand child (//) of a 'release-year' DIV
Console.WriteLine(("Year  :" & node.SelectSingleNode("div[@class='release-year']//span").InnerText))

' get all A elements which are child or grand child(//) of a 'genre' DIV
Dim nodes As HtmlNodeCollection = node.SelectNodes("div[@class='genre']//a")
Dim i As Integer
For i = 0 To nodes.Count - 1
    Console.WriteLine(String.Concat(New Object() { "Genre", (i + 1), ":", nodes.Item(i).InnerText }))
Next i

' get to the A tag which is a child or grand child (//) of a 'release' DIV
Console.WriteLine(("Url   :" & node.SelectSingleNode("div[@class='release']//a").GetAttributeValue("href", CStr(Nothing))))


Related

许可下: CC-BY-SA with attribution
不隶属于 Stack Overflow
这个KB合法吗? 是的,了解原因
许可下: CC-BY-SA with attribution
不隶属于 Stack Overflow
这个KB合法吗? 是的,了解原因