Convert HTML table to CSV using Html Agility Pack

c# csv html-agility-pack linq

Question

I want to extract every element from a table with the id statsTable so that I can read the information into a CSV file.

What I have thus far is as follows:

// Create a request for the URL. 
WebRequest request = WebRequest.Create("http://www.pgatour.com/stats/stat.120.html");
Console.WriteLine("Requesting data from: http://www.pgatour.com/stats/stat.120.html");

// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;

WebResponse response = request.GetResponse();

using (Stream stream = response.GetResponseStream())
{
    StreamReader reader = new StreamReader(stream);

    // covert html to string
    String responseString = reader.ReadToEnd();

    HtmlDocument doc = new HtmlDocument();

    doc.LoadHtml(responseString);

    var desktopFolder = Environment.GetFolderPath(Environment.SpecialFolder.DesktopDirectory);
    var fullFileName = Path.Combine(desktopFolder, "GolfStats.csv");

    using (var PlayerFile = new StreamWriter(fullFileName))
    {
        PlayerFile.WriteLine("Data downloaded: " + DateTime.Now);

        var myTable = doc.DocumentNode
                        .Descendants("table")
                        .Where(table => table.Attributes.Contains("id"))
                        .SingleOrDefault(table => table.Attributes["id"].Value == "statsTable");

        var myTableValues = myTable.Descendants("td");

        foreach (var tdV in myTableValues)
        {
            PlayerFile.WriteLine(tdV.InnerText);
            Console.WriteLine(tdV.InnerText);
        }

        PlayerFile.Flush();
    }
}

The issue is that my csv only picks up an ad that is put in the database and lists the data in a single column (see url in the webRequest). It would be great if you could assist me with exporting the data in tabular format.

1
0
3/8/2014 11:03:38 PM

Accepted Answer

For each table cell, a new line is created. Replace with the following code to make each table row have its own line:

var myTableValues = myTable.Descendants("td");
foreach (var tdV in myTableValues)
{
    PlayerFile.WriteLine(tdV.InnerText);
    Console.WriteLine(tdV.InnerText);
}

with

var myTableRows = myTable.Descendants("tr").Where(tr => tr.Attributes.Contains("id"));
foreach (var tr in myTableRows)
{
    string line = string.Join(";", tr.Descendants("td").Select(td => td.InnerText));
    PlayerFile.WriteLine(line);
    Console.WriteLine(line);
}

The .Where(tr => tr.Attributes.Contains("id")) the table row containing the advertisement is excluded since it lacks an id but all player rows have.

1
3/8/2014 11:02:38 PM


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow