HtmlAgilityPack
을 사용하여 대부분의 웹 사이트에서 HtmlAgilityPack
작동하는 URL의 텍스트를 가져 왔으며 일부는 오늘 오류를 반환하기 시작했습니다.
오류는 회선 코드 doc = webGet.Load(url);
오류 메시지 : The underlying connection was closed: An unexpected error occurred on a send.
내가이 오류를 얻는 이유를 잘 모르겠다. 이전에이 웹 사이트 URL url : link
나는 bbc.com과 같은 https URL을 시도하고 그것을 위해 작동합니다. 코드에 문제가있는 경우 모든 포인터
HtmlDocument doc = new HtmlDocument();
var url = txtGrabNewsURL.Text.Trim();
var webGet = new HtmlWeb();
doc = webGet.Load(url);
var baseUrl = new Uri(url);
// doc.LoadHtml(response);
String title = (from x in doc.DocumentNode.Descendants()
where x.Name.ToLower() == "title"
select x.InnerText).FirstOrDefault();
String desc = (from x in doc.DocumentNode.Descendants()
where x.Name.ToLower() == "meta"
&& x.Attributes["name"] != null
&& x.Attributes["name"].Value.ToLower() == "description"
select x.Attributes["content"].Value).FirstOrDefault();
String ogImage = (from x in doc.DocumentNode.Descendants()
where x.Name.ToLower() == "meta"
&& x.Attributes["property"] != null
&& x.Attributes["property"].Value.ToLower() == "og:image"
select x.Attributes["content"].Value).FirstOrDefault();
List<String> imgs = (from x in doc.DocumentNode.Descendants()
where x.Name.ToLower() == "img"
&& x.Attributes["src"] != null
select x.Attributes["src"].Value).ToList<String>();
List<String> imgList = (from x in doc.DocumentNode.Descendants("img")
where x.Attributes["src"] != null
select x.Attributes["src"].Value.ToLower()).ToList<String>();
전체 오류 세부 정보
System.Net.WebException was caught
HResult=-2146233079
Message=The underlying connection was closed: An unexpected error occurred on a send.
Source=System
StackTrace:
at System.Net.HttpWebRequest.GetResponse()
at HtmlAgilityPack.HtmlWeb.Get(Uri uri, String method, String path, HtmlDocument doc, IWebProxy proxy, ICredentials creds) in D:\Source\htmlagilitypack.new\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1355
at HtmlAgilityPack.HtmlWeb.LoadUrl(Uri uri, String method, WebProxy proxy, NetworkCredential creds) in D:\Source\htmlagilitypack.new\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1479
at HtmlAgilityPack.HtmlWeb.Load(String url, String method) in D:\Source\htmlagilitypack.new\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1106
at HtmlAgilityPack.HtmlWeb.Load(String url) in D:\Source\htmlagilitypack.new\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1061
at _admin_News.btnGrabNews_Click(Object sender, EventArgs e) in c:\path\News.aspx.cs:line 361
InnerException: System.IO.IOException
HResult=-2146232800
Message=Authentication failed because the remote party has closed the transport stream.
Source=System
StackTrace:
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.TlsStream.CallProcessAuthentication(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result)
at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.ConnectStream.WriteHeaders(Boolean async)
InnerException:
HTTP S 리소스에서만 발생하는 경우 .Net 4를 타겟팅하고 있으며 기본 SSL / TLS 지원과 관련 될 수 있습니다. 다음을 시도하십시오.
using System.Net;
static void Main()
{
//place this anywhere in your code prior to invoking the Web request
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12 | SecurityProtocolType.Ssl3;
}
내 로컬 컴퓨터에서 코드를 실행하고있어 정상적으로 작동하며 오류없이 결과를 얻었습니다. 그 시간 웹 사이트가 작동하지 않는 줄 알았는데, 연결 문제가 발생했습니다.
HtmlDocument doc = new HtmlDocument();
var url = "https://m.gulfnews.com/business/sectors/banking/rebuilding-lives-10-years-after-lehman-s-fall-1.2277318"
var webGet = new HtmlWeb();
doc = webGet.Load(url);
String title = (from x in doc.DocumentNode.Descendants()
where x.Name.ToLower() == "title"
select x.InnerText).FirstOrDefault();
산출물 : 생명력 재건, 10. . . . . 곧.