.NET 4.5 HTTPWebRequest - 大量并发 HTTPS POST 请求导致间歇性速度减慢

Question

我有大量(50+)线程，它们向 Web api 发出 HTTPS POST 请求，以尽快轮​​询不同的数据集。它在一半的时间里工作正常，但在另一半的时间里就会出现问题，应用程序使用的带宽会显着下降。

更新 1 下面列出的代码不再是最新的，它已根据 Domin8urMind 的建议进行了修改。

所以我查看了fiddler中的所有流量。一旦速度开始变慢，我注意到我的客户端建立了新的 HTTPS 隧道，并且速度变慢在新隧道建立后结束。程序首次启动时建立的 50 多个隧道只需要大约 1 秒即可建立，但在减速期间则需要大约 30 秒才能建立所有隧道。这种重建/减速似乎每 2 分钟就会发生一次。

我还注意到 CDN 位于我的客户端和服务器之间，并且它确实具有 DoS 保护。我不确定这个性能问题是否与我的代码有关，或者与 CDN 有关。

所以我想我现在的问题是:

• 启用保活和管道功能后，在什么情况下 HTTPWebRequest 的 HTTPS 连接会被关闭？
• 为什么需要这么长时间才能重新建立这些连接？

我的方法

private static string myMethod(string method, Dictionary<string, string> paramList = null, int maxRetryCount = 1)
{
    System.Net.ServicePointManager.Expect100Continue = false;
    System.Net.ServicePointManager.DefaultConnectionLimit = 400;
    System.Net.ServicePointManager.UseNagleAlgorithm = false;

    HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("My URL");
    String postData = "My POST Data"

    request.ContentType = "application/x-www-form-urlencoded";
    request.ContentLength = postData.Length;
    request.Method = "POST";
    request.Timeout = 1000;
    request.Headers.Add("Accept-Encoding", "gzip,deflate");
    request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
    request.Proxy = null;

    try
    {
        Stream stream = request.GetRequestStream();
        stream.Write(postData, 0, postData.Length);
        stream.Close();

        HttpWebResponse response = (HttpWebResponse)request.GetResponse();
        Stream responseStream = response.GetResponseStream();
        Stream decompressedStream = null;
        if (response.ContentEncoding.ToLower().Contains("gzip"))
            decompressedStream = new GZipStream(responseStream, CompressionMode.Decompress);
        else if (response.ContentEncoding.ToLower().Contains("deflate"))
            decompressedStream = new DeflateStream(responseStream, CompressionMode.Decompress);
        else
            decompressedStream = responseStream;

        StreamReader postreqreader = new StreamReader(decompressedStream);
        var json = postreqreader.ReadToEnd();

        postreqreader.Close();
        decompressedStream.Close();
        responseStream.Close();

        return json;
    }
    catch (Exception e)
    {
        if(e.Message != "The operation has timed out")
            Console.Write("\nGot exception " + e.Message + " " + e.StackTrace + "\n\n\n");
    }
    return null;
}

}

异常 1

Got exception The request was aborted: The request was canceled.    at System.Ne
t.GZipWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.IO.StreamReader.ReadBuffer()
   at System.IO.StreamReader.ReadToEnd()
   at myMethod in myFile:line 337

第337行是“var json = postreqreader.ReadToEnd();”

异常(exception)2

Got exception Safe handle has been closed    at System.Runtime.InteropServices.S
afeHandle.DangerousAddRef(Boolean& success)
   at System.StubHelpers.StubHelpers.SafeHandleAddRef(SafeHandle pHandle, Boolea
n& success)
   at System.Net.UnsafeNclNativeMethods.OSSOCK.setsockopt(SafeCloseSocket socket
Handle, SocketOptionLevel optionLevel, SocketOptionName optionName, Int32& optio
nValue, Int32 optionLength)
   at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, S
ocketOptionName optionName, Int32 optionValue, Boolean silent)
   at System.Net.Sockets.NetworkStream.SetSocketTimeoutOption(SocketShutdown mod
e, Int32 timeout, Boolean silent)
   at System.Net.ConnectStream.DrainSocket()
   at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean abort
ing)
   at System.Net.ConnectStream.System.Net.ICloseEx.CloseEx(CloseExState closeSta
te)
   at System.Net.HttpWebRequest.SetAndOrProcessResponse(Object responseOrExcepti
on)
   at System.Net.ConnectionReturnResult.SetResponses(ConnectionReturnResult retu
rnResult)
   at System.Net.Connection.ReadComplete(Int32 bytesRead, WebExceptionStatus err
orStatus)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetriev
edStream, Boolean probeRead)
   at System.Net.ConnectStream.ProcessWriteCallDone(ConnectionReturnResult retur
nResult)
   at System.Net.HttpWebRequest.CheckDeferredCallDone(ConnectStream stream)
   at System.Net.HttpWebRequest.GetResponse()
   at myMethod in myFile:line 325

第325行是“HttpWebResponse response = (HttpWebResponse)request.GetResponse();”

Answer 1

我建议您使用“using”子句更改代码，这将正确关闭连接，例如:

HttpWebRequest httpWebRequest = (HttpWebRequest)HttpWebRequest.Create("My URL");

using (HttpWebResponse httpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse())
{
  using (Stream stream = httpWebResponse.GetResponseStream())
  {
    using (StreamReader reader = new StreamReader(stream))
    {
      ...
    }
  }
}

您现有循环遇到的问题是，当您捕获异常时，连接不会被正确处理，您需要等待垃圾收集器清理这些连接，然后才能创建另一个连接。

我怀疑这是你的延误。您可以通过观察 perfmon 中的 .net GC 计数器来验证这一点，如果“% in GC”在这些“减速”期间出现峰值，那么您就找到了罪魁祸首。