vb.net - 如何使用带有BlockingCollection的WebClient正确实现线程下载？

Question

我正在尝试制作一个限制为4个并发下载的多线程下载管理器。在我的研究中，我遇到了以下问题:C# Downloader: should I use Threads, BackgroundWorker or ThreadPool?

[编辑]更新代码:

Imports System.Net
Imports System.Collections.Concurrent
Imports System.ComponentModel
Imports System.Threading

Public Class Form1

    Const MaxClients As Integer = 4
    ' create a queue that allows the max items
    Dim ClientQueue As New BlockingCollection(Of WebClient)(MaxClients)

    ' queue of urls to be downloaded (unbounded)
    Dim UrlQueue As New Queue(Of String)()

    Dim downloadThread As Thread

    'Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click

        ' create four WebClient instances and put them into the queue
        For i As Integer = 0 To MaxClients - 1
            Dim cli = New WebClient()
            AddHandler cli.DownloadFileCompleted, AddressOf DownloadFileCompleted
            AddHandler cli.DownloadProgressChanged, AddressOf DownloadProgressChanged

            ClientQueue.Add(cli)
        Next

        ' Fill the UrlQueue here
        UrlQueue.Enqueue("http://www.gnu.org/licenses/gpl-1.0.txt")
        UrlQueue.Enqueue("http://www.gnu.org/licenses/gpl-2.0.txt")
        UrlQueue.Enqueue("http://www.gnu.org/licenses/gpl-3.0.txt")
        UrlQueue.Enqueue("http://www.gnu.org/licenses/lgpl-2.1.txt")
        UrlQueue.Enqueue("http://www.gnu.org/licenses/lgpl-3.0.txt")
        UrlQueue.Enqueue("http://www.gnu.org/licenses/fdl-1.1.txt")
        UrlQueue.Enqueue("http://www.gnu.org/licenses/fdl-1.2.txt")
        UrlQueue.Enqueue("http://www.gnu.org/licenses/fdl-1.3.txt")

        downloadThread = New Thread(AddressOf downloadQueue)
        downloadThread.IsBackground = True
        downloadThread.Start()
    End Sub

    Private Sub downloadQueue()
        ' Now go until the UrlQueue is empty
        While UrlQueue.Count > 0
            Dim cli As WebClient = ClientQueue.Take() ' blocks if there is no client available

            Dim url As String = UrlQueue.Dequeue()

            Dim fname As String = CreateOutputFilename(url)
            cli.DownloadFileAsync(New Uri(url), fname, New DownloadArgs(url, fname, cli))
            AppendText(url & " started" & vbCrLf)
        End While
    End Sub

    Private Sub DownloadProgressChanged(sender As Object, e As DownloadProgressChangedEventArgs)
        Dim args As DownloadArgs = DirectCast(e.UserState, DownloadArgs)
        ' Do status updates for this download
    End Sub

    Private Sub DownloadFileCompleted(sender As Object, e As AsyncCompletedEventArgs)
        Dim args As DownloadArgs = DirectCast(e.UserState, DownloadArgs)
        ' do whatever UI updates

        Dim url As String = "Filename" '<============I'd like to be able to pass the filename or URL but can't figure this out
        AppendText(url & " completed" & vbCrLf)

        ' now put this client back into the queue
        ClientQueue.Add(args.Client)
    End Sub

    Public Function CreateOutputFilename(ByVal url As String) As String
        Try
            Return url.Substring(url.LastIndexOf("/") + 1)
        Catch ex As Exception
            Return url
        End Try        
    End Function

    Private Delegate Sub SetTextCallback(text As String)

    Private Sub AppendText(text As String)
        If Me.TextBox1.InvokeRequired Then
            TextBox1.Invoke(New Action(Of String)(AddressOf AppendText), text)
            Return
        End If
        Me.TextBox1.AppendText(text)
        Me.TextBox1.SelectionStart = TextBox1.TextLength
        Me.TextBox1.ScrollToCaret()
    End Sub
End Class

Class DownloadArgs
    Public ReadOnly Url As String
    Public ReadOnly Filename As String
    Public ReadOnly Client As WebClient
    Public Sub New(u As String, f As String, c As WebClient)
        Url = u
        Filename = f
        Client = c
    End Sub
End Class

这将成功下载UrlQueue中的前4个文件，但随后似乎冻结了，不再下载其他文件。我以为问题出在从C#转换到vb.net的过程中我错过了一些小问题，但我似乎无法弄清楚。

Answer 1

ClientQueue.Take()阻止UI线程。另外，WebClient将要引发UI线程上的DownloadFileCompleted事件-但已被ClientQueue.Take()阻止。你陷入僵局。

为了解决这个问题，您必须将阻塞循环移至另一个后台线程。