powershell - 检查网址以获取特定文本，如果匹配则下载，如果名称无法解析则继续下一个

Question

我需要此脚本来执行以下操作:

在下载

之前，请先验证$url的某些字符串或正则表达式，以验证我没有被重定向(例如This address有效，this address无效并且重定向到了首页。)

即使出现“无法解析远程名称:”(我正在尝试下载阻止列表，其中一些列表相当定期地获得DDoS)，仍继续处理sources.txt。

当前行为:脚本正在处理5个$url中的3个。第3个错误(故意)是伪造的地址，第4个和第5个错误也是如此。但是，从来没有尝试过第四和第五。

try {
    import-Csv $intelSource | ForEach-Object {
         $storageDir = "$($intelDir)\$($_.threatType) $($_.threatSubtype)"
         $storageName = (Get-Culture).TextInfo
         $threat = $_.threatType + "_" + $_.threatSubtype    # Set this variable to save headaches passing it later
         $storageFile = "$($storageDir)\$($threat)_$($(get-date -f MM-dd-yy)).csv"    # Filename specified by sources.csv fields and today's date
         $url = $_.threatLocation
         if((Test-Path $storageDir) -eq 0) {
            mkdir $storageName.ToTitleCase($storageDir.ToLower());
            }
         # Begin Error Logging
         try {
            $intelCount++
            $webclient.DownloadFile($url,$storageFile)
         }
         catch {
            $intelErrorCount++
            $intelError = "" | Select "Time","Item","Message"
            $intelError.time = get-date -f G
            $intelError.item = $threat
            $intelError.message = $_.Exception.Message
            $errorLog += $intelError
            $intelError = $null
            echo $_.Exception|format-list -force | Out-File $intelDir\$threat"_ErrorLog_"$(get-date -f MM-dd-yy).txt -Append
            Write-Host $_.Exception
            Continue
         }
         # End Error Logging
         # Cleanup
         finally {
            $webclient.Dispose()
         }
         # Throttling (mainly future use with multiple sources)
         Start-Sleep 5
     }
}
finally { 
    if ($intelErrorCount -gt 0) {
        # Table Generation
        $errorLogTable = $errorLog | New-HTMLTable
        $HTML = New-HTMLHead
        $HTML += "<h3>Threat Intel Feed - Errors</h3>"
        $HTML += $errorLogTable | Close-HTML
        $email.Item('Subject') += "$($intelErrorCount) Error(s)"
        $email.Item('Body') += "$($intelErrorCount) of $($intelCount) sources failed. See attached for verbose log(s) `r`n`r`n $($HTML)"
        Get-ChildItem -Path $intelDir | Where {$_.Name -match "$($(get-date -f MM-dd-yy))"} | foreach{$_.fullname} | Send-MailMessage @email -bodyashtml
    }
    else {
        $email.Item('Subject') += "Success"
        $email.Item('Body') += "No errors for $($intelCount) sources."
        Send-MailMessage @email
    }
    $intelErrorCount = $null
    $intelCount = $null
    $errorLogTable = $null
    $errorLog = $null
}

Answer 1

问题是Continue语句。继续和中断在foreach-object脚本块中无法正常工作。更改为foreach语句:

foreach ($_ in (import-Csv $intelSource) {
  # Your existing code will probably work as is here
  # But you might want to rename $_ since it is an automatic variable
}