function - Get-ChildItem -Recurse 抛出 System.OutOfMemoryException

Question

我在一个大目录上运行这个函数:

定义

# (searches for all ips within files in the current directory recursively)

function searchips
{
    param(
        [Parameter(Mandatory=$false)][string]$dir = $(pwd)
    )

    ls -Recurse -Force `
    | Select-String -Pattern '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' -AllMatches `
    | ? { 
        $matches = ($_.Matches | Select-Object -Unique)
        return $matches.Count -gt 1 -or $matches[0].Value -ne '127.0.0.1' 
    } `
    | select Path,Matches,FileName,LineNumber,Line `
    | Format-Table -AutoSize `
    | Out-String -Width 4096
}

调用

PS C:\path\to\huge> searchips hugeDirectory >> outfile.txt

但我每次都会收到此错误:

out-lineoutput : Exception of type 'System.OutOfMemoryException' was thrown.
At C:\Users\myUserName\Documents\WindowsPowerShell\profile.ps1:73 char:2
+     ls -Recurse `
+     ~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [out-lineoutput], OutOfMemoryException
    + FullyQualifiedErrorId : System.OutOfMemoryException,Microsoft.PowerShell.Commands.OutLineOutputCommand

目前我的 PS 内存设置如下所示:

   WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Shell

Name                      Value         Type
----                      -----         ----
AllowRemoteShellAccess    true          System.String
IdleTimeout               7200000       System.String
MaxConcurrentUsers        10            System.String
MaxShellRunTime           2147483647    System.String
MaxProcessesPerShell      25            System.String
MaxMemoryPerShellMB       9000000       System.String
MaxShellsPerUser          30            System.String

有什么想法吗？

Answer 1

该问题是由 Out-String 引起的。该 cmdlet 将其输入合并为单个字符串(在将该字符串返回给调用者之前)。为此，它必须收集内存中的所有输出。有点明显，现在我想起来了。

我建议使用 ConvertTo-Csv 而不是 Format-Table |输出字符串。它避免了内存耗尽，并且更容易使用输出进行进一步处理。

function Find-IPAddresses {
    Param(
        [Parameter(Mandatory=$false)]
        [ValidateScript({Test-Path -LiteralPath $_})]
        [string]$dir = $PWD.Path
    )

    Get-ChildItem $dir -Recurse -Force |
        Select-String -Pattern '\d{1,3}(?:\.\d{1,3}){3}' -AllMatches |
        Where-Object {
            $matches = ($_.Matches | Select-Object -Unique)
            $matches.Count -gt 1 -or $matches[0].Value -ne '127.0.0.1'
        } |
        Select-Object Path, Matches, FileName, LineNumber, Line |
        ConvertTo-Csv -NoType
}

Find-IPAddresses 'C:\some\folder' > 'outfile.csv'

或者根本不返回文本输出。只需返回对象列表，并在调用函数时进行所有格式化/输出并知道要如何处理数据:

function Find-IPAddresses {
    Param(
        [Parameter(Mandatory=$false)]
        [ValidateScript({Test-Path -LiteralPath $_})]
        [string]$dir = $PWD.Path
    )

    Get-ChildItem $dir -Recurse -Force |
        Select-String -Pattern '\d{1,3}(?:\.\d{1,3}){3}' -AllMatches |
        Where-Object {
            $matches = ($_.Matches | Select-Object -Unique)
            $matches.Count -gt 1 -or $matches[0].Value -ne '127.0.0.1'
        } |
        Select-Object Path, Matches, FileName, LineNumber, Line
}

Find-IPAddresses 'C:\some\folder' | Export-Csv 'outfile.csv' -NoType