Bash:并行化 FOR 循环执行等待来自 grep

标签 bash ubuntu parallel-processing grep

关闭。这个问题需要debugging details .它目前不接受答案。












编辑问题以包含 desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem .这将帮助其他人回答问题。


1年前关闭。







Improve this question




grep 进入 FOR 循环以对找到的行执行一些操作,在 FOR 循环内循环搜索要搜索的值,提供并行执行,但结果并不总是可预测的。我想强制执行根本不平行。
和代码示例:

for __loopVar in $(seq 1 32)
do 
    echo "do some stuff no so much time consuming"
    echo "calculate __someTextVar"

    for each_line in  $(grep -HiRF "$__someTextvar" --include \*.log $PATH_logFiles)
    do
        echo "some other stuff, with each line returned by grep"
    done
done
问题,(我假设)在等待 grep 返回值时,它从以下循环(“一些东西”)开始,混合了一些全局变量。
注意:通过多 channel 运行 Ubuntu 20.04.1 LTS。

最佳答案

您的循环是嵌套的,我相信它会给您带来一些困惑,因为我看不到外循环的目的。
如果您添加这些值的一些基本 echo ,我希望它变得更加清晰/明显:

for __loopVar in $(seq 1 3)

do
    echo "__loopVar value is ${__loopVar}"
    for each_line in  $(grep -HiRF "no" --include \sometext.txt $PATH_logFiles)

    do
    echo "__loopVar value2 is ${__loopVar}"
    echo "each_line value2 is ${each_line}"
    done

done
输出:
__loopVar value is 1
__loopVar value2 is 1
each_line value2 is sometext.txt:another
__loopVar value2 is 1
each_line value2 is b
__loopVar value2 is 1
each_line value2 is sometext.txt:and
__loopVar value2 is 1
each_line value2 is another
__loopVar value2 is 1
each_line value2 is d
__loopVar value is 2
__loopVar value2 is 2
each_line value2 is sometext.txt:another
__loopVar value2 is 2
each_line value2 is b
__loopVar value2 is 2
each_line value2 is sometext.txt:and
__loopVar value2 is 2
each_line value2 is another
__loopVar value2 is 2
each_line value2 is d
__loopVar value is 3
__loopVar value2 is 3
each_line value2 is sometext.txt:another
__loopVar value2 is 3
each_line value2 is b
__loopVar value2 is 3
each_line value2 is sometext.txt:and
__loopVar value2 is 3
each_line value2 is another
__loopVar value2 is 3
each_line value2 is d
我认为你想要做的更像是这样的:
IFS=$'\n'
for __loopVar in $(seq 1 3)
do
echo ${__loopVar}
    for each_line in  $(cat sometext.txt)
    do
        echo "some other stuff, with each line returned by grep ${each_line}"
    done
done
所以转义空格(让文件的猫得到整行)
然后循环 _loopVar 次。
$cat sometext.txt
Line a
another b
some other c
and another d
last value is e
$ bash 2.sh
1
some other stuff, with each line returned by grep Line a
some other stuff, with each line returned by grep another b
some other stuff, with each line returned by grep some other c
some other stuff, with each line returned by grep and another d
some other stuff, with each line returned by grep last value is e
2
some other stuff, with each line returned by grep Line a
some other stuff, with each line returned by grep another b
some other stuff, with each line returned by grep some other c
some other stuff, with each line returned by grep and another d
some other stuff, with each line returned by grep last value is e
3
some other stuff, with each line returned by grep Line a
some other stuff, with each line returned by grep another b
some other stuff, with each line returned by grep some other c
some other stuff, with each line returned by grep and another d
some other stuff, with each line returned by grep last value is e
由于您一开始就提供了一个不完整的脚本,因此此代码仍需要更改以匹配您的预期行为。 (例如,将 cat 命令更改为您的工作 grep)。

关于Bash:并行化 FOR 循环执行等待来自 grep,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/64983341/

相关文章:

linux - 如果语句在 bash 中的函数调用后不执行

linux - bash 中的参数

node.js - 在 ubuntu 上安装 npm 模块出现错误

postgresql - AWS Aurora Postgres 中的并行计划/查询

Bash - 打印出匹配参数的行

linux - sudo使用什么shell

ubuntu - Rabbitmq 忽略 Ubuntu 12 上的配置

ruby-on-rails - Nginx:无法将 HTTP 响应转发回 HTTP 客户端

java - 如何使用 java 8 Streams 同时或并行运行 3 个方法

Java 8 并行排序与 Scala 并行排序