linux - 即使某些进程完成，也要不断填充并行进程的四个槽

Question

我有一个脚本，一次运行 4 个进程的批处理，我不关心获取每个进程的返回代码。我不想同时运行超过 4 个进程。以下方法的问题是它一次最多不能填充 4 个进程。例如，如果 proc2 和 proc3 提前完成，我希望 proc 5 和 6 启动，而不是仅在 1-4 完成后启动。我怎样才能在 bash 中实现这一点？

run_func_1 &
run_func_2 &
run_func_3 &
run_func_4 &
wait
run_func_5 &
run_func_6 &
run_func_7 &
run_func_8 &
wait

Answer 1

我尝试使用工作人员池和作业队列进行自定义实现。 一旦前一个工作完成，新工作人员就会从队列中获取工作。

您可以根据需要调整此脚本，但我希望您能明白我的意图。

这是脚本:

#!/bin/bash

f1() { echo Started f1; sleep 10; echo Finished f1; }
f2() { echo Started f2; sleep 8; echo Finished f2; }
f3() { echo Started f3; sleep 12; echo Finished f3; }
f4() { echo Started f4; sleep 14; echo Finished f4; }
f5() { echo Started f5; sleep 7; echo Finished f5; }

declare -r MAX_WORKERS=2
declare -a worker_pids
declare -a jobs=('f1' 'f2' 'f3' 'f4' 'f5')

available_worker_index() {
    # If number of workers is less than MAX_WORKERS
    # We still have workers that are idle
    declare worker_count="${#worker_pids[@]}"
    if [[ $worker_count -lt $MAX_WORKERS ]]; then
        echo "$worker_count"
        return 0
    fi

    # If we reached this code it means
    # All workers are already created and executing a job
    # We should check which of them finished and return it's index as available
    declare -i index=0
    for pid in "${worker_pids[@]}"; do
        is_running=$(ps -p "$pid" > /dev/null; echo "$?")
        if [[ $is_running != 0 ]]; then
            echo "$index"
            return 0
        fi
        index+=1
    done

    echo "None"
}

for job in "${jobs[@]}"; do
    declare worker_index
    worker_index=$(available_worker_index)
    while [[ $worker_index == "None" ]]; do
        # Wait for available worker
        sleep 3
        worker_index=$(available_worker_index)
    done

    # Run the job in background
    "$job" &

    # Save it's pid for later
    pid="$!"
    worker_pids["$worker_index"]="$pid"
done

# Wait all workers to finish
wait

您只需更改 MAX_WORKERS 变量即可轻松更改工作池的大小。