linux - Shell 脚本文件观察器并发性

Question

我有以下脚本(在 OEL5.6 上运行的 shell 脚本)当前通过 cron 计划从给定目录(在数据库表中指定)中获取文件并在目录和文件掩码上调用处理脚本基础。该脚本目前工作正常，但如果一个文件夹有大量文件要处理，即使所有其他文件夹完成，脚本也不会退出，直到该脚本完成，这意味着文件不会登陆其他文件夹被拾起直到下一次运行。我想对此使用类似的方法，但让它不断检查文件夹中的新文件，而不是按顺序运行一次所有文件夹然后退出，这样它就可以在后台不断地作为守护进程运行。有什么想法而不是将其包装在 while true 循环中吗？我从这个例子中过滤掉了一些代码以保持简短。

readonly HOME_DIR="$(cd $(dirname $0)/;echo $PWD)"
export LOCK_DIR="/tmp/lock_folder"

check_lock() {
    # Try and create the $LOCK_DIR lock directory. Exit script if failure.
    # Do some checks to make sure the script is actually running and hasn't just failed and left a lock dir.
}

main(){
    # Check to see if there's already an instance of the watcher running.
    check_lock

    # when the watcher script exits remove the lock directory for the next run
    trap 'rm -r $LOCK_DIR;' EXIT

    # Pull folder and file details into a csv file from the database -> $FEEDS_FILE

    # Loop through all the files in given folders
    while IFS="," read feed_name feed_directory file_mask
    do

        # Count the number of files to process using the  directory and file mask
        num_files=$(find $feed_directory/$file_mask -mmin +5 -type f 2> /dev/null | wc -l 2> /dev/null)
        if [[ $num_files < 1 ]]; then
            # There's no files older than 5 mins to pickup here. Move on to next folder.
            continue 
        fi

        # Files found! Try and create a new feed_name lock dir. This should always pass first loop. 
        if mkdir $LOCK_DIR/$feed_name 2>/dev/null; then 
            $HOME_DIR/another_script.sh "$feed_name" "$feed_directory" "$file_mask" & # Call some script to do processing. This script removes it's child lock dir when done. 
        else
            log.sh "Watcher still running" f
            continue
        fi

        # If the amount of processes running as indicated by child lock dirs present in $LOCK_DIR is greater than or equal to the max allowed then wait before re-trying another.
        while [ $(find $LOCK_DIR -maxdepth 1 -type d -not -path $LOCK_DIR | wc -l) -ge 5 ]; do
            sleep 10
        done

    done < $FEEDS_FILE

    # Now all folders have been processed make sure that this script doesn't exit until all child scripts have completed (and removed their lock dirs).
    while [ $(find $LOCK_DIR -type d | wc -l) -gt 1 ]; do
        sleep 10
    done

    exit 0
}

main "$@"

Answer 1

一个想法是使用 inotify-tools 中的 inotifywait 来监视目录的变化。这比不断扫描目录以查找更改更有效。类似的东西

inotifywait -m -r -e create,modify,move,delete /dir1 /dir2 |
    while IFS= read -r event; do
        # parse $event, act accordingly
    done