linux shell - 检查今天的文件，如果今天没有就上传到hdfs 如果不是今天就不要上传。给出文件已存在的消息

Question

我想检查一个目录是否在今天创建。如果是，则将其上传到 HDFS，但如果目录的修改日期不同，则输出该目录已复制到 HDFS。

#!/bin/sh
export DATA_PATH=/data/1/sanket
#We will enter the directory where we want to check other directories
cd $DATA_PATH

#Details of directories will be transfered into no_of_files.txt
ls -lh $DATA_PATH/ >> temp/no_of_files.txt

#We will extract name of the file from above file.
nameoffile=$(awk '{print $9}' temp/no_of_files.txt)

#Now we want today's date.
echo $(date) >> temp/date.txt

#So the modifying date and todays date will be copied to a variable.
filedate=$(awk '{print $6 $7}' temp/no_of_files.txt)
todaydate=$(awk '{print $2 $3}' temp/date.txt)

export "nameoffile"
export "filedate"
export "todaydate"

rm -fr $DATA_PATH/temp/no_of_files.txt
rm -fr $DATA_PATH/temp/name_of_files.txt
rm -fr $DATA_PATH/temp/date.txt

#Directory on HDFS where we want to copy data    
path=sanket_data

#First to check that modifying date of directory and today's date to match and if so
#then copy the data on HDFS, if they dont match then give error as file already copied.

if [[ "$filedate" == "$todaydate" ]]; then
for filename in $nameoffile; do
    #path=sanket_data
    #nameoffile=$(awk '{print $9}' temp/no_of_files.txt)
    #for filename in $nameoffile
    /usr/bin/hadoop fs -put $DATA_PATH/$filename /user/sanket/$path
    #echo $filename already copied!
    done
elif [[ "$filedate" != "$todaydate" ]]; then

    #/usr/bin/hadoop fs -put $DATA_PATH/$filename /user/sanket/$path
    echo $filename already copied!
    #hdfs dfs -put $filename /user/sanket/$path
fi

Answer 1

你需要的是touch、date和newer。让我们首先使用 touch 和 date 创建一个临时空文件，最后修改日期 (LMD) 为今天 00:00:

touch -d $( date +%F ) today0000

现在让我们使用 newer 来测试 $DATA_PATH 是否有比 today0000 更新的 LMD:

if $( newer $DATA_PATH today0000 ); then
  /usr/bin/hadoop fs -put $DATA_PATH/* /user/sanket/$path
else
  echo "$DATA_PATH older than today ($( date +%F ))"
fi