Bash - 基于以毫秒为单位的时间戳搜索

Question

我正在搜索一个时间间隔内所有已完成(数百个)的 Hadoop 作业。此时间间隔以毫秒为单位。

格式如下:

JobId      State           StartTime      UserName           Queue      Priority       UsedContainers  RsvdContainers      UsedMem         RsvdMem        NeededMem                 AM info

job_xxxxxxx  SUCCEEDED       1458844667431    default    default          NORMAL                  N/A             N/A          N/A           N/A        N/A             http://xxxxxxxx:8088/proxy/application_xxxxxxxxxx/jobhistory/job/job_xxxxxxxx
job_xxxxxxx  SUCCEEDED       1459449718363    default    default         NORMAL                  N/A             N/A        N/A             N/A              N/A        http://xxxxxx.xxxxx.com:8088/proxy/application_xxxxxxxxx/jobhistory/job/job_xxxxx

以下是我的格式:

STARTTIME="Tue Apr 12 10:24:29 EDT 2016"
ENDTIME="Tue Apr 12 15:24:29 EDT 2016"

CONVERTTIME_1=`date --date="$STARTTIME" +%s%3N`
CONVERTTIME_2=`date --date="$ENDTIME" +%s%3N`

echo $CONVERTTIME_1, $CONVERTTIME_2

mapred job -list all | sed -n '/$CONVERTTIME_1/,/$CONVERTTIME_2/p' > out

输出:timerange 内所有类似上述的作业。

请问谁能帮忙弄到这些？

映射作业的输出 - 列出所有

映射作业-列出所有

 job_1457613852865_5163  SUCCEEDED       1459199337140    zzzzzzzzzz    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5163/jobhistory/job/job_1457613852865_5163

 job_1457613852865_4633  SUCCEEDED       1458992402216       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4633/jobhistory/job/job_1457613852865_4633

 job_1457613852865_4821  SUCCEEDED       1459078845580       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4821/jobhistory/job/job_1457613852865_4821

 job_1457613852865_0322  SUCCEEDED       1457717313217        zzzddd    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_0322/jobhistory/job/job_1457613852865_0322

 job_1457613852865_5304  SUCCEEDED       1459254375921       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_5304/jobhistory/job/job_1457613852865_5304

 job_1457613852865_8744  SUCCEEDED       1460195126188       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8744/jobhistory/job/job_1457613852865_8744

 job_1457613852865_3384  SUCCEEDED       1458649020794       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_3384/jobhistory/job/job_1457613852865_3384

 job_1457613852865_9038  SUCCEEDED       1460291694279       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_9038/jobhistory/job/job_1457613852865_9038

 job_1457613852865_8487  SUCCEEDED       1460115319590       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8487/jobhistory/job/job_1457613852865_8487

 job_1457613852865_8321  SUCCEEDED       1460038991587        dddyyy    uuuuuuuu_critical          NORMAL                  N/A             N/A      N/A           N/A        N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_8321/jobhistory/job/job_1457613852865_8321

 job_1457613852865_4661  SUCCEEDED       1458994901619       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_4661/jobhistory/job/job_1457613852865_4661

 job_1457613852865_1975  SUCCEEDED       1458216683800       zzzyyyy    yyyyyy_default       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://xxxxx254.yyyyyy.com:8088/proxy/application_1457613852865_1975/jobhistory/job/job_1457613852865_1975

我用过这个:

#!/bin/bash


STARTTIME="Tue Apr 12 10:13:01 EDT 2016"
ENDTIME="Tue Apr 12 10:13:59 EDT 2016"

start=$(date -d "$STARTTIME" '+%s%3N')
end=$(date -d "$ENDTIME" '+%s%3N')

echo "start=$start :: end=$end"

mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'

多了一份工作:

job_1457613852865_9785  SUCCEEDED       1460470436726       yyyyyyyyyy    nnnnnnnnnn       NORMAL                  N/A             N/A      N/A             N/A              N/A      http://888888.xxxxxxxxxx.com:8088/proxy/application_1457613852865_9785/jobhistory/job/job_1457613852865_9785

Answer 1

您可以将日期转换为毫秒值，然后使用 awk 过滤数据:

STARTTIME="Tue Apr 12 10:24:29 EDT 2016"
ENDTIME="Tue Apr 12 15:24:29 EDT 2016"

start=$(date -d "$STARTTIME" '+%s%3N')
end=$(date -d "$ENDTIME" '+%s%3N')"

echo "start=$start :: end=$end"
mapred job -list all | awk -v start="$start" -v end="$end" '$3>=start && $3<=end'