python-2.7 - Hadoop的STDERR输出,这是否意味着一些问题?

标签 python-2.7 hadoop mrjob

我正在将Mrjob-Hadoop与Python2.7,Ubuntu 14.04结合使用,并且具有以下屏幕输出:

no configs found; falling back on auto-configuration
no configs found; falling back on auto-configuration
creating tmp directory /tmp/word-document.hduser.20160122.065849.953886
writing wrapper script to /tmp/word-document.hduser.20160122.065849.953886/setup-wrapper.sh

PLEASE NOTE: Starting in mrjob v0.5.0, protocols will be strict by default. It's recommended you run your job with --strict-protocols or set up mrjob.conf as described at https://pythonhosted.org/mrjob/whats-new.html#ready-for-strict-protocols

writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00000
> sh -ex setup-wrapper.sh /usr/bin/python word-document.py --step-num=0 --mapper /tmp/word-document.hduser.20160122.065849.953886/input_part-00000 > /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00000
writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00001
> sh -ex setup-wrapper.sh /usr/bin/python word-document.py --step-num=0 --mapper /tmp/word-document.hduser.20160122.065849.953886/input_part-00001 > /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00001
STDERR: + __mrjob_PWD=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/0
STDERR: + exec
STDERR: + /usr/bin/python -c import fcntl; fcntl.flock(9, fcntl.LOCK_EX)
STDERR: + export PYTHONPATH=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/0/mrjob.tar.gz:/home/ignacio/shogun-install/lib/python2.7/dist-packages:/home/ignacio/shogun/examples/undocumented/python_modular:
STDERR: + exec
STDERR: + cd /tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/0
STDERR: + /usr/bin/python word-document.py --step-num=0 --mapper /tmp/word-document.hduser.20160122.065849.953886/input_part-00000
STDERR: + __mrjob_PWD=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/1
STDERR: + exec
STDERR: + /usr/bin/python -c import fcntl; fcntl.flock(9, fcntl.LOCK_EX)
STDERR: + export PYTHONPATH=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/1/mrjob.tar.gz:/home/ignacio/shogun-install/lib/python2.7/dist-packages:/home/ignacio/shogun/examples/undocumented/python_modular:
STDERR: + exec
STDERR: + cd /tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/1
STDERR: + /usr/bin/python word-document.py --step-num=0 --mapper /tmp/word-document.hduser.20160122.065849.953886/input_part-00001
Counters from step 1:
  (no counters found)
writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper-sorted
> sort /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00000 /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00001
writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00000
> sh -ex setup-wrapper.sh /usr/bin/python word-document.py --step-num=0 --reducer /tmp/word-document.hduser.20160122.065849.953886/input_part-00000 > /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00000
writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00001
> sh -ex setup-wrapper.sh /usr/bin/python word-document.py --step-num=0 --reducer /tmp/word-document.hduser.20160122.065849.953886/input_part-00001 > /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00001
STDERR: + __mrjob_PWD=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/0
STDERR: + exec
STDERR: + /usr/bin/python -c import fcntl; fcntl.flock(9, fcntl.LOCK_EX)
STDERR: + export PYTHONPATH=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/0/mrjob.tar.gz:/home/ignacio/shogun-install/lib/python2.7/dist-packages:/home/ignacio/shogun/examples/undocumented/python_modular:
STDERR: + exec
STDERR: + cd /tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/0
STDERR: + /usr/bin/python word-document.py --step-num=0 --reducer /tmp/word-document.hduser.20160122.065849.953886/input_part-00000
STDERR: + __mrjob_PWD=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/1
STDERR: + exec
STDERR: + /usr/bin/python -c import fcntl; fcntl.flock(9, fcntl.LOCK_EX)
STDERR: + export PYTHONPATH=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/1/mrjob.tar.gz:/home/ignacio/shogun-install/lib/python2.7/dist-packages:/home/ignacio/shogun/examples/undocumented/python_modular:
STDERR: + exec
STDERR: + cd /tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/1
STDERR: + /usr/bin/python word-document.py --step-num=0 --reducer /tmp/word-document.hduser.20160122.065849.953886/input_part-00001
Counters from step 1:
  (no counters found)
Moving /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00000 -> /tmp/word-document.hduser.20160122.065849.953886/output/part-00000
Moving /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00001 -> /tmp/word-document.hduser.20160122.065849.953886/output/part-00001
Streaming final output from /tmp/word-document.hduser.20160122.065849.953886/output
removing tmp directory /tmp/word-document.hduser.20160122.065849.953886

你能说出什么问题吗?我的意思是,工作已经完成,但是STDERR:关键字对我来说很吵。

先感谢您。

最佳答案

看来您的工作没有产生任何输出。您能张贴word-document.py和您的输入数据吗?

关于python-2.7 - Hadoop的STDERR输出,这是否意味着一些问题?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/34940976/

相关文章:

python - 带有可选参数的 ArgumentParser(在互斥组中)

hadoop - 如何计算 Hive 表中小于特定行的行数?

xml - 为什么hadoop不提供XmlInputFormat?

hadoop - Namenode 守护进程没有正常启动

amazon-web-services - 运行 EMR 作业的最低 AWS 政策要求

python - "Counters from Step 1: No Counters found"使用 Hadoop 和 mrjob

python - Python 的 re.compile 需要与 re.search 不同的语法吗?

python - 从 OrderedDict 获取键计数,其中键是一个元组

python - Python 中作为函数指针的类字段

snappy 文件上的 hadoop python 作业产生 0 大小的输出