我正在将Mrjob-Hadoop与Python2.7,Ubuntu 14.04结合使用,并且具有以下屏幕输出:
no configs found; falling back on auto-configuration
no configs found; falling back on auto-configuration
creating tmp directory /tmp/word-document.hduser.20160122.065849.953886
writing wrapper script to /tmp/word-document.hduser.20160122.065849.953886/setup-wrapper.sh
PLEASE NOTE: Starting in mrjob v0.5.0, protocols will be strict by default. It's recommended you run your job with --strict-protocols or set up mrjob.conf as described at https://pythonhosted.org/mrjob/whats-new.html#ready-for-strict-protocols
writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00000
> sh -ex setup-wrapper.sh /usr/bin/python word-document.py --step-num=0 --mapper /tmp/word-document.hduser.20160122.065849.953886/input_part-00000 > /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00000
writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00001
> sh -ex setup-wrapper.sh /usr/bin/python word-document.py --step-num=0 --mapper /tmp/word-document.hduser.20160122.065849.953886/input_part-00001 > /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00001
STDERR: + __mrjob_PWD=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/0
STDERR: + exec
STDERR: + /usr/bin/python -c import fcntl; fcntl.flock(9, fcntl.LOCK_EX)
STDERR: + export PYTHONPATH=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/0/mrjob.tar.gz:/home/ignacio/shogun-install/lib/python2.7/dist-packages:/home/ignacio/shogun/examples/undocumented/python_modular:
STDERR: + exec
STDERR: + cd /tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/0
STDERR: + /usr/bin/python word-document.py --step-num=0 --mapper /tmp/word-document.hduser.20160122.065849.953886/input_part-00000
STDERR: + __mrjob_PWD=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/1
STDERR: + exec
STDERR: + /usr/bin/python -c import fcntl; fcntl.flock(9, fcntl.LOCK_EX)
STDERR: + export PYTHONPATH=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/1/mrjob.tar.gz:/home/ignacio/shogun-install/lib/python2.7/dist-packages:/home/ignacio/shogun/examples/undocumented/python_modular:
STDERR: + exec
STDERR: + cd /tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/mapper/1
STDERR: + /usr/bin/python word-document.py --step-num=0 --mapper /tmp/word-document.hduser.20160122.065849.953886/input_part-00001
Counters from step 1:
(no counters found)
writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper-sorted
> sort /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00000 /tmp/word-document.hduser.20160122.065849.953886/step-0-mapper_part-00001
writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00000
> sh -ex setup-wrapper.sh /usr/bin/python word-document.py --step-num=0 --reducer /tmp/word-document.hduser.20160122.065849.953886/input_part-00000 > /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00000
writing to /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00001
> sh -ex setup-wrapper.sh /usr/bin/python word-document.py --step-num=0 --reducer /tmp/word-document.hduser.20160122.065849.953886/input_part-00001 > /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00001
STDERR: + __mrjob_PWD=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/0
STDERR: + exec
STDERR: + /usr/bin/python -c import fcntl; fcntl.flock(9, fcntl.LOCK_EX)
STDERR: + export PYTHONPATH=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/0/mrjob.tar.gz:/home/ignacio/shogun-install/lib/python2.7/dist-packages:/home/ignacio/shogun/examples/undocumented/python_modular:
STDERR: + exec
STDERR: + cd /tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/0
STDERR: + /usr/bin/python word-document.py --step-num=0 --reducer /tmp/word-document.hduser.20160122.065849.953886/input_part-00000
STDERR: + __mrjob_PWD=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/1
STDERR: + exec
STDERR: + /usr/bin/python -c import fcntl; fcntl.flock(9, fcntl.LOCK_EX)
STDERR: + export PYTHONPATH=/tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/1/mrjob.tar.gz:/home/ignacio/shogun-install/lib/python2.7/dist-packages:/home/ignacio/shogun/examples/undocumented/python_modular:
STDERR: + exec
STDERR: + cd /tmp/word-document.hduser.20160122.065849.953886/job_local_dir/0/reducer/1
STDERR: + /usr/bin/python word-document.py --step-num=0 --reducer /tmp/word-document.hduser.20160122.065849.953886/input_part-00001
Counters from step 1:
(no counters found)
Moving /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00000 -> /tmp/word-document.hduser.20160122.065849.953886/output/part-00000
Moving /tmp/word-document.hduser.20160122.065849.953886/step-0-reducer_part-00001 -> /tmp/word-document.hduser.20160122.065849.953886/output/part-00001
Streaming final output from /tmp/word-document.hduser.20160122.065849.953886/output
removing tmp directory /tmp/word-document.hduser.20160122.065849.953886
你能说出什么问题吗?我的意思是,工作已经完成,但是
STDERR:
关键字对我来说很吵。先感谢您。
最佳答案
看来您的工作没有产生任何输出。您能张贴word-document.py和您的输入数据吗?
关于python-2.7 - Hadoop的STDERR输出,这是否意味着一些问题?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/34940976/