mysql - 导入到 hdfs 时出现 Sqoop 错误

Question

我正在尝试使用 sqoop 将一个小表从 mysql 导出到 HDFS。该表有 2 列 id (主键)和名称。我能够通过 sqoop 列出数据库和表。但是在将表导入 HDFS 时出现异常。请帮忙。下面是错误日志。

13/12/04 02:05:38 WARN conf.Configuration: session.id is deprecated. 
Instead, use   dfs.metrics.session-id
13/12/04 02:05:38 INFO jvm.JvmMetrics: 
Initializing JVM Metrics  withprocessName=JobTracker,sessionId=
13/12/04 02:05:39 INFO mapreduce.JobSubmitter: 
Cleaning up the staging area file:/tmp/hadoop-hadoop/mapred/staging/hadoop1439217057
/.staging/job_local1439217057_0001
13/12/04 02:05:39 ERROR 
security.UserGroupInformation:PriviledgedActionException  as:hadoop      
(auth:SIMPLE) 
cause:java.io.FileNotFoundException: 
File does not exist: hdfs://prat1:9000/home/hadoop/usr/sqoop-1.4.3-cdh4.3.0/lib/commons-    compress-1.4.1.jar
13/12/04 02:05:39 DEBUG util.ClassLoaderStack: 
Restoring classloader:sun.misc.Launcher$AppClassLoader@35a16869
13/12/04 02:05:39 ERROR tool.ImportTool: 
Encountered IOException running import job:  java.io.FileNotFoundException: File does not  exist:
hdfs://prat1:9000/home/hadoop/usr/
sqoop-1.4.3-cdh4.3.0/lib/commons-compress-1.4.1.jar
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:824)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:93)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:254)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:290)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:361)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1269)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1266)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1266)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1287)
    at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:173)
    at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:151)
    at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:226)
    at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:555)
    at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:111)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)

Answer 1

由于您没有写入权限，因此您将获得 security.UserGroupInformation:PriviledgedActionException

尝试以 hdfs 用户身份登录，然后运行 ​​sqoop 命令。

苏根

输入密码

su HDFS

然后运行sqoop导出命令。