hadoop - Sqoop失败，并带有密码文件参数

Question

我有一个sqoop脚本，可将数据从SAP HANA提取到Hive。当我将密码作为参数“--password Password $$”输入时，sqoop脚本运行良好，但是为了保护密码，我将其放在名为sap.password的文件中，并使用了参数“--password-file / dev / configs / sap.password“，但是sqoop脚本返回execption。

以下是我的sqoop脚本，发生了异常:

sqoop import 
--connect jdbc:sap://hostname?currentschema=SCHEMA_REF 
--driver com.sap.db.jdbc.Driver 
--username SERVICE_ACCOUNT 
--password-file /dev/configs/sap.password 
--table TABLE1
--hive-import 
--hive-overwrite 
--hive-database cdc_stg 
--hive-table HIVE_TABLE1
--as-parquetfile 
--m 1

我得到的异常是(我确定凭据正确):

9/11/14 05:47:08 ERROR manager.SqlManager: Error executing statement:
com.sap.db.jdbc.exceptions.jdbc40.SQLInvalidAuthorizationSpecException: [10]: authentication failed
 com.sap.db.jdbc.exceptions.jdbc40.SQLInvalidAuthorizationSpecException: [10]: authentication failed
 at com.sap.db.jdbc.exceptions.jdbc40.SQLInvalidAuthorizationSpecException.createException(SQLInvalidAuthorizationSpecException.java:40)
 at com.sap.db.jdbc.exceptions.SQLExceptionSapDB.createException(SQLExceptionSapDB.java:290)
 at com.sap.db.jdbc.exceptions.SQLExceptionSapDB.generateDatabaseException(SQLExceptionSapDB.java:174)
 at com.sap.db.jdbc.packet.ReplyPacket.buildExceptionChain(ReplyPacket.java:100)
 at com.sap.db.jdbc.ConnectionSapDB.execute(ConnectionSapDB.java:1141)
 at com.sap.db.jdbc.ConnectionSapDB.execute(ConnectionSapDB.java:888)
 at com.sap.db.util.security.AbstractAuthenticationManager.connect(AbstractAuthenticationManager.java:43)
 at com.sap.db.jdbc.ConnectionSapDB.openSession(ConnectionSapDB.java:586)
 at com.sap.db.jdbc.ConnectionSapDB.doConnect(ConnectionSapDB.java:436)
 at com.sap.db.jdbc.ConnectionSapDB.<init>(ConnectionSapDB.java:195)
 at com.sap.db.jdbc.ConnectionSapDBFinalize.<init>(ConnectionSapDBFinalize.java:13)
 at com.sap.db.jdbc.Driver.connect(Driver.java:255)
 at java.sql.DriverManager.getConnection(DriverManager.java:664)
 at java.sql.DriverManager.getConnection(DriverManager.java:247)
 at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:903)
 at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:59)
 at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:762)
 at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:785)
 at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:288)
 at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:259)
 at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:245)
 at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:333)
 at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1879)
 at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1672)
 at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
 at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:515)
 at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:633)
 at org.apache.sqoop.Sqoop.run(Sqoop.java:146)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
 at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:182)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:233)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:242)
 at org.apache.sqoop.Sqoop.main(Sqoop.java:251)
19/11/14 05:47:08 ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter
 at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1678)
 at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
 at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:515)
 at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:633)
 at org.apache.sqoop.Sqoop.run(Sqoop.java:146)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
 at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:182)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:233)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:242)
 at org.apache.sqoop.Sqoop.main(Sqoop.java:251)

Answer 1

我怀疑密码文件可能是用换行符创建的，因为--password可以正常工作，唯一的区别或更改是转换为使用密码文件。

您能否使用下面说明的sqoop docs警告子句重新创建密码文件。

引用:SqoopUserGuide

Sqoop将读取密码文件的全部内容，并将其用作密码。这将包括大多数文本编辑器默认添加的任何尾随空格字符，例如换行符。您需要确保密码文件仅包含属于密码的字符。在命令行上，可以将命令echo与switch -n一起使用，以存储密码，且不包含任何结尾的空格字符。

例如:要存储密码 secret ，请在下面使用。

echo -n "secret" > password.file

另外，使用代替sqoop导入，请尝试使用列表数据库或列表表或eval 来测试与密码文件的连接。