hadoop - 找不到带有 key dfs.encryption.key.provider.uri 的 uri 以在 CDH 5.4 的 HDFS 加密中创建 key 提供程序

Question

CDH版本:CDH5.4.5

问题:当使用 Hadoop CDH 5.4 中可用的 KMS 启用 HDFS 加密时，将文件放入加密区域时出错。

步骤:

Hadoop的加密步骤如下:

1. 创建 key [成功]

   [tester@master ~]$ hadoop key create 'TDEHDP' 
   -provider kms://https@10.1.118.1/key_generator/kms -size 128
   tde group has been successfully created with options 
   Options{cipher='AES/CTR/NoPadding', bitLength=128, description='null', attributes=null}.
   KMSClientProvider[https://10.1.118.1/key_generator/kms/v1/] has been updated.
   

2.创建目录[成功]

[tester@master ~]$ hdfs dfs -mkdir /user/tester/vs_key_testdir

1. 添加加密区 [成功]

   [tester@master ~]$ hdfs crypto -createZone -keyName 'TDEHDP' 
   -path /user/tester/vs_key_testdir
   Added encryption zone /user/tester/vs_key_testdir
   

2. 正在将文件复制到加密区域 [错误]

   [tdetester@master ~]$ hdfs dfs -copyFromLocal test.txt /user/tester/vs_key_testdir
   

15/09/04 06:06:33 ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !! copyFromLocal: No KeyProvider is configured, cannot access an encrypted file 15/09/04 06:06:33 ERROR hdfs.DFSClient: Failed to close inode 20823 org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/tester/vs_key_testdir/test.txt.COPYING (inode 20823): File does not exist. Holder DFSClient_NONMAPREDUCE_1061684229_1 does not have any open files.

任何想法/建议都会有所帮助。

Answer 1

此问题已在此处交叉发布:https://community.cloudera.com/t5/Storage-Random-Access-HDFS/Could-not-find-uri-with-key-dfs-encryption-key-provider-uri-to/td-p/31637

主要结论:这不是问题

这是支持人员提供的答案:

CDH's base release versions are just that: base. The fix for the harmless log print due to HDFS-7931 is present in all CDH5 releases since CDH 5.4.1.

If you see that error in context of having configured a KMS, then its a worthy one to consider. If you do not use KMS or EZs, then the error may be ignored. Alternatively upgrade to the latest CDH5 (5.4.x or 5.5.x) releases to receive a bug fix that makes the error only appear when in the context of a KMS being configured over an encrypted path.

Per your log snippet, I don't see a problem (the canary does not appear to be failing?). If you're trying to report a failure, please send us more characteristics of the failure, as HDFS-7931 is a minor issue with an unnecessary log print.