amazon-web-services - IntelliJ:设置Gradle运行配置的AWS环境变量

标签 amazon-web-services gradle amazon-s3 intellij-idea amazon-iam

我正在尝试为配置了gradle的Java Spring Boot设置本地开发环境,这需要一个有效的AWS CLI环境。重要的是要注意,存在不同的配置文件(.aws/config)(未配置default),它们是MFA(role_arn)的不同角色切换(mfa_serial)。

描述
目前,我正在使用Windows 10 Build 18363(“19.09”),IntelliJ Ultimate 20.02,gradle 5.6.4。
为了运行此应用程序,我使用gradle clean bootRun任务和以下env vars设置了运行配置:

  • AWS_SECRET_ACCESS_KEY
  • AWS_ACCESS_KEY_ID
  • AWS_REGION
  • AWS_DEFAULT_REGION
  • AWS_PROFILE
  • AWS_DEFAULT_PROFILE
  • #.aws/config
    [profile prod]
    region=eu-central-1
    output=json
    role_arn=<role_arn_prod>
    source_profile=site-iam
    mfa_serial=<arn_iam_user_mfa_1>
    
    
    [profile sit]
    region=eu-central-1
    output=json
    role_arn=<role_arn_sit>
    source_profile=site-iam
    mfa_serial=<arn_iam_user_mfa_1>
    
    [profile dev]
    region=eu-central-1
    output=json
    role_arn=<role_arn_dev>
    source_profile=site-iam
    mfa_serial=<arn_iam_user_mfa_1>
    
    [profile site-iam]
    region=eu-central-1
    output=json
    
    [default]
    
    
    #.aws/credentials
    [default]
    aws_access_key_id = <access_key_id_1>
    aws_secret_access_key = <secret_access_key_1>
    
    [site-iam]
    aws_access_key_id = <access_key_id_1>
    aws_secret_access_key = <secret_access_key_1>
    

    行为
    这个Java应用程序内部的请求非常简单。
    GetObjectRequest getObjectRequest = GetObjectRequest.builder()
        .bucket(this.bucket)
        .key(this.key)
        .build();
    String response = this
        .s3Client
        .getObjectAsBytes(getObjectRequest).asString(Charset.forName(StandardCharsets.UTF_8.name()));
    
    以下错误表明没有角色切换和/或身份验证失败(因为我们需要提交TOTP才能进行身份验证。
    Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Access Denied (Service: S3, Status Code: 403, Request ID: <REQUEST_ID>)
        at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:123) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:79) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:59) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:40) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:64) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:34) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:189) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:121) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:147) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:52) ~[sdk-core-2.13.13.jar:na]
        at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:62) ~[aws-core-2.13.13.jar:na]
        at software.amazon.awssdk.services.s3.DefaultS3Client.getObject(DefaultS3Client.java:3606) ~[s3-2.13.13.jar:na]
        at software.amazon.awssdk.services.s3.S3Client.getObjectAsBytes(S3Client.java:7563) ~[s3-2.13.13.jar:na]
    
    通过cmd可重现类似的行为
    aws s3 ls
    
    An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied
    

    不同的方法
    STS获取 session token |环境变量
    我已经通过获取了 session token
    aws sts get-session-token --serial <arn_iam_user_mfa_1> --token-code <TOTP> --duration-seconds 129600
    
    作为json
    {
        "Credentials": {
            "AccessKeyId": "<access_key_id_2>",
            "SecretAccessKey": "<secret_access_key_2>",
            "SessionToken": "<session_token_2>",
            "Expiration": "2020-08-02T22:57:12+00:00"
        }
    }
    
    并将更新以下环境变量设置为获得的值:
  • AWS_ACCESS_KEY_ID:<access_key_id_2>
  • AWS_SECRET_ACCESS_KEY:<secret_access_key_2>
  • AWS_SESSION_TOKEN:<session_token_2>
  • AWS_SECURITY_TOKEN:<session_token_2>

  • 但这返回相同的错误
    Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Access Denied (Service: S3, Status Code: 403, Request ID: <request_id>)
    

    快速肮脏的.aws/config修复
    经过几次调试,default配置文件以以下方式更改
    [default]
    region=eu-central-1
    output=json
    role_arn=<role_arn_prod>
    source_profile=site-iam
    mfa_serial=<arn_iam_user_mfa_1>
    
    导致
    Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(): To use assumed roles in the '<ROLE_PROFILE>' profile, the 'sts' service module must be on the class path., ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Unable to load credentials from service endpoint.]
    

    设置AWS_WEB_IDENTITY_TOKEN_FILE通过WSL sytle路径(AWS_WEB_IDENTITY_TOKEN_FILE)将~/.aws/cli/cache env var设置为位于\wsl$\...中的现有 token 文件。
    即使存在和/或更新了env var,也会导致与上述相同的错误
    Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(): To use assumed roles in the '<ROLE_PROFILE>' profile, the 'sts' service module must be on the class path., ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Unable to load credentials from service endpoint.]
    

    调试
    没有权限问题。
    此特定文件的下载使用正确的角色arn进行工作。
    
    C:\Users\tunnelblick>aws s3 ls
    
    An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied
    
    C:\Users\tunnelblick>aws s3 ls --profile prod
    Enter MFA code for <arn_iam_user_mfa_1>:
    2020-02-05 09:28:41 <bucket_1>
    2020-06-23 05:16:07 <bucket_2>
    
    C:\Users\tunnelblick>aws s3 cp s3://<bucket_1>/<file>.json .
    fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
    
    C:\Users\tunnelblick>aws s3 cp s3://<bucket_1>/<file>.json . --profile prod
    download: s3://<bucket_1>/<file>.json to .\<file>.json
    
    在我的WSL 1终端中设置这些环境变量并执行aws s3 ls会导致相同的Access Denied错误
    (An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied)

    最佳答案

    稍后,我还尝试在~/.aws/credentials文件中设置这些值。似乎我缺少一些可以通过aws-mfa(AWS STS的python包装器)解决的选项。
    我正在使用以下example

    aws-mfa --duration 1800 --device arn:aws:iam::123456788990:mfa/dudeman --assume-role arn:aws:iam::123456788990:role/some-role --role-session-name some-role-session`
    
    这会为default配置文件填充其他选项,以进行适当的角色切换,例如assumed_roleassume_role_arnaws_session_tokenaws_security_tokenexpiration,并且在我的运行配置中像一个 super 按钮一样工作。

    关于amazon-web-services - IntelliJ:设置Gradle运行配置的AWS环境变量,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/63204732/

    相关文章:

    amazon-web-services - AWS 云信息 : How do I check if a bucket exists from within the Cloudformation template?

    asp.net-mvc-3 - AWS : EC2 micro, 不足以用于 .NET MVC 3 应用程序?

    amazon-web-services - AWS 中针对 SPA 和 API 网关的跨堆栈引用

    android - FaSTLane beta lane 无法在 android Flutter 中创建应用构建

    Gradle - 从 .properties 文件加载属性

    gradle - 子项目的插件符号

    typescript - 使用 CDK 将文件上传到 S3 存储桶

    amazon-web-services - 如何在 CDK 中获取 lambda 函数的 ARN

    amazon-s3 - 以编程方式与 Amazon S3 存储桶交互的最佳方式 AWS SDK for .net 或 tntDrive?

    amazon-web-services - 阻止下载的 Amazon S3 存储桶策略?