python - aws Sagemaker 的 AnnotationConsolidation lambda 事件的空字典

Question

我开始使用 aws sagemaker 开发我的机器学习模型，我正在尝试构建一个 lambda 函数来处理 sagemaker 标记作业的响应。我已经创建了自己的 lambda 函数，但是当我尝试读取事件内容时，我发现事件字典完全是空的，所以我没有读取任何数据。

我已经为 lambda 函数的角色赋予了足够的权限。包括: - AmazonS3FullAccess。 - AmazonSagemakerFullAccess。 - AWSLambdaBasicExecutionRole

我已经尝试将此代码用于注释后 Lambda(适用于 python 3.6):

https://docs.aws.amazon.com/sagemaker/latest/dg/sms-custom-templates-step2-demo1.html#sms-custom-templates-step2-demo1-post-annotation

以及这个 git 存储库中的这个:

https://github.com/aws-samples/aws-sagemaker-ground-truth-recipe/blob/master/aws_sagemaker_ground_truth_sample_lambda/annotation_consolidation_lambda.py

但它们似乎都不起作用。

为了创建标签作业，我使用了 boto3 的 sagemaker 函数: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_labeling_job

这是我创建标签作业的代码:

def create_labeling_job(client,bucket_name ,labeling_job_name, manifest_uri, output_path):

    print("Creating labeling job with name: %s"%(labeling_job_name))

    response = client.create_labeling_job(
        LabelingJobName=labeling_job_name,
        LabelAttributeName='annotations',
        InputConfig={
            'DataSource': {
                'S3DataSource': {
                    'ManifestS3Uri': manifest_uri
                }
            },
            'DataAttributes': {
                'ContentClassifiers': [
                    'FreeOfAdultContent',
                ]
            }
        },
        OutputConfig={
            'S3OutputPath': output_path
        },
        RoleArn='arn:aws:myrolearn',
        LabelCategoryConfigS3Uri='s3://'+bucket_name+'/config.json',
        StoppingConditions={
            'MaxPercentageOfInputDatasetLabeled': 100,
        },
        LabelingJobAlgorithmsConfig={
            'LabelingJobAlgorithmSpecificationArn': 'arn:image-classification'
        },
        HumanTaskConfig={
            'WorkteamArn': 'arn:my-private-workforce-arn',
            'UiConfig': {
                'UiTemplateS3Uri':'s3://'+bucket_name+'/templatefile'
            },
            'PreHumanTaskLambdaArn': 'arn:aws:lambda:us-east-1:432418664414:function:PRE-BoundingBox',
            'TaskTitle': 'Title',
            'TaskDescription': 'Description',
            'NumberOfHumanWorkersPerDataObject': 1,
            'TaskTimeLimitInSeconds': 600,
            'AnnotationConsolidationConfig': {
                'AnnotationConsolidationLambdaArn': 'arn:aws:my-custom-post-annotation-lambda'
            }
        }
    )

    return response

这是我的 lambda 函数:

    print("Received event: " + json.dumps(event, indent=2))
    print("event: %s"%(event))
    print("context: %s"%(context))
    print("event headers: %s"%(event["headers"]))

    parsed_url = urlparse(event['payload']['s3Uri']);
    print("parsed_url: ",parsed_url)

    labeling_job_arn = event["labelingJobArn"]
    label_attribute_name = event["labelAttributeName"]

    label_categories = None
    if "label_categories" in event:
        label_categories = event["labelCategories"]
        print(" Label Categories are : " + label_categories)

    payload = event["payload"]
    role_arn = event["roleArn"]

    output_config = None # Output s3 location. You can choose to write your annotation to this location
    if "outputConfig" in event:
        output_config = event["outputConfig"]

    # If you specified a KMS key in your labeling job, you can use the key to write
    # consolidated_output to s3 location specified in outputConfig.
    kms_key_id = None
    if "kmsKeyId" in event:
        kms_key_id = event["kmsKeyId"]

    # Create s3 client object
    s3_client = S3Client(role_arn, kms_key_id)

    # Perform consolidation
    return do_consolidation(labeling_job_arn, payload, label_attribute_name, s3_client)

我试过使用以下方法调试事件对象:

    print("Received event: " + json.dumps(event, indent=2))

但它只是打印一个空字典:Received event: {}

我希望输出是这样的:

    #Content of an example event:
    {
        "version": "2018-10-16",
        "labelingJobArn": <labelingJobArn>,
        "labelCategories": [<string>],  # If you created labeling job using aws console, labelCategories will be null
        "labelAttributeName": <string>,
        "roleArn" : "string",
        "payload": {
            "s3Uri": <string>
        }
        "outputConfig":"s3://<consolidated_output configured for labeling job>"
    }

最后，当我尝试使用以下方法获取标记作业 ARN 时:

    labeling_job_arn = event["labelingJobArn"]

我只是得到一个 KeyError(这是有道理的，因为字典是空的)。

Answer 1

我正在做同样的事情，但在标记对象部分我得到失败的结果，在我的输出对象中我从 Post Lambda 函数得到以下错误:

"annotation-case0-test3-metadata": {
        "retry-count": 1,
        "failure-reason": "ClientError: The JSON output from the AnnotationConsolidationLambda function could not be read. Check the output of the Lambda function and try your request again.",
        "human-annotated": "true"
    }
}