python - AWS Lambda(Python): Pass list of file paths and execute each file as a separate lambda

Question

我有一个 python lambda 函数，它将列出 S3 存储桶中的每个文件(代码如下)。我不清楚如何将每个文件对象传递给另一个 lambda 函数作为输入并单独执行。目标是让列表中有 x 个文件来创建 x 个要并发执行的第二个 lambda(即，如果列表中有 20 个文件，则执行第二个 lambda 执行 20 次，每个文件分别传递给 lambda 函数)。该文件将在第二个 lambda 函数中用于 Pandas 中的联接。

非常感谢任何帮助!

文件列表(lambda 1)

import boto3

#Start session with Profile
session = 
boto3.session.Session(profile_name='<security_token_service_profile>')

client = session.client('s3') #low-level functional API
resource = session.resource('s3') #high-level object-oriented API

#State S3 bucket
my_bucket = resource.Bucket('<bucket>') #subsitute this for your s3 bucket name. 

#List all files
files = list(my_bucket.objects.filter(Prefix='<path_to_file>'))

print(files)

Answer 1

谢谢@jarmod!那行得通。对于那些将来可能需要这个的人，我上面的 lambda 脚本已修改如下:

import boto3
import json

print('[INFO]  Loading Function')

def lambda_handler(event, context):

    print("[INFO]  Received event: " + json.dumps(event, indent=2))

    #Start session with region details for authentication
    session = boto3.session.Session(region_name='<region>')
    client = session.client('s3') #low-level functional API
    resource = session.resource('s3') #high-level object-oriented API

    #Identify S3 bucket
    my_bucket = resource.Bucket('<bucket>') #subsitute this for your s3 bucket name. 

    #List all files
    files = list(my_bucket.objects.filter(Prefix='<file_path>'))

    for file in files:
        payload = json.dumps({"key": file.key})
        print(payload)
        client_lambda = session.client('lambda')
        client_lambda.invoke(
            FunctionName='<lambda_function_name_to_call>',
            InvocationType='Event',
            LogType='None',
            Payload=payload
        )


if __name__ == '__main__':
    lambda_handler()`