logback - 如何配置 AWS CDK ApplicationLoadBalancedFargateService 以使用 Firelens 和 Firebit 记录解析的 JSON 行

Question

当我使用 Firelens 日志驱动程序创建 ApplicationLoadBalancedFargateService，并且应用程序将 JSON 行写入日志消息时，例如将 net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder 与 Logback 结合使用时，日志消息将显示在我的日志记录存储库中(例如 Sumo Logic)，作为转义字符串，例如:

如何将日志消息保存为已解析的 JSON？

Answer 1

扫描 CDK 源代码、浏览几个相关引用文献(我将在此处提供链接以帮助引导适当的流量)并使用 cdk diff 直到唯一的更改是启用 json 解析后，我能够使其工作，如以下代码所示。这里的关键是使用 addFirelensLogRouter 方法以及其中包含的 Firelens 配置。

TaskDefinition 代码不会自动创建 LogRouter 容器(如果任务定义已包含一个容器)，这允许我们覆盖默认行为。

protected _createFargateService() {
    const logDriver = LogDrivers.firelens({
        options: {
            Name: 'http',
            Host: this._props.containerLogging.endpoint,
            URI: this._props.containerLogging.uri,
            Port: '443',
            tls: 'on',
            'tls.verify': 'off',
            Format: 'json_lines'
        }
    });
    const fargateService = new ApplicationLoadBalancedFargateService(this, this._props.serviceName, {
        cluster: this._accountEnvironmentLookups.getComputeCluster(),
        cpu: this._props.cpu, // Default is 256
        desiredCount: this._props.desiredCount, // Default is 1
        taskImageOptions: {
            image: ContainerImage.fromEcrRepository(this._props.serviceRepository, this._props.imageVersion),
            environment: this._props.environment,
            containerPort: this._props.containerPort,
            logDriver
        },
        memoryLimitMiB: this._props.memoryLimitMiB, // Default is 512
        publicLoadBalancer: this._props.publicLoadBalancer, // Default is false
        domainName: this._props.domainName,
        domainZone: !!this._props.hostedZoneDomain ? HostedZone.fromLookup(this, 'ZoneFromLookup', {
            domainName: this._props.hostedZoneDomain
        }) : undefined,
        certificate: !!this._props.certificateArn ? Certificate.fromCertificateArn(this, 'CertificateFromArn', this._props.certificateArn) : undefined,
        serviceName: `${this._props.accountShortName}-${this._props.deploymentEnvironment}-${this._props.serviceName}`,
        // The new ARN and resource ID format must be enabled to work with ECS managed tags.
        //enableECSManagedTags: true,
        //propagateTags: PropagatedTagSource.SERVICE,
        // CloudMap properties cannot be set from a stack separate from the stack where the cluster is created.
        // see https://github.com/aws/aws-cdk/issues/7825
    });
    if (this._props.logMessagesAreJsonLines) {
        // The default log driver setup doesn't enable json line parsing.
        const firelensLogRouter = fargateService.service.taskDefinition.addFirelensLogRouter('log-router', {
            // Figured out how get the default fluent bit ECR image from here https://github.com/aws/aws-cdk/blob/60c782fe173449ebf912f509de7db6df89985915/packages/%40aws-cdk/aws-ecs/lib/base/task-definition.ts#L509
            image: obtainDefaultFluentBitECRImage(fargateService.service.taskDefinition, fargateService.service.taskDefinition.defaultContainer?.logDriverConfig),
            essential: true,
            firelensConfig: {
                type: FirelensLogRouterType.FLUENTBIT,
                options: {
                    enableECSLogMetadata: true,
                    configFileType: FirelensConfigFileType.FILE,
                    // This enables parsing of log messages that are json lines
                    configFileValue: '/fluent-bit/configs/parse-json.conf'
                }
            },
            memoryReservationMiB: 50,
            logging: new AwsLogDriver({streamPrefix: 'firelens'})
        });
        firelensLogRouter.logDriverConfig;
    }
    fargateService.targetGroup.configureHealthCheck({
        path: this._props.healthUrlPath,
        port: this._props.containerPort.toString(),
        interval: Duration.seconds(120),
        unhealthyThresholdCount: 5
    });
    const scalableTaskCount = fargateService.service.autoScaleTaskCount({
        minCapacity: this._props.desiredCount,
        maxCapacity: this._props.maxCapacity
    });
    scalableTaskCount.scaleOnCpuUtilization(`ScaleOnCpuUtilization${this._props.cpuTargetUtilization}`, {
        targetUtilizationPercent: this._props.cpuTargetUtilization
    });
    scalableTaskCount.scaleOnMemoryUtilization(`ScaleOnMemoryUtilization${this._props.memoryTargetUtilization}`, {
        targetUtilizationPercent: this._props.memoryTargetUtilization
    });
    this.fargateService = fargateService;
}

资源:

• 我是如何第一次发现它是可能的。 https://github.com/aws-samples/amazon-ecs-firelens-examples/tree/master/examples/fluent-bit/parse-json
• 我如何发现 CDK 可以实现这一点 https://github.com/aws/aws-cdk/pull/6322
• 从 AWS 服务的角度理解 https://docs.aws.amazon.com/AmazonECS/latest/userguide/using_firelens.html
• 缩小其在 CDK 源代码中的位置。 https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ecs.FirelensLogRouter.html
• 最终我来到这里并发现了https://github.com/aws/aws-cdk/blob/60c782fe173449ebf912f509de7db6df89985915/packages/%40aws-cdk/aws-ecs/lib/base/task-definition.ts#L509