amazon-web-services - Aws KInesis Terraform - 如何将数据流连接到 Data Firehose

标签 amazon-web-services amazon-kinesis amazon-kinesis-firehose

我想使用 terraform、Kinesis datastream 和 data firehose 创建并连接它们(作为管道)。当我使用 UI 时,当我转到 firehose 时,我可以转到 source->Kinesis stream,然后选择我创建的 kinesis 流。但我想使用 terraform 来完成。

这是创建运动流的代码(我从官方运动文档中获取):

resource "aws_kinesis_stream" "test_stream" {
  name             = "terraform-kinesis-test"
  shard_count      = 1
  retention_period = 30

  shard_level_metrics = [
    "IncomingBytes",
    "OutgoingBytes",
  ]

  tags = {
    Environment = "test"
  }

这是数据流水线的代码:

resource "aws_elasticsearch_domain" "test_cluster" {
  domain_name = "firehose-es-test"
  elasticsearch_version = "6.4"
    cluster_config {
    instance_type = "t2.small.elasticsearch"
  }
  ebs_options{
    ebs_enabled = true
    volume_size = 10
  }

}

resource "aws_iam_role" "firehose_role" {
  name = "firehose_test_role"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "firehose.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF
}
resource "aws_kinesis_firehose_delivery_stream" "test_stream" {
  name        = "terraform-kinesis-firehose-test-stream"
  destination = "elasticsearch"
/*
  s3_configuration {
    role_arn           = "${aws_iam_role.firehose_role.arn}"
    bucket_arn         = "${aws_s3_bucket.bucket.arn}"
    buffer_size        = 10
    buffer_interval    = 400
    compression_format = "GZIP"
  }
*/
  elasticsearch_configuration {
    domain_arn = "${aws_elasticsearch_domain.test_cluster.arn}"
    role_arn   = "${aws_iam_role.firehose_role.arn}"
    index_name = "test"
    type_name  = "test"

    processing_configuration {
      enabled = "true"

    }
  }
}

那么我如何连接它们,是像 ${aws_kinesis_stream.test_stream.arn} 这样的东西吗?或类似的东西?

我用的是aws_kinesis_stream的官方文档和 aws_kinesis_firehose_delivery_stream ( Elasticsearch 目的地)。

最佳答案

这在 kinesis_firehose_delivery_stream] 文档中。通过示例滚动到 Argument Reference部分,你会看到这个:

The kinesis_source_configuration object supports the following:

kinesis_stream_arn (Required) The kinesis stream used as the source of the firehose delivery stream. role_arn (Required) The ARN of the role that provides access to the source Kinesis stream.

关于amazon-web-services - Aws KInesis Terraform - 如何将数据流连接到 Data Firehose,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/55421995/

相关文章:

amazon-web-services - Kinesis Analytics 目标指南 : Lambda vs Kinesis Stream to Lambda

amazon-web-services - 如何使用 KCL 确定特定分区键的分片 ID?

docker - 如何配置ECS以允许访问Kinesis Stream

python - 使用 AWS Lambda/CloudFormation 转换 AWS Kinesis Firehose 中的数据

amazon-web-services - 在 AWS 中创建加密运动流时,%FIREHOSE_POLICY_TEMPLATE_PLACEHOLDER% 是什么意思?

node.js - 将 NodeJS 运行时从 8.10 更新到 10.x 或 12.x - AWS Amplify

java - 无法解析导入 com.amazonaws.services.s3.AmazonS3ClientBuilder

amazon-web-services - 通过负载将 AWS Kinesis Firehose 数据分区到 s3

amazon-web-services - AWS 云组建;将模板分解为多个文件并使用 cfn-include 传入变量

performance - 什么是ECU(AWS)?这意味着什么?