python - 如果网络资源处于自定义子网模式，网络接口(interface)必须指定一个子网

Question

尝试使用 Python SDK 运行 Google Dataflow 示例。

我可以在本地运行:

python -m apache_beam.examples.wordcount --output OUTPUT_FILE

但是当尝试在 GCP 上运行时:

python -m apache_beam.examples.wordcount \
    --project myproject \
    --job_name myproject-wordcount \
    --runner DataflowRunner \
    --staging_location gs://myproject/staging \
    --output gs://myproject/output \
    --network myproject-network \
    --zone europe-west1-b \
    --subnetwork regions/europe-west1/subnetworks/europe-west1 \
    --temp_location gs://myproject/temp

我收到以下错误:

apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
(9d6636d5e214c789): Workflow failed. Causes: (ab9869cb8161ec27): Error:
 Message: Invalid value for field 'resource.properties.networkInterfaces[0].subnetwork': ''. Network interface must specify a subnet if the network resource is in custom subnet mode.
 HTTP Code: 400

我正在使用 apache-beam Python SDK 0.6.0

有人能帮忙吗？

Answer 1

添加 --subnetwork 和 default 子网解决了我的问题:

mvn compile exec:java \
-Dexec.mainClass=my.MainClass \
-Dexec.args="--runner=DataflowRunner \
--project=my-project \
--stagingLocation=gs://my-bucket/staging \
--gcpTemplateLocation=gs://my-bucket/template \
--templateLocation=gs://my-bucket/template \
--gcpTempLocation=gs://my-bucket/tmp \
--region=us-central1 \
--subnetwork=regions/us-central1/subnetworks/default" \
-P dataflow-runner