Azure datafactory v2 使用 For Each 执行管道

标签 azure azure-data-factory

我正在尝试使用“执行管道”来调用具有 ForEach 事件的管道。我收到错误。

  1. 执行管道的 Json:
[
    {
        "name": "pipeline3",
        "properties": {
            "activities": [
                {
                    "name": "Test_invoke1",
                    "type": "ExecutePipeline",
                    "dependsOn": [],
                    "userProperties": [],
                    "typeProperties": {
                        "pipeline": {
                            "referenceName": "MAIN_SA_copy1",
                            "type": "PipelineReference"
                        },
                        "waitOnCompletion": true
                    }
                }
            ],
            "annotations": []
        }
    }
]
  • Jason 为每个事件调用管道:
  • [
        {
            "name": "MAIN_SA_copy1",
            "properties": {
                "activities": [
                    {
                        "name": "Collect_SA_Data",
                        "type": "ForEach",
                        "dependsOn": [],
                        "userProperties": [],
                        "typeProperties": {
                            "items": {
                                "value": "@pipeline().parameters.TableNames",
                                "type": "Expression"
                            },
                            "batchCount": 15,
                            "activities": [
                                {
                                    "name": "Sink_SAdata_toDL",
                                    "type": "Copy",
                                    "dependsOn": [],
                                    "policy": {
                                        "timeout": "7.00:00:00",
                                        "retry": 0,
                                        "retryIntervalInSeconds": 30,
                                        "secureOutput": false,
                                        "secureInput": false
                                    },
                                    "userProperties": [
                                        {
                                            "name": "Destination",
                                            "value": "@{pipeline().parameters.DLFilePath}/@{item()}"
                                        }
                                    ],
                                    "typeProperties": {
                                        "source": {
                                            "type": "SqlServerSource",
                                            "sqlReaderQuery": {
                                                "value": "@concat('SELECT * FROM ',item())",
                                                "type": "Expression"
                                            }
                                        },
                                        "sink": {
                                            "type": "AzureBlobFSSink"
                                        },
                                        "enableStaging": false,
                                        "parallelCopies": 1,
                                        "dataIntegrationUnits": 4
                                    },
                                    "inputs": [
                                        {
                                            "referenceName": "SrcDS_StructuringAnalytics",
                                            "type": "DatasetReference"
                                        }
                                    ],
                                    "outputs": [
                                        {
                                            "referenceName": "ADLS",
                                            "type": "DatasetReference",
                                            "parameters": {
                                                "FilePath": "@pipeline().parameters.DLFilePath",
                                                "FileName": {
                                                    "value": "@concat(item(),'.orc')",
                                                    "type": "Expression"
                                                }
                                            }
                                        }
                                    ]
                                }
                            ]
                        }
                    }
                ],
                "parameters": {
                    "DLFilePath": {
                        "type": "string",
                        "defaultValue": "extracts/StructuringAnalytics"
                    },
                    "TableNames": {
                        "type": "array",
                        "defaultValue": [
                            "fom.FOMLineItem_manual"
                        ]
                    }
                },
                "variables": {
                    "QryTableColumn": {
                        "type": "String"
                    },
                    "QryTable": {
                        "type": "String"
                    }
                },
                "folder": {
                    "name": "StructuringAnalytics"
                },
                "annotations": []
            },
            "type": "Microsoft.DataFactory/factories/pipelines"
        }
    ]
    

    我收到错误:

    [
        {
            "errorCode": "BadRequest",
            "message": "Operation on target Collect_SA_Data failed: The execution of template action 'Collect_SA_Data' failed: the result of the evaluation of 'foreach' expression '@pipeline().parameters.TableNames' is of type 'String'. The result must be a valid array.",
            "failureType": "UserError",
            "target": "Test_invoke1",
            "details": ""
        }
    ]
    

    输入:

    "pipeline": {
        "referenceName": "MAIN_SA_copy1",
        "type": "PipelineReference"
    },
    "waitOnCompletion": true,
    "parameters": {
        "DLFilePath": "extracts/StructuringAnalytics",
        "TableNames": "[\"fom.FOMLineItem_manual\"]"
    }
    

    最佳答案

    我猜您正在使用 UI 来设置管道及其参数,并且我猜您希望将被调用管道的数组参数像其他地方一样放置,如下所示: (这都是我的猜测,因为我只是做了完全相同的事情,得到了相同的结果)

    Wrong

    技巧是在代码中定义数组(["table1", "table2"]): Code

    用户界面中的输入将如下所示:

    Right

    现在可以了!
    看起来,数据工厂以其他方式将整个数组视为某个数组的一个元素。因此,使用 array() 函数的解决方案有时会起作用。
    它看起来像一个错误,定义数组参数输入..

    (必须编辑答案,我首先认为省略 UI 输入中的冒号就足够了)

    关于Azure datafactory v2 使用 For Each 执行管道,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/61109985/

    相关文章:

    visual-studio - 如何重新打开 azure 部署警告模式 "deployment environment in use"?

    azure - 授权超时: Azure Data Lake Store Configuration in Azure Data Factory

    c# - 将多行 JSON 反序列化为 C# 对象

    azure-data-factory - 使用参数的 Azure 数据工厂数据流表达式生成器

    azure-data-factory - GetMetadata 获取 Azure 数据工厂中的完整文件目录

    postgresql - Azure数据工厂: Which Version of npgsql is used?

    azure - 通过 Azure 数据工厂从 ftp 站点获取 zip 和非压缩文件 - 解压缩

    Azure DevOps 发布错误 "Microsoft.TeamFoundation.DistributedTask.Pipelines.PipelineValidationException (type ReleaseManagementHandledException)"

    c# - 从服务器卸载 SSL 时,ASP.NET OWIN 身份错误 AAD 身份验证失败

    Azure 资源管理器 : move VM to availability group