azure - 如何进行 Azure Blob 存储和 Azure SQL Db 原子事务

Question

We have a Blob storage container in Azure for uploading application specific documents and we have Azure Sql Db where meta data for particular files are saved during the file upload process. This upload process needs to be consistent so that we should not have files in the storage for which there is no record of meta data in Sql Db and vice versa. We are uploading list of files which we get from front-end as multi-part HttpContent. From Web Api controller we call the upload service passing the httpContent, file names and a folder path where the files will be uploaded. The Web Api controller, service method, repository, all are asyn.

var files = await this.uploadService.UploadFiles(httpContent, fileNames, pathName);

服务方法如下:

public async Task<List<FileUploadModel>> UploadFiles(HttpContent httpContent, List<string> fileNames, string folderPath)
        {
            var blobUploadProvider = this.Container.Resolve<UploadProvider>(
                new DependencyOverride<UploadProviderModel>(new UploadProviderModel(fileNames, folderPath)));

            var list = await httpContent.ReadAsMultipartAsync(blobUploadProvider).ContinueWith(
                task =>
                {
                    if (task.IsFaulted || task.IsCanceled)
                    {
                        throw task.Exception;
                    }

                    var provider = task.Result;
                    return provider.Uploads.ToList();
                });

            return list;
        }

The service method uses a customized upload provider which is derived from System.Net.Http.MultipartFileStreamProvider and we resolve this using a dependency resolver. After this, we create the meta deta models for each of those files and then save in the Db using Entity framework. The full process works fine in ideal situation.

The problem is if the upload process is successful but somehow the Db operation fails, then we have files uploaded in Blob storage but there is no corresponding entry in Sql Db, and thus there is data inconsistency.

以下是系统中使用的不同技术:

• Azure Api 应用
• Azure Blob 存储
• 网络 API
• .Net 4.6.1
• Entity Framework 6.1.3
• Azure MSSql 数据库(我们不使用任何虚拟机)

我尝试使用 TransactionScope 来保持一致性，但这似乎不适用于 Blob 和 Db(仅适用于 Db)

• 我们如何解决这个问题？

• 是否有任何内置或支持的功能？

• 这种情况下的最佳做法是什么？

Answer 1

Is there any built in or supported feature for this?

截至今天，没有。本质上，Blob 服务和 SQL 数据库是两个独立的服务，因此不可能像您期望的那样实现“原子事务”功能。

How do we solve this issue?

我可以想出两种方法来解决这个问题(我相信还会有其他方法):

1. 实现您自己的事务功能:主要检查数据库事务是否失败，如果发生这种情况，请手动删除 blob。
2. 使用一些后台进程:在这里，您将继续将数据保存在 Blob 存储中，然后通过一些后台进程定期找出孤立的 Blob，并删除这些 Blob。