我一直在尝试修改有关图像分类的 ML.NET 教程，但遇到了架构不匹配问题，其中模型需要 VarVector，但它传递的是 Vector。
原始示例创建一个 IDataView，其中名为“ImagePath”的列填充了图像路径，然后使用 MLContext.Transforms.LoadRawImageBytes() 生成字节数据的“Image”列。然后将生成的 IDataView 输入到 Multiclassification.Trainers.ImageClassification() 对象。这非常简单。
我希望将其概括为涉及更多的图像预处理。我希望预处理管道降低图像分辨率，然后将图像裁剪为正确的大小，然后再将其输入 ImageClassification() 训练器。它遵循与示例相同的流程，但更改了处理管道以在图像处理管道中使用 LoadImages()、ResizeImage() 和 ExtractPixels() 估计器。下面是我正在使用的代码片段。
// Fill an IDataView with the Label and ImagePath information:
// The ImageData class is defined for this purpose and it is just:
//
// public class ImageData {
// public string ImagePath { get; set; }
// public string Label { get; set; }
// }
//
// Create a bunch of image names and their labels into a list of ImageData objects.
IEnumerable<ImageData> images = LoadImagesFromDirectory(folder: assetsRelativePath,
useFolderNameAsLabel: true);
IDataView imageData = mlContext.Data.LoadFromEnumerable(images);
// Define the preprocessing pipeline to:
// 1. Map the Label to a unique key
// 2. Load the image data using the ImagePath column
// 3. Reduce the resolution of the image data
// 4. Crop the reduced resolution image data
// 5. Transform the image data from type "Image" to a byte array.
var preprocessingPipeline = mlContext.Transforms.Conversion.MapValueToKey(
inputColumnName: "Label",
outputColumnName: "LabelAsKey")
.Append(mlContext.Transforms.LoadImages(
inputColumnName: "ImagePath",
outputColumnName: "RawImage",
imageFolder: assetsRelativePath))
.Append(mlContext.Transforms.ResizeImages(
inputColumnName: "RawImage",
outputColumnName: "ResReduxImage",
imageWidth: 512,
imageHeight: 459,
resizing: Microsoft.ML.Transforms.Image.ImageResizingEstimator.ResizingKind.Fill))
.Append(mlContext.Transforms.ResizeImages(
inputColumnName: "ResReduxImage",
outputColumnName: "CroppedImage",
imageWidth: 512,
imageHeight: 459,
resizing: Microsoft.ML.Transforms.Image.ImageResizingEstimator.ResizingKind.IsoCrop,
cropAnchor: Microsoft.ML.Transforms.Image.ImageResizingEstimator.Anchor.Center))
.Append(mlContext.Transforms.ExtractPixels(
inputColumnName: "CroppedImage",
outputColumnName: "Image",
outputAsFloatArray: false,
colorsToExtract: ImagePixelExtractingEstimator.ColorBits.Red));
// Transform the raw ImageData into the TransformedData that the Model will train on
IDataView preProcessedData = preprocessingPipeline.Fit(imageData).Transform(imageData);
// Partition the data set for training and validation
TrainTestData trainSplit = mlContext.Data.TrainTestSplit(data: preProcessedData, testFraction: 0.4);
TrainTestData validationTestSplit = mlContext.Data.TrainTestSplit(trainSplit.TestSet);
IDataView trainSet = trainSplit.TrainSet;
IDataView validationSet = validationTestSplit.TrainSet;
IDataView testSet = validationTestSplit.TestSet;
var classifierOptions = new ImageClassificationTrainer.Options()
{
FeatureColumnName = "Image",
LabelColumnName = "LabelAsKey",
ValidationSet = validationSet,
Arch = ImageClassificationTrainer.Architecture.ResnetV2101,
MetricsCallback = (metrics) => Console.WriteLine(metrics),
TestOnTrainSet = false,
ReuseTrainSetBottleneckCachedValues = true,
ReuseValidationSetBottleneckCachedValues = true,
WorkspacePath=workspaceRelativePath
};
var trainingPipeline =
mlContext.MulticlassClassification.Trainers.ImageClassification(classifierOptions)
.Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
// When I call Fit() in the next line of code, the following runtime error occurs:
// System.ArgumentOutOfRangeException: 'Schema mismatch for feature column 'Image':
// expected VarVector<Byte>, got Vector<Byte> Parameter name: inputSchema'
ITransformer trainedModel = trainingPipeline.Fit(trainSet);
我对 VarVector 和 Vector 不匹配感到困惑。这两天我尝试了以下方法，但没有成功:
- 定义要传递给转换的自定义 DataViewSchema
- 通过从 IDataView 派生来定义自定义 DataView
- 使用 CustomMapping 定义自定义映射<>更改类型
任何人都可以提供一些关于如何让自己摆脱困境的指导吗？我真的很想避免读取图像，对其进行预处理，将其写入文件，然后调用原始的 LoadRawImageBytes() 函数。我不想要额外的文件系统工作。
最佳答案
我想出了如何实现这一点。这里的问题是底层类型是正确的，但架构是错误的。我能够通过 CustomMapping 和一些属性魔法解决我的问题。请参阅 CustomMapping documentation了解 CustomMapping 的工作原理。
更正后的代码片段如下。
// SOMEWHERE OUTSIDE OF THE FUNCTION DEFINE:
// Define a class to represent the input column type for the custom transform
class InputData
{
[VectorType(1)] // attribute specifies vector type of known length
public VBuffer<Byte> Image1; // the VBuffer<> type actually represents the data
}
// Define a class to represent the output column type for the custom transform
class OutputData
{
// THE MAGICAL FIX: attribute specifies vector type of unknown length (i.e. VarVector)
[VectorType()]
public VBuffer<Byte> Image; // the VBuffer<> type actually represents the data
}
// ------------------------------------------------------------------
// INSIDE THE FUNCTION THAT WILL DO THE TRAINING
// ------------------------------------------------------------------
// Fill an IDataView with the Label and ImagePath information:
// The ImageData class is defined for this purpose and it is just:
//
// public class ImageData {
// public string ImagePath { get; set; }
// public string Label { get; set; }
// }
//
// Create a bunch of image names and their labels into a list of ImageData objects.
IEnumerable<ImageData> images = LoadImagesFromDirectory(folder: assetsRelativePath,
useFolderNameAsLabel: true);
IDataView imageData = mlContext.Data.LoadFromEnumerable(images);
// Define the preprocessing pipeline to:
// 1. Map the Label to a unique key
// 2. Load the image data using the ImagePath column
// 3. Reduce the resolution of the image data
// 4. Crop the reduced resolution image data
// 5. Transform the image data from type "Image" to a byte array.
var preprocessingPipeline = mlContext.Transforms.Conversion.MapValueToKey(
inputColumnName: "Label",
outputColumnName: "LabelAsKey")
.Append(mlContext.Transforms.LoadImages(
inputColumnName: "ImagePath",
outputColumnName: "RawImage",
imageFolder: assetsRelativePath))
.Append(mlContext.Transforms.ResizeImages(
inputColumnName: "RawImage",
outputColumnName: "ResReduxImage",
imageWidth: 512,
imageHeight: 459,
resizing: Microsoft.ML.Transforms.Image.ImageResizingEstimator.ResizingKind.Fill))
.Append(mlContext.Transforms.ResizeImages(
inputColumnName: "ResReduxImage",
outputColumnName: "CroppedImage",
imageWidth: 512,
imageHeight: 459,
resizing: Microsoft.ML.Transforms.Image.ImageResizingEstimator.ResizingKind.IsoCrop,
cropAnchor: Microsoft.ML.Transforms.Image.ImageResizingEstimator.Anchor.Center))
.Append(mlContext.Transforms.ExtractPixels(
inputColumnName: "CroppedImage",
outputColumnName: "Image1",
outputAsFloatArray: false,
colorsToExtract: ImagePixelExtractingEstimator.ColorBits.Red));
// Transform the raw ImageData into the TransformedData that the Model will train on
IDataView preProcessedData =
preprocessingPipeline.Fit(imageData).Transform(imageData);
// Create an action representing the custom transform...
// The data itself does not need to be changed at all, so this is just an
// identity transform
Action<InputData, OutputData> convertVecType
= (input, output) => output.Image = input.Image1;
var convertTypePipeline = mlContext.Transforms.CustomMapping(convertVecType,
"convertVecType");
preProcessedData =
convertTypePipeline.Fit(preProcessedData).Transform(preProcessedData);
// Partition the data set for training and validation
TrainTestData trainSplit = mlContext.Data.TrainTestSplit(data: preProcessedData, testFraction: 0.4);
TrainTestData validationTestSplit = mlContext.Data.TrainTestSplit(trainSplit.TestSet);
IDataView trainSet = trainSplit.TrainSet;
IDataView validationSet = validationTestSplit.TrainSet;
IDataView testSet = validationTestSplit.TestSet;
var classifierOptions = new ImageClassificationTrainer.Options()
{
FeatureColumnName = "Image",
LabelColumnName = "LabelAsKey",
ValidationSet = validationSet,
Arch = ImageClassificationTrainer.Architecture.ResnetV2101,
MetricsCallback = (metrics) => Console.WriteLine(metrics),
TestOnTrainSet = false,
ReuseTrainSetBottleneckCachedValues = true,
ReuseValidationSetBottleneckCachedValues = true,
WorkspacePath=workspaceRelativePath
};
var trainingPipeline =
mlContext.MulticlassClassification.Trainers.ImageClassification(classifierOptions)
.Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
// When I call Fit() in the next line of code, the following runtime error occurs:
// System.ArgumentOutOfRangeException: 'Schema mismatch for feature column 'Image':
// expected VarVector<Byte>, got Vector<Byte> Parameter name: inputSchema'
ITransformer trainedModel = trainingPipeline.Fit(trainSet);
关于c# - ML.NET 架构不匹配 : expected VarVector<Byte> got Vector<Byte>，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/63930046/