azure 计算机视觉检测文本

Question

我正在使用 azure 计算机视觉从图像中提取文本，它按预期工作，但现在我面临着一个挑战，我必须只检索特定文本，而不是图像中的所有文本，以及我想提取一张图像与另一张图像可能不同的内容。(也许这些图像可以有一个共同的文本) Azure 计算机视觉有没有办法可以帮助我做到这一点？ 示例:成分，因为有很多成分照片与简单文本混合，我如何才能提取成分:

谢谢

Answer 1

Azure Cognitive Service Vision API能够将连续文本检测为“区域”，区域由“行”组成，行包含“单词”。一个简单的方法是检测某个关键字(例如“成分”)是否出现在某个区域中，如果出现，则将出现在该关键字之后的所有单词保存在同一区域中(使用编辑距离来允许一些偏差):

import requests
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import matplotlib.patches as patches
import Levenshtein  # install with pip install python-Levenshtein

def draw_bounding_box(ax, coordinates, color, size):
    coordinates_array = coordinates.split(",")
    x = int(coordinates_array[0])
    y = int(coordinates_array[1])
    w = int(coordinates_array[2])
    h = int(coordinates_array[3])
    try:
        rect = patches.Rectangle((x, y), w, h, linewidth=size, edgecolor=color, facecolor='none')
        ax.add_patch(rect)
    except:
        pass

def text_detection(service_endpoint, subscription_key, filename):
    with open(filename, 'rb') as f:
        data = f.read()
    headers = {'Ocp-Apim-Subscription-Key': subscription_key, 'Content-Type': 'application/octet-stream'}
    params = {'language': 'fr', 'detectOrientation': 'false'}
    response = requests.post(service_endpoint, headers=headers, params=params, data=data)

    analysis = response.json()
    regions = analysis['regions']

    im = np.array(Image.open(filename), dtype=np.uint8)
    fig, ax = plt.subplots(1)
    ax.imshow(im)
    ingredients = ""

    for region in regions:
        in_ingredients = False
        draw_bounding_box(ax, region["boundingBox"], "g", 3)
        for line in region["lines"]:
            draw_bounding_box(ax, line["boundingBox"], "b", 2)
            for word in line["words"]:
                text = word["text"]
                if Levenshtein.distance(text.lower(), "ingredients") <= 2:
                    in_ingredients = True
                if in_ingredients:
                    ingredients += text + " "

    plt.show()
    print(ingredients)

service_endpoint = "https://<region>.api.cognitive.microsoft.com/vision/v3.1/ocr"
subscription_key = "<your key>"
image_path = "/path/to/image.jpg"

text_detection(service_endpoint, subscription_key, image_path)

输出(绿色边界框是区域， azure 线条):

INGRÉDIENTS : Nouilles 88 % : farine de blé ; huile de palme amidon modifié de pomme de terre ; sel , poudres à lever E500,E501. Assaisonnement 12 % : sel ; exhausteurs degoù g\utamate monosodique, E635 ; maltodextrine , sucre, carotte lactose ; curry 5 % (curcuma, cumin, cannelle, fenugrec, muscade, cardamome, clou de girofle, gingembre, piment, coriandre, poivre noir) ; extrait de poulet en poudre arômes (dont blé, lait) ; curcuma en poudre ; poireau 2,2 asse de poulet ; extrait de levure ; ail en poudre ; saucede sojaen poudre (soja, blé, maltodextrine, sel) ; ciboulette deshydratée % ; coriandre en poudre ; Piment ; poivrenar colorants : extrait de paprika, E150c. de céleri, crustacés, mollusques, moutarde, œufs, Poissons, graines de sésame et sulfites

另一个例子:

INGREDIENTS: WATER, TOMATO PASTE), SUGAR! SOYBEAN OIL, DISTILLED OF: MODIÅED CORN STARCH, DEHYDRATED ONION, PHOSPHORIC ACID, XANTHAN GUM, NATURAL FLAVOR, DEHYDRATED GARLIC, POTASSIUM SORBATE AND CALCIUM DISODIUM EDTA (PRESERVATIVES), CITRIC ACID, GUAR GUM, RED 40, YELLOW 6, BLUE 1. OUR PROMISE, QUALITY & SATISFACTION 100% GUARANTEED OR YOUR MONEY BACK

这种方法肯定不是万无一失的。如果关键字出现在文本的其他部分或者成分表像本示例中那样进行划分，则会失败:

ingredient to be naturally derived if it is unchanged from its natural state or has undergone processing yet still retains greater than 50% of its molecular structure from its original plant or mineral source. This formula consists of 96% naturally derived ingredients