python - 从 2 个单独的列表中提取信息

Question

我想使用 python 从大文件中提取某些信息。 我有 3 个输入文件。 第一个输入文件 (input_file) 是数据文件，它是一个 3 列制表符分隔的文件，如下所示:

engineer-n imposition-n 2.82169386609e-05
motor-n imposition-n 0.000102011705117
creature-n imposition-n 0.000121321951973
bomb-n imposition-n 0.000680302090112
sedation-n oppression-n 0.000397074586994
roadblock-n oppression-n 5.96190620847e-05
liability-n oppression-n 0.012845281978
currency-n oppression-n 0.000793989880202

第二个输入文件 (colA_file) 是一个 1 列列表，如下所示:

bomb-n
sedation-n
roadblock-n
surrender-n

第三个输入文件(colB_file)也是一个 1 列列表(与 colA_file 相同，但信息不同)，如下所示:

adjective-n
homeless-n
imposition-n
oppression-n

我想从 colA 和 colB 中找到的输入文件中提取信息。 根据我提供的示例数据，这意味着过滤除以下几行之外的所有信息:

bomb-n imposition-n 0.000680302090112
sedation-n oppression-n 0.000397074586994
roadblock-n oppression-n 5.96190620847e-05

我用 Python 编写了以下代码来解决此任务:

def test_fnc(input_file, colA_file, colB_file, output_file):
    nounA = []
    with open(colA_file, "rb") as opened_colA:
        for aLine in opened_colA:
            nounA.append(aLine.strip())
            #print nounA

    nounB = []
    with open(colB_file, "rb") as opened_colB:
        for bLine in opened_colB:
            nounB.append(bLine.strip())
            #print nounB

    with open(output_file, "wb") as outfile:
        with open(input_file, "rb") as opened_input:
            for cLine in opened_input:
                splitted_cLine = cLine.split()
                #print splitted_cLine
                if splitted_cLine[0] in nounA and splitted_cLine[1] in nounB:
                    outstring = "\t".join(splitted_cLine)
                    outfile.write(outstring + "\n")

test_fnc(input_file, colA_file, colB_file, output_file)

但是，它仅输出 1 行，就好像它没有迭代提供的列表输入一样。 似乎我的列表是相互附加的，从一项开始，并随着每个附加项的增加而增加。 因此，我也尝试引用以下列表:

    for bLine in opened_colB:
        nounB = bLine

结果与上面相同。

Answer 1

如果您不介意依赖性，我会使用pandas或numpy。与 pandas.DataFrame然后你可以执行isin检查其列。否则我建议使用集合，因为正则表达式应该慢得多。像这样的事情:

with open(colA_file, "rb") as file_h:
    noun_a = set(line.strip() for line in file_h)

with open(colB_file, "rb") as file_h:
    noun_b = set(line.strip() for line in file_h)

with open(output_file, "wb") as outfile:
    with open(input_file, "rb") as opened_input:
        for line in opened_input:
            split_line = line.split()
            if split_line[0] in noun_a and split_line[1] in noun_b:
                outfile.write(line)