regex - Perl 使用 file2 从 file1 中删除单词

Question

我正在使用 perl 脚本删除文本中的所有停用词。停用词逐行存储。我正在使用 Mac OSX 命令行并且 perl 已正确安装。

此脚本运行不正常，存在边界问题。

#!/usr/bin/env perl -w
# usage: script.pl words text >newfile
use English;

# poor man's argument handler
open(WORDS, shift @ARGV) || die "failed to open words file: $!";
open(REPLACE, shift @ARGV) || die "failed to open replacement file: $!";

my @words;
# get all words into an array
while ($_=<WORDS>) { 
  chop; # strip eol
  push @words, split; # break up words on line
}

# (optional)
# sort by length (makes sure smaller words don't trump bigger ones); ie, "then" vs "the"
@words=sort { length($b) <=> length($a) } @words;

# slurp text file into one variable.
undef $RS;
$text = <REPLACE>;

# now for each word, do a global search-and-replace; make sure only words are replaced; remove possible following space.
foreach $word (@words) { 
     $text =~ s/\b\Q$word\E\s?//sg;
}

# output "fixed" text
print $text;

样本.txt

$ cat sample.txt
how about i decide to look at it afterwards what
across do you think is it a good idea to go out and about i 
think id rather go up and above

停用词.txt

I
a
about
an
are
as
at
be
by
com
for
from
how
in
is
it
..

输出:

$ ./remove.pl stopwords.txt sample.txt 
i decide look fterwards cross do you think good idea go out d i 
think id rather go up d bove

如您所见，它使用 as fterwards 替换 afterwards。认为这是一个正则表达式问题。请有人可以帮我快速修补这个问题吗？感谢所有帮助:J

Answer 1

在 $word 的两边使用单词边界。目前，您只是在开始时检查它。

您不需要 \s? 条件和 \b 就位:

$text =~ s/\b\Q$word\E\b//sg;