performance - 为什么使用 Inline::C 用虚拟函数替换 Perl 的 s///会导致显着变慢？

Question

我有一个大约 100,000 个元素的字符串数组。我需要遍历每个元素并用其他词替换一些词。在纯 perl 中这需要几秒钟。我需要尽可能加快速度。我正在使用以下代码段进行测试:

use strict;

my $string = "This is some string. Its only purpose is for testing.";
for( my $i = 1; $i < 100000; $i++ ) {
  $string =~ s/old1/new1/ig;
  $string =~ s/old2/new2/ig;
  $string =~ s/old3/new3/ig;
  $string =~ s/old4/new4/ig;
  $string =~ s/old5/new5/ig;
}

我知道这实际上并没有替换测试字符串中的任何内容，但它仅用于速度测试。

我的希望寄托在 Inline::C .我从未与 Inline::C 合作过之前但在阅读了一点之后，我认为它实现起来相当简单。但显然，即使调用一个什么都不做的 stub 函数也会慢很多。这是我测试的片段:

use strict;
use Benchmark qw ( timethese );

use Inline 'C';

timethese(
   5,
   {
      "Pure Perl"  => \&pure_perl,
      "Inline C"   => \&inline_c
   }
);

sub pure_perl {
  my $string = "This is some string. Its only purpose is for testing.";
  for( my $i = 1; $i < 1000000; $i++ ) {
    $string =~ s/old1/new1/ig;
    $string =~ s/old2/new2/ig;
    $string =~ s/old3/new3/ig;
    $string =~ s/old4/new4/ig;
    $string =~ s/old5/new5/ig;
  }
}

sub inline_c {
  my $string = "This is some string. Its only purpose is for testing.";
  for( my $i = 1; $i < 1000000; $i++ ) {
    $string = findreplace( $string, "old1", "new1" );
    $string = findreplace( $string, "old2", "new2" );
    $string = findreplace( $string, "old3", "new3" );
    $string = findreplace( $string, "old4", "new4" );
    $string = findreplace( $string, "old5", "new5" );
  }
}

__DATA__
__C__

char *
findreplace( char *text, char *what, char *with ) {

  return text;

}

在我的 Linux 机器上，结果是:

Benchmark: timing 5 iterations of Inline C, Pure Perl...
  Inline C:  6 wallclock secs ( 5.51 usr +  0.02 sys =  5.53 CPU) @  0.90/s (n=5)
  Pure Perl:  2 wallclock secs ( 2.51 usr +  0.00 sys =  2.51 CPU) @  1.99/s (n=5)

纯 Perl 的速度是调用空 C 函数的两倍。完全不是我所期望的!同样，我以前从未使用过 Inline::C，所以也许我在这里遗漏了一些东西？

Answer 1

在使用 Inline::C 的版本中，您保留了原始纯 Perl 脚本中的所有内容，并且只更改了一件事:此外，您替换了 Perl 高度优化的 s///实现更差。调用您的虚拟函数实际上涉及工作，而 s///在这种情况下，调用做了很多事情。 Inline::C 是先验不可能的。运行速度更快的版本。

在 C 端，函数

char *
findreplace( char *text, char *what, char *with ) {

  return text;

}

不是“什么都不做”的功能。调用它涉及解包参数。 text 指向的字符串必须复制到返回值。每次调用都会产生一些开销。

鉴于 s///没有替代品，不涉及复制。另外，Perl 的 s///高度优化。你确定你可以写一个更好的查找和替换来弥补调用外部函数的开销吗？

如果您使用以下实现，您应该获得相当的速度:

sub inline_c {
  my $string = "This is some string. It's only purpose is for testing.";
  for( my $i = 1; $i < 1000000; $i++ ) {
    findreplace( $string );
    findreplace( $string );
    findreplace( $string );
    findreplace( $string );
    findreplace( $string );
  }
}

__END__
__C__

void findreplace( char *text ) {
    return;

}

Benchmark: timing 5 iterations of Inline C, Pure Perl...
  Inline C:  6 wallclock secs ( 5.69 usr +  0.00 sys =  5.69 CPU) @  0.88/s (n=5)
 Pure Perl:  6 wallclock secs ( 5.70 usr +  0.00 sys =  5.70 CPU) @  0.88/s (n=5)

The one possibility of gaining speed is to exploit any special structure involved in the search pattern and replacements and write something to implement that.

On the Perl side, you should at least pre-compile the patterns.

Also, since your problem is embarrassingly parallel, you are better off looking into chopping up the work into as many chunks as you have cores to work with.

For example, take a look at the Perl entries in the regex-redux task in the Benchmarks Game:

Perl #4 (fork only): 14.13 seconds

and

Perl #3 (fork & threads): 14.47 seconds

versus

Perl #1: 34.01 seconds

That is, some primitive exploitation of parallelization possibilities results in a 60% speedup. That problem is not exactly comparable because the substitutions must be done sequentially, but still gives you an idea.

If you have eight cores, dole out the work to eight cores.

Also, consider the following script:

#!/usr/bin/env perl

use warnings;
use strict;

use Data::Fake::Text;
use List::Util qw( sum );
use Time::HiRes qw( time );

use constant INPUT_SIZE => $ARGV[0] // 1_000_000;

run();

sub run {
    my @substitutions = (
        sub { s/dolor/new1/ig   },
        sub { s/fuga/new2/ig    },
        sub { s/facilis/new3/ig },
        sub { s/tempo/new4/ig   },
        sub { s/magni/new5/ig   },
    );

    my @times;
    for (1 .. 5) {
        my $data = read_input();
        my $t0 = time;
        find_and_replace($data, \@substitutions);
        push @times, time - $t0;
    }

    printf "%.4f\n", sum(@times)/@times;

    return;
}

sub find_and_replace {
    my $data = shift;
    my $substitutions = shift;

    for ( @$data ) {
        for my $s ( @$substitutions ) {
            $s->();
        }
    }
    return;
}

{
    my @input;
    sub read_input {
        @input
            or @input = map fake_sentences(1)->(), 1 .. INPUT_SIZE;
        return [ @input ];
    }
}

在这种情况下，find_and_replace 的每次调用我的笔记本电脑大约需要 2.3 秒。五次复制在大约 30 秒内运行。开销是生成 1,000,000 个句子数据集并将其复制四次的综合成本。