我是一个全新的 Perl 新手,正在寻求有关我的第一个 Perl 脚本的帮助
我有一些 30-50GB 的大文件,它们的构造是这样的 - 数百万列和数千行:
A B C D E 1 2 3 4 5 6 7 8 9 10
A B C D E 1 2 3 4 5 6 7 8 9 10
A B C D E 1 2 3 4 5 6 7 8 9 10
A B C D E 1 2 3 4 5 6 7 8 9 10
A B C D E 1 2 3 4 5 6 7 8 9 10
A B C D E 1 2 3 4 5 6 7 8 9 10
A B C D E 1 2 3 4 5 6 7 8 9 10
我想删除列“A”和“C”列,然后是数字列的三分之一,因此“3”列和“6”列,然后是“9”列,直到文件末尾。空格分隔。
我的尝试是这样的:
#!/usr/local/bin/perl
use strict;
use warnings;
my @dataColumns;
my $dataColumnCount;
if(scalar(@ARGV) != 2){
print "\nNo files supplied, please supply file name\n";
exit;
}
my $Infile = $ARGV[0];
my $Outfile = $ARGV[1];
open(INFO,$Infile) || die "Could not open $Infile for reading";
open(OUT,">$Outfile") || die "Could not open $Outfile for writing";
while (<INFO>) {
chop;
@dataColumns = split(" ");
$dataColumnCount = @dataColumns + 1;
#Now remove the first element of the list
shift(@dataColumns);
#Now remove the third element (Note that it is now the second - after removal of the first)
splice(@dataColumns,1,1); # remove the third element (now the second)
#Now remove the 6th (originally the 8th) and every third one thereafter
#NB There are now $dataColumnCount-1 columns
for (my $i = 5; $i < $dataColumnCount-1; $i = $i + 3 ) {
splice($dataColumns; $i; 1);
}
#Now join the remaining elements of the list back into a single string
my $AmendedLine = join(" ",@dataColumns);
#Finally print out the line into your new file
print OUT "$AmendedLine/n";
}
但是我收到了一些奇怪的错误:
全局符号“$i”需要在 Convertversion2.pl 第 36 行显式包名。
全局符号“$i”需要在 Convertversion2.pl 第 36 行显式包名。
全局符号“$i”需要在 Convertversion2.pl 第 36 行显式包名。
全局符号“$i”需要在 Convertversion2.pl 第 36 行显式包名。
Convertversion2.pl 第 37 行的语法错误,靠近“@dataColumns;”
Convertversion2.pl 第 37 行的语法错误,靠近“1)”
我不确定如何纠正这个错误,我想我快到了,但不确定语法错误到底是什么,我不确定如何修复它。
先感谢您。
最佳答案
在我写了关于这个问题的博客后,一位评论者指出,可以将我的测试用例的运行时间减少 45%。我稍微解释了一下他的代码:
my @keep;
while (<>) {
my @data = split;
unless (@keep) {
@keep = (0, 1, 0, 1, 1);
for (my $i = 5; $i < @data; $i += 3) {
push @keep, 1, 1, 0;
}
}
my $i = 0;
print join(' ', grep $keep[$i++], @data), "\n";
}
这几乎是我原来的解决方案所用时间的一半:
$ time ./zz.pl input.data > /dev/null real 0m21.861s user 0m21.310s sys 0m0.280s
Now, it is possible to gain another 45% performance by using Inline::C in a rather dirty way:
#!/usr/bin/env perl
use strict;
use warnings;
use Inline C => <<'END_C'
/*
This code 'works' only in a limited set of circumstances!
Don't expect anything good if you feed it anything other
than plain ASCII
*/
#include <ctype.h>
SV *
extract_fields(char *line, AV *wanted_fields)
{
int ch;
IV current_field = 0;
IV wanted_field = -1;
unsigned char *cursor = line;
unsigned char *field_begin = line;
unsigned char *save_field_begin;
STRLEN field_len = 0;
IV i_wanted = 0;
IV n_wanted = av_len(wanted_fields);
AV *ret = newAV();
while (i_wanted <= n_wanted) {
SV **p_wanted = av_fetch(wanted_fields, i_wanted, 0);
if (!(*p_wanted)) {
croak("av_fetch returned NULL pointer");
}
wanted_field = SvIV(*p_wanted);
while ((ch = *(cursor++))) {
if (!isspace(ch)) {
continue;
}
field_len = cursor - field_begin - 1;
save_field_begin = field_begin;
field_begin = cursor;
current_field += 1;
if (current_field != wanted_field) {
continue;
}
av_push(ret, newSVpvn(save_field_begin, field_len));
break;
}
i_wanted += 1;
}
return newRV_noinc((SV *) ret);
}
END_C
;
而且,这里是 Perl 部分。请注意,我们
split
只需一次找出要保留的字段索引。一旦我们知道这些,我们就将行和(基于 1 的)索引传递给 C 例程以进行切片和切块。my @keep;
while (my $line = <>) {
unless (@keep) {
@keep = (2, 4, 5);
my @data = split ' ', $line;
push @keep, grep +(($_ - 5) % 3), 6 .. scalar(@data);
}
my $fields = extract_fields($line, \@keep);
print join(' ', @$fields), "\n";
}
$ time ./ww.pl input.data >/dev/null
真正的 0m11.539s
用户 0m11.083s
系统 0m0.300s
input.data
是使用以下方法生成的:$ perl -E 'say join("", "A".. "ZZZZ") for 1 .. 100' > input.data
它的大小约为 225MB。
关于perl - 尝试在 Perl 中使用 splice 删除特定列,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/18392406/