ruby - 在 UTF-8 中提取无效字节序列

Question

突然间，我在运行 rails 3.2 和 Ruby 1.9.3p125 的网络服务器上收到任何 rake 命令的奇怪错误，无论什么 rake 任务，堆栈跟踪都是相同的。 Rakefile 和 lib/tasks 中只存在 ascii 中的内容。

堆栈跟踪:

rake --trace
rake aborted!
invalid byte sequence in UTF-8
/usr/local/lib/ruby/1.9.1/rake/application.rb:183:in `glob'
/usr/local/lib/ruby/1.9.1/rake/application.rb:183:in `block in have_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:181:in `each'
/usr/local/lib/ruby/1.9.1/rake/application.rb:181:in `have_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:468:in `find_rakefile_location'
/usr/local/lib/ruby/1.9.1/rake/application.rb:486:in `raw_load_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:82:in `block in load_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:133:in `standard_exception_handling'
/usr/local/lib/ruby/1.9.1/rake/application.rb:81:in `load_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:65:in `block in run'
/usr/local/lib/ruby/1.9.1/rake/application.rb:133:in `standard_exception_handling'
/usr/local/lib/ruby/1.9.1/rake/application.rb:63:in `run'
/usr/local/bin/rake:32:in `<main>'

违规的方法是

def have_rakefile
      @rakefiles.each do |fn|
        if File.exist?(fn)
          others = Dir.glob(fn, File::FNM_CASEFOLD)
          return others.size == 1 ? others.first : fn
        elsif fn == ''
          return fn
        end
      end
      return nil
    end

由于堆栈跟踪对我没有帮助，所以我在 block 的开头插入了一个 puts "#{fn} #{File::FNM_CASEFOLD}" 并得到了这个:

rakefile 8
Rakefile 8
rake aborted!
invalid byte sequence in UTF-8
/usr/local/lib/ruby/1.9.1/rake/application.rb:184:in `glob'
/usr/local/lib/ruby/1.9.1/rake/application.rb:184:in `block in have_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:181:in `each'
/usr/local/lib/ruby/1.9.1/rake/application.rb:181:in `have_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:469:in `find_rakefile_location'
/usr/local/lib/ruby/1.9.1/rake/application.rb:487:in `raw_load_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:82:in `block in load_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:133:in `standard_exception_handling'
/usr/local/lib/ruby/1.9.1/rake/application.rb:81:in `load_rakefile'
/usr/local/lib/ruby/1.9.1/rake/application.rb:65:in `block in run'
/usr/local/lib/ruby/1.9.1/rake/application.rb:133:in `standard_exception_handling'
/usr/local/lib/ruby/1.9.1/rake/application.rb:63:in `run'
/usr/local/bin/rake:32:in `<main>'

rakefile 只是 rails 生成的默认文件

# Add your own tasks in files placed in lib/tasks ending in .rake,
# for example lib/tasks/capistrano.rake, and they will automatically be available to Rake.

require File.expand_path('../config/application', __FILE__)
require 'rake/dsl_definition'
require 'rake'

MyApp::Application.load_tasks

lib/tasks 中唯一的任务文件是

 desc "Resets the help files in the db by deleting all existing and rereading the yaml files"
    task :help_reset => :environment do
      HelpSystem.delete_all
      HelpSystem.seed_help
    end

我不知道下一步该去哪里，非常感谢任何帮助。

Answer 1

好的，我的问题与您的略有不同，但我会发布我是如何解决它的，以防它对 future 的 Google 员工有所帮助。

我的问题是每次尝试运行 rake stats 时都会收到以下错误:

rake aborted!
ArgumentError: invalid byte sequence in UTF-8
/Users/george/.rvm/gems/ruby-2.1.5/gems/railties-4.1.6/lib/rails/code_statistics_calculator.rb:61:in `=~'
/Users/george/.rvm/gems/ruby-2.1.5/gems/railties-4.1.6/lib/rails/code_statistics_calculator.rb:61:in `add_by_io'
/Users/george/.rvm/gems/ruby-2.1.5/gems/railties-4.1.6/lib/rails/code_statistics_calculator.rb:43:in `block in add_by_file_path'
... # more stacktrace

所以我打开了 code_statistics_calculator.rb(堆栈跟踪顶部的文件并更改了:

def add_by_file_path(file_path)
  File.open(file_path) do |f|
    self.add_by_io(f, file_type(file_path)) # <- this line is raising the error
  end
end

到:

def add_by_file_path(file_path)
  File.open(file_path) do |f|
    begin
      self.add_by_io(f, file_type(file_path))
    rescue ArgumentError
      debugger
      puts # An extra statement is needed between 'debugger' and 'end' or debugger screws up.
    end
  end
end

再次运行rake stats，我进入调试器，此时我可以看到file_path此时指向app/models中的一个特定文件 它无法解析为 utf-8。

果然，我在 vim 中打开了那个文件，当我输入 :set fileencoding? 时，它返回了 latin-1。所以我将它设置为 utf-8(set fileencoding=utf-8 然后保存文件)果然，rake stats 再次起作用了!瞧。

(请注意，在您的情况下，可能有多个文件不是 utf-8 格式。另外，当您完成后，请确保您不要忘记将 code_statistics_calculator.rb 改回来恢复原状!)