我正在按照the mahout site上的说明将现有文件转换为序列文件:
VectorWriter vectorWriter = SequenceFile.createWriter(filesystem,
configuration,
outfile,
LongWritable.class,
SparseVector.class);
long numDocs = vectorWriter.write(new VectorIterable(), Long.MAX_VALUE);
我已经将mahout jar包含在我的Maven项目中:
<dependency>
<groupId>org.apache.mahout</groupId>
<artifactId>mahout-core</artifactId>
<version>0.9</version>
</dependency>
但是它不会写文件。
我收到此错误:
Caused by: java.lang.NullPointerException
at org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73)
at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:963)
at org.apache.hadoop.io.SequenceFile$RecordCompressWriter.<init>(SequenceFile.java:1136)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:397)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:284)
at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:265)
经过进一步调查,其原因是:
Serilization class not found: java.lang.ClassNotFoundException: org.apache.hadoop.io.serializer.WritableSerialization
这向我暗示我想念一个 jar -有人知道哪个吗?
最佳答案
问题是我在Lenskit中使用了它,并且Configuration
类尝试使用Thread.currentThread().getContextClassLoader()
(无论出于何种原因)没有mahout或hadoop软件包。完整的代码是:
final Configuration configuration = new Configuration();
configuration.setClassLoader(Configuration.class.getClassLoader());
final Path path = new Path(POINTS_PATH + "/pointsFile");
LocalFileSystem fs = new LocalFileSystem();
fs.initialize(URI.create(POINTS_PATH + "/pointsFile"), configuration);
final SequenceFile.Writer writer =
SequenceFile.createWriter(
fs,
configuration,
path,
Text.class,
VectorWritable.class);
关于java - Mahout:缺少创建序列文件的类,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/28988792/