apache - 无法运行 giraph SimpleInDegreeCountComputation

Question

我正在尝试运行 Giraph 中包含的 SimpleInDegreeCountComputation 示例。我的做法如下:

SimpleInDegreeCountComputation.java:

    public class SimpleInDegreeCountComputation extends BasicComputation
              <LongWritable, LongWritable, DoubleWritable, DoubleWritable> {
    .......

然后我尝试像这样运行它:

    hadoop jar /path-to-giraph-folder/giraph-examples/target/giraph-examples-1.1.0- 
    SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar 
    org.apache.giraph.GiraphRunner  
    org.apache.giraph.examples.SimpleInDegreeCountComputation 
    -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat 
    -vip /path-to-input-file 
    -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat 
    -op /path-to-output-file -w 1 

结果如下:

    14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge input format specified. 
    Ensure your InputFormat does not require one.
    14/05/18 18:58:40 INFO utils.ConfigurationUtils: No edge output format specified.  
    Ensure your OutputFormat does not require one.
    Exception in thread "main" java.lang.IllegalArgumentException: checkClassTypes: vertex  
    value types not assignable, computation - class org.apache.hadoop.io.LongWritable,   
    VertexInputFormat - class org.apache.hadoop.io.DoubleWritable
at org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
at org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

我不太确定我做错了什么。如果有人能指出我正确的方向，或者链接到一个资源来解释我正在尝试做的事情的更简单方法，我将不胜感激!我认为问题可能是格式错误 (-vif)。 我使用的输入文件如下:

    [0,0,[[1,5],[2,9]]]
    [1,0,[[0,5],[3,3]]]
    [2,0,[[0,9],[3,3],[4,3]]]
    [3,0,[[1,3],[2,3],[4,2]]]
    [4,0,[[2,3],[3,3]]]

Answer 1

查看计算和顶点输入类的定义，似乎 JsonLongDoubleFloatDoubleVertexInputFormat 与 SimpleInDegreeCountComputation 不兼容

SimpleInDegreeCountComputation :

public class SimpleInDegreeCountComputation extends BasicComputation<
    LongWritable, LongWritable, DoubleWritable, DoubleWritable> {

BasicComputation :

/**
 * Computation in which both incoming and outgoing message types are the same.
 *
 * @param <I> Vertex id
 * @param <V> Vertex data
 * @param <E> Edge data
 * @param <M> Message type
 */
public abstract class BasicComputation<I extends WritableComparable,
    V extends Writable, E extends Writable, M extends Writable>
    extends AbstractComputation<I, V, E, M, M> {
}

你可以看到:

• 顶点 id 的类型为 LongWritable
• 顶点数据的类型是LongWritable
• 边缘数据类型为 DoubleWritable

...另一方面，您尝试使用的 InputFormat...

JsonLongDoubleFloatDoubleVertexInputFormat :

public class JsonLongDoubleFloatDoubleVertexInputFormat extends
    TextVertexInputFormat<LongWritable, DoubleWritable, FloatWritable> {

TextVertexInputFormat :

/**
 * Abstract class that users should subclass to use their own text based
 * vertex input format.
 *
 * @param <I> Vertex index value
 * @param <V> Vertex value
 * @param <E> Edge value
 */
@SuppressWarnings("rawtypes")
public abstract class TextVertexInputFormat<I extends WritableComparable,
    V extends Writable, E extends Writable>
    extends VertexInputFormat<I, V, E> {

你可以看到:

• 顶点 id 的类型为 LongWritable
• 顶点数据的类型是DoubleWritable
• 边缘数据类型为FloatWritable

---

因为它是 LongWritable、DoubleWritable 和 FloatWritable 而不是 Long、Double和 Float - 这些类型无法自动转换。

我找不到您可以使用的任何 InputFormat，因此您需要修改现有的 JsonLongDoubleFloatDoubleVertexInputFormat 或修改算法以对 Edge 使用 NullWritable数据类型。我在任何地方都看不到要使用的边缘数据，因此它也可以为空。在这种情况下，您可以使用 LongLongNullTextInputFormat .