java - FlatMapFunction 没有定义serialversionUid

Question

@Component("loader")
    @Slf4j
    public class LoaderSpark implements Serializable , SparkJob {
            private static final int MAX_VERSIONS = 1;
            private static final long serialVersionUID = 5911491576792653184L;
           spark.sparkContext().getConf().toDebugString());
          HBaseUtil hBaseUtil = new HBaseUtil(props);

        byte[][] prefixes = new byte[][]{toBytes("document"),
                toBytes("dataSource"),
                toBytes("hold:")};

        Filter filter = new MultipleColumnPrefixFilter(prefixes);

        Scan scan = new Scan();
        scan.addFamily(toBytes("data"));        
        scan.setFilter(filter);

    @Override
    public void run(final SparkSession spark, final String... args) {

        JavaPairRDD<ImmutableBytesWritable, Result> ScanRDD
                = hBaseUtil.createScanRdd(spark, "Test", scan);

        ScanRDD.mapToPair(new FlatMapFunction<Tuple2<ImmutableBytesWritable,Result>, String, String>() {


            @Override
            public Iterator<String> call(Tuple2<ImmutableBytesWritable, Result> immutableBytesWritableResultTuple2) throws Exception {
                return null;
            }

        });

我的ScanRDD.mapToPair函数提示serialversionUid丢失并要求在ScanRDD.mapToPair(new FlatMapFunction<Tuple2<ImmutableBytesWritable,Result>, String, String>() {内添加一个

我想知道为什么需要将serialversionUid 添加到我的FlatMapFunction 中？我已经为类(class)准备了一份。

有什么帮助吗？

Answer 1

其背后的原因只是一个静态分析，验证实现 Serialized 的类的每个实例是否具有 serialVersionUID 字段。

在您的例子中，FlatMapFunction 实现了 java.io.Serialized 并且您正在创建它的匿名类。

所以只需添加serialVersionUID

ScanRDD.mapToPair(new FlatMapFunction<Tuple2<ImmutableBytesWritable,Result>, String, String>() {
//inside the anonymous class
private final serialVersionUID = 1L;

@Override
public Iterator<String> call(Tuple2<ImmutableBytesWritable, Result> immutableBytesWritableResultTuple2) throws Exception {
                return null;
}

})

参见https://spark.apache.org/docs/2.2.2/api/java/org/apache/spark/api/java/function/FlatMapFunction.html