java - 在java中,如何将一个对象写入文件,然后从文件中读取它并将其转换回HDFS中的原始对象?

标签 java hdfs

我正在尝试将 double[] 写入 HDFS 中的文件中,稍后,我需要从文件中读回它并将其转换回 double[]。这里有人知道怎么做吗?

谢谢

最佳答案

<强> ObjectOutputStream

An ObjectOutputStream writes primitive data types and graphs of Java objects to an OutputStream. The objects can be read (reconstituted) using an ObjectInputStream. Persistent storage of objects can be accomplished by using a file for the stream. If the stream is a network socket stream, the objects can be reconstituted on another host or in another process.

Only objects that support the java.io.Serializable interface can be written to streams. The class of each serializable object is encoded including the class name and signature of the class, the values of the object's fields and arrays, and the closure of any other objects referenced from the initial objects.

The method writeObject is used to write an object to the stream. Any object, including Strings and arrays, is written with writeObject. Multiple objects or primitives can be written to the stream. The objects must be read back from the corresponding ObjectInputstream with the same types and in the same order as they were written.

Primitive data types can also be written to the stream using the appropriate methods from DataOutput. Strings can also be written using the writeUTF method.

<强> ObjectInputStream

An ObjectInputStream deserializes primitive data and objects previously written using an ObjectOutputStream.

ObjectOutputStream and ObjectInputStream can provide an application with persistent storage for graphs of objects when used with a FileOutputStream and FileInputStream respectively. ObjectInputStream is used to recover those objects previously serialized. Other uses include passing objects between hosts using a socket stream or for marshaling and unmarshaling arguments and parameters in a remote communication system.

ObjectInputStream ensures that the types of all objects in the graph created from the stream match the classes present in the Java Virtual Machine. Classes are loaded as required using the standard mechanisms.

例如,我可以保存 key 生成器以这种方式加密数据:

public static void saveKeyToFile(SecretKey key)
        throws FileNotFoundException, IOException {
    ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(
            "/path/to/mysavedobject"));
    oos.writeObject(key);
    oos.close();
}

public static SecretKey getKeyFromFile(String dir) throws IOException,
        ClassNotFoundException {
    if (dir == null) { 
        dir = "/path/to/mysavedobject";
    }
    SecretKey key = null;
    ObjectInputStream ois = new ObjectInputStream(new FileInputStream(
            dir));
    key = (SecretKey) ois.readObject();
    ois.close();
    return key;
}

关于java - 在java中,如何将一个对象写入文件,然后从文件中读取它并将其转换回HDFS中的原始对象?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/9946590/

相关文章:

java - 为什么 Selenium Server 找不到定位器

java - 使用没有端点的处理器

java - 如何基于XSD "Auto-Repair"XML?

hadoop - HDFS 目录的 "t"权限是什么?

Java Android 比较字符串格式的时间

java.lang.NoSuchMethodError : org. hibernate.ogm.cfg.OgmConfiguration.addAnnotatedClass

python - 直接将数据流式传输到 hdfs 中,无需复制

amazon-web-services - HdfsRpcException:无法在服务器上调用RPC调用 “getFsStats”

hadoop - NameNode 地址的 URI 无效,s3a 不是模式 'hdfs'

java - 在 hadoop HDFS 中存储大文件?