c++ - 使用 C++ fstream 进行序列化

Question

我尝试使用 fstream 进行序列化。流语法是: “IndexLengthDataIndexLengthData...”。例如，11c22cc33ccc。读文件时，输入流会把“11”作为一个整体读入索引。

索引在 [1, INT_MAX] 范围内。长度限制在 516 以内。

我可以在不使用分隔符的情况下执行此操作吗？例如，在索引和长度之间使用“@”或“#”？

int main() {
  std::ofstream ofs;
  ofs.open("myfile.txt", std::ofstream::out | std::ofstream::trunc);
  for(int i = 1; i <= 10; ++i) {
    ofs << i; // for index
    ofs << i; // for length
    for (int j = 0; j < i; ++j) ofs << 'c';
  }
  ofs.close();
  std::ifstream ifs;
  ifs.open("myfile.txt", std::ifstream::in);
  for (int i = 0; !ifs.eof() && ifs.good(); ++i) {
    int index = 0, length = 0;
    ifs >> index;
    ifs >> length;
    std::cout << "index is " << index << "length is " << length << std::endl;
    // Jump to the next entry
    ifs.seekg(length, std::ios_base::cur);
  }
}

Answer 1

是的，如果您有固定大小的格式，那么索引为 10 个字符，长度为 3 个字符，您的示例将被编码为:
“1 1c 2 2cc 3 3ccc”。

您还谈到了 fstream，但看起来您正在追求一种文本(人类可读)序列化，而不是二进制序列化。如果是这种情况，但您不需要真正的人类可读形式，您可以用一些位标记长度的第一个字节(ASCII 中的数字编码为 0x30 到 0x39 值，因此您可以在不破坏数据字节的情况下设置 0x40 位。那么您的示例如下所示:
1qc2rcc3sccc (q = 0x71 = 0x40|0x31 = 0x40|'1' )

对于一些更长的值，它看起来像:113q00123456789 ... ARGH 我想序列化 10 个字符长的字符串“0123456789”，看看发生了什么，我得到了长度 100 而不是 10 (或者更糟的是 100123456789，如果你不限制的话)，所以长度的开始和结束都必须以某种方式被污染，可能使用 bit 0x80 标记长度结束。
1\361c2\362cc3\363ccc(\361 = 0xF1 = 0x40|0x80|0x31 = 0x40|0x80|'1')

第二次尝试更长的值:
113q°0123456789(索引113，长度10，数据“0123456789”，q = 0x40|'1'，° = 0x80|'0').

你不想要二进制形式吗？会更短。

---

顺便说一句，如果你不介意污染值，但你想保留在 7 位 ASCII 中，你可以污染不是长度的开始和结束，而是索引和长度的结束，并且只能使用 0x40。所以 11c 会变成 qqc。而 113 10 0123456789 将是 11s1p0123456789。

---

二进制写入/读取与平台不可知字节序(即在小字节序上写入的文件将在其他平台上使用大字节序)。

#include <iostream>
#include <cstdint>
#include <vector>

/**
 * Writes index+length+data in binary form to "out" stream.
 * 
 * Returns number of bytes written to out stream.
 * 
 * Does no data validation (the variable types are only limits for input data).
 * 
 * writeData and readData are done in endiannes agnostic way.
 * So file saved at big-endian platform will be restored correctly on little-endian platform.
 **/
size_t writeData(std::ostream & out,
        const uint32_t index, const uint16_t length, const uint8_t *data) {
    // Write index and length bytes to out stream, resolve endiannes of host platform.
    out.put((char)((index>>0)&0xFF));
    out.put((char)((index>>8)&0xFF));
    out.put((char)((index>>16)&0xFF));
    out.put((char)((index>>24)&0xFF));
    out.put((char)((length>>0)&0xFF));
    out.put((char)((length>>8)&0xFF));
    // If any data, write them to stream
    if (0 < length) out.write(reinterpret_cast<const char *>(data), length);
    return 4 + 2 + length;
}

/**
 * Read data from stream "in" stream into variables index, length and data.
 * 
 * If "in" doesn't contain enough bytes for index+length, zero index/length is returned
 * 
 * If "in" contains more than index+length bytes, but the data are shorter than length,
 * then "repaired" shorter data are returned with shorter "length" (not the read one).
 **/
void readData(std::istream & in,
        uint32_t & index, uint16_t & length, std::vector<uint8_t> & data) {
    // clear current values in index, length, data
    index = length = 0; data.clear();
    // read index+length header from stream
    uint8_t buffer[6];
    in.read(reinterpret_cast<char *>(buffer), 6);
    if (6 != in.gcount()) return;   // header data (index+legth) not found
    // Reassemble read bytes together to index/length numbers in host endiannes.
    index = (buffer[0]<<0) | (buffer[1]<<8) | (buffer[2]<<16) | (buffer[3]<<24);
    length = (buffer[4]<<0) | (buffer[5]<<8);
    if (0 == length) return;    // zero length, nothing more to read
    // Read the binary data of expected length
    data.resize(length);  // reserve memory for read
    in.read(reinterpret_cast<char *>(data.data()), length);
    if (length != in.gcount()) {    // data read didn't have expected length, damaged file?
        // TODO you may want to handle damaged data in other way, like returning index 0
        // This code will simply accept shorter data, and "repair" length
        length = in.gcount();
        data.resize(length);
    }
}

要查看实际效果，您可以在 cpp.sh 上试用。