sql - 为什么 SQL 数据库在命令日志上使用预写日志？

Question

我读到了 Voltdb 的 command log .命令日志记录事务调用，而不是像预写日志中那样记录每一行更改。通过仅记录调用，命令日志保持在最低限度，限制了磁盘 I/O 对性能的影响。

谁能解释一下为什么 Voltdb 使用命令日志以及为什么标准 SQL 数据库(例如 Postgres、MySQL、SQLServer、Oracle)使用预写日志背后的数据库理论？

Answer 1

我认为改写一下比较好:

Why does new distributed VoltDB use a command log over write-ahead log?

让我们做一个实验，假设您要编写自己的存储/数据库实现。毫无疑问，您已经足够先进，可以抽象文件系统并使用 block 存储以及一些额外的优化。

一些基本术语:

• 状态:在给定时间点存储的信息
• 命令:指示存储以改变其状态

因此您的数据库可能如下所示:

下一步是执行一些命令:

请注意几个重要方面:

1. 一个命令可能会影响很多存储的实体，所以很多 block 都会变脏
2. 下一个状态是当前状态和命令的函数

可以跳过一些中间状态，因为有一个命令链就足够了。

最后，您需要保证数据的完整性。

• Write-Ahead Logging - 核心概念是State 更改应该在对永久存储进行任何重大更新之前进行记录。按照我们的想法，我们可以记录每个 block 的增量更改。
• 命令日志记录 - 中心概念是仅记录用于生成状态的命令。

这两种方法各有利弊。 Write-Ahead log 包含所有更改的数据，Command log 将需要额外处理，但速度快且轻量级。

VoltDB: Command Logging and Recovery

The key to command logging is that it logs the invocations, not the consequences, of the transactions. By recording only the invocation, the command logs are kept to a bare minimum, limiting the impact the disk I/O will have on performance.

附加说明

SQLite: Write-Ahead Logging

The traditional rollback journal works by writing a copy of the original unchanged database content into a separate rollback journal file and then writing changes directly into the database file.

A COMMIT occurs when a special record indicating a commit is appended to the WAL. Thus a COMMIT can happen without ever writing to the original database, which allows readers to continue operating from the original unaltered database while changes are simultaneously being committed into the WAL.

PostgreSQL: Write-Ahead Logging (WAL)

Using WAL results in a significantly reduced number of disk writes, because only the log file needs to be flushed to disk to guarantee that a transaction is committed, rather than every data file changed by the transaction.

The log file is written sequentially, and so the cost of syncing the log is much less than the cost of flushing the data pages. This is especially true for servers handling many small transactions touching different parts of the data store. Furthermore, when the server is processing many small concurrent transactions, one fsync of the log file may suffice to commit many transactions.

结论

命令记录:

1. 更快
2. 占地面积小
3. 有更重的“重播”程序
4. 需要频繁的快照

Write Ahead Logging 是一种提供原子性的技术。更好的命令日志记录性能也应该改进事务处理。 Databases on 1 Foot

确认

VoltDB Blog: Intro to VoltDB Command Logging

One advantage of command logging over ARIES style logging is that a transaction can be logged before execution begins instead of executing the transaction and waiting for the log data to flush to disk. Another advantage is that the IO throughput necessary for a command log is bounded by the network used to relay commands and, in the case of Gig-E, this throughput can be satisfied by cheap commodity disks.

重要的是要记住 VoltDB 本质上是分布式的。因此处理事务有点棘手，性能影响也很明显。

VoltDB Blog: VoltDB’s New Command Logging Feature

The command log in VoltDB consists of stored procedure invocations and their parameters. A log is created at each node, and each log is replicated because all work is replicated to multiple nodes. This results in a replicated command log that can be de-duped at replay time. Because VoltDB transactions are strongly ordered, the command log contains ordering information as well. Thus the replay can occur in the exact order the original transactions ran in, with the full transaction isolation VoltDB offers. Since the invocations themselves are often smaller than the modified data, and can be logged before they are committed, this approach has a very modest effect on performance. This means VoltDB users can achieve the same kind of stratospheric performance numbers, with additional durability assurances.