迁移后mongodb数据文件变小

Question

在我的第一台服务器上，我得到:

root@prod ~ # du -hs /var/lib/mongodb/
909G    /var/lib/mongodb/

使用 mongodump/mongorestore 迁移 this 数据库后 在我的第二台服务器上，我得到:

root@prod ~ # du -hs /var/lib/mongodb/
30G /var/lib/mongodb/

等了几个小时后，mongo 完成了我得到的索引:

root@prod ~ # du -hs /var/lib/mongodb/
54G /var/lib/mongodb/

我测试了数据库，没有损坏或丢失的数据。

为什么迁移前后大小差异如此之大？

Answer 1

当由于数据删除以及其他原因导致实际数据大小下降时，MongoDB 不会恢复磁盘空间。在线文档中有一个不错的解释:

Why are the files in my data directory larger than the data in my database?

The data files in your data directory, which is the /data/db directory in default configurations, might be larger than the data set inserted into the database. Consider the following possible causes:

Preallocated data files.

In the data directory, MongoDB preallocates data files to a particular size, in part to prevent file system fragmentation. MongoDB names the first data file .0, the next .1, etc. The first file mongod allocates is 64 megabytes, the next 128 megabytes, and so on, up to 2 gigabytes, at which point all subsequent files are 2 gigabytes. The data files include files with allocated space but that hold no data. mongod may allocate a 1 gigabyte data file that may be 90% empty. For most larger databases, unused allocated space is small compared to the database.

On Unix-like systems, mongod preallocates an additional data file and initializes the disk space to 0. Preallocating data files in the background prevents significant delays when a new database file is next allocated.

You can disable preallocation by setting preallocDataFiles to false. However do not disable preallocDataFiles for production environments: only use preallocDataFiles for testing and with small data sets where you frequently drop databases.

On Linux systems you can use hdparm to get an idea of how costly allocation might be:

time hdparm --fallocate $((1024*1024)) testfile

The oplog.

If this mongod is a member of a replica set, the data directory includes the oplog.rs file, which is a preallocated capped collection in the local database. The default allocation is approximately 5% of disk space on 64-bit installations, see Oplog Sizing for more information. In most cases, you should not need to resize the oplog. However, if you do, see Change the Size of the Oplog.

The journal.

The data directory contains the journal files, which store write operations on disk prior to MongoDB applying them to databases. See Journaling Mechanics.

Empty records.

MongoDB maintains lists of empty records in data files when deleting documents and collections. MongoDB can reuse this space, but will never return this space to the operating system.

To de-fragment allocated storage, use compact, which de-fragments allocated space. By de-fragmenting storage, MongoDB can effectively use the allocated space. compact requires up to 2 gigabytes of extra disk space to run. Do not use compact if you are critically low on disk space.

Important

compact only removes fragmentation from MongoDB data files and does not return any disk space to the operating system.

To reclaim deleted space, use repairDatabase, which rebuilds the database which de-fragments the storage and may release space to the operating system. repairDatabase requires up to 2 gigabytes of extra disk space to run. Do not use repairDatabase if you are critically low on disk space.

http://docs.mongodb.org/manual/faq/storage/

他们没有告诉您的是恢复/恢复磁盘空间的另外两种方法 - mongodump/mongorestore 就像您所做的那样，或者将新成员添加到具有空磁盘的副本集中，以便它从头开始写入它的数据库文件.

如果您对此感兴趣，db.stats() 命令会返回有关数据、索引、存储和文件大小的大量数据:

http://docs.mongodb.org/manual/reference/command/dbStats/