Mongodb Speed Decrease

I use mongodb to store compressed html files . Basically, a complete document of mongod is like: {'_id': 1, 'p1': data, 'p2': data2, 'p3': data3} where data, data1, data3 are :bso

Solution 1:

Document relocation could be an issue if you continue to add pages of html as new attributes. Would it really be an issue to move pages to a new collection where you could simply add them one record each? Also I don't really think MongoDB is a good fit for your use case. E.g. Redis would be much more efficient. Another thing you should take care of is to have enough ram for your _id index. Use db.mongocol.stats() to check the index size.

Solution 2:

When inserting new Documents into MongoDB, a Document can grow without moving it up to a certain point. Because the DB is analyzing the incoming Data and adds a padding to the Document. So do deal with less Document movements you can do two things:

1. manually tweaking the padding factor

2. preallocate space (attributes) for each document.

See Article about Padding or MongoDB Docs for more Information about the padding factor.

Btw. insetad of using save for creating new documents, you should use .insert() which will throw a duplicate key error if the _id is already there (.save() will overwrite your document)