Encrypting a TinyDB storage

TinyDB is a pure-python, document oriented database. It neither provides the performance nor the features of other databases, but if you need a simple database for your projects to store some values, then TinyDB is a reasonable choice. It’s incredibly easy to use and it comes with very little overhead. The data typically gets stored in a file, however, you can also write your own storage adapters to store the data differently.

One thing that bugged me is that typically this file is just a plain text file lying around on your computer to be read. Thus, a while ago I decided to implement a storage layer that features password protection that encrypts the file. This blog post introduces the library (sources), which has been released under MIT license.

Implementation

The implementation of the encrypted JSON storage is pretty straight forward, as the TinyDB storage API only requires you to implement four methods: init, read, write, and close. Each of these four methods has does what the name indicates - init sets up the storage, read loads the data from the file into the memory, write dumps the data from memory to the file and close tears down the storage layer, e.g. closes file pointers and such. In addition to that, I added a method for changing the password.

Init Initializing encrypted storage takes care for the setup. I use AES encryption with 256 bit keys, cipher block chaining (CBC) and a random initialization vector (IV). The IV will be stored at the beginning of the begin of the encrypted database file, alongside the length of the file. The IV will be loaded from the database if it does not exist, and created otherwise. To ensure the key has a length of 256 bit, I hash it using SHA256, which is at the moment of writing a NIST-approved secure hash algorithm.

Read Reading the encrypted data is pretty straight forward. TinyDB requires the read function to pass all the database’s data, so to implement the read functionality you need to decrypt the database, read it an pass it on.

Write Writing the encrypted storing is a bit more problematic than reading, because if anything happens during the write process (e.g. an exception, or a power outage), the whole database would be corrupted. With that in mind, I’ve designed the write method such that it creates a backup of the encrypted storage before it starts to write, which can be used for recovery in case of unforseen events.

Close Closing the encrypted database only requires to close the file handle.

Changing the password Changing the password is – similarly to writing the data – a bit problematic for the case when something unexpected happens. That is why I first create a new database which copies all the data from the old database to the new one, and once that has successfully finished I replace the old database with the new one. Finally, I reinitialize the database with the new encryption key.

Usage and requirements

TinyDB is built on top of PyCryptodom and TinyDB, which require Python 3.4 and Python 3.5, respectively, which means that you need to support Python 3.5 to use this plugin.

Installation You can install the plugin using the python package manager pip, simply by executing the following command or adding it to your requirements file.

pip install tinydb-encrypted-jsonstorage

Creating a database To use the encrypted JSON storage, you need to set the storage argument of the TinyDB constructor to EncryptedJSONStorage and specify an encryption key.

from tinydb import TinyDB
import tinydb_encrypted_jsonstorage as tae
KEY = "hello"
PATH = ".encrypted_db"
db = TinyDB(encryption_key=KEY, path=PATH, storage=tae.EncryptedJSONStorage)

Changing the encryption key For changing the encryption key, you simply need to access the storage property of your database and call the change_encryption_key method.

db = ...
db.storage.change_encryption_key("NEW_KEY"))

Conclusion

TinyDB is a python database that allows you to add hassle-free persistence to small projects. Here, I introduced a storage plugin for TinyDB that allows you to store your data as encrypted JSON. It can be installed via pip, and using it requires only a few lines of code.