How to restore data from MapDB wal (write ahead log) files?

Question

My objective is to enable MapDB crash protection and if a crash happens, how to restore data.

I have enabled transaction for MapDB

DB db;
HTreeMap< String, String > mapDb;

db = DBMaker.fileDB( "walTest/file1.db" )
        .transactionEnable()
        .allocateStartSize( 64 *1024 *1024 )
        .allocateIncrement( 32 *1024 *1024 )
        .fileMmapEnable()
        .fileMmapEnableIfSupported()
        .fileMmapPreclearDisable()
        .cleanerHackEnable()
        .closeOnJvmShutdown()
        .make();

mapDb = db.hashMap( "Test" ) //$NON-NLS-1$
        .keySerializer( Serializer.STRING )
        .valueSerializer( Serializer.STRING )
        .createOrOpen();

for(int i=0; i<10000;i++)
{
    mapDb.put(""+i, "aaaaaaaaaaa bbbbbbbbbbbbbbbbbb cccccccccccccccccccc dddddddddddddddddd");
    if(i % 100 == 0) {
        try {
            Thread.sleep(100);
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
    if(i == 5000)
    {
        db.commit();
    }
}

I see that it creates 2 files, file1.db and file1.db.wal.0

And now, I restart the server and read the MapDB using below code

System.out.println(mapDb.size());

Output is 5001 The other 5000 which was not committed is not getting restored.

I tried using db.commit before doing mapDb.size() and its still not helping.

So, how is the wal file protecting against server or jvm crashes? How can we restore the wal files?

Answer 1

As of the latest release (3.0.8) I don't think you can.

Looking through the source code it looks there's some code that should do this but it seems to be a work in progress feature, that isn't used internally and also with no external way to trigger the code.

For some added details on what I've found: With .transactionEnable() MapDB will trigger the creation of the WAL file file1.db.wal.0, though it only uses this to allow for .rollback() while the application is running. On the next start up, even though the WAL file exists, it ignores the contents, and immediately starts writing into the file as if it were newly created and empty.

.transactionEnable() will set MapDB's store class to be StoreWAL which has a class variable wal, which is an instance of WriteAheadLog. WriteAheadLog has a subclass WALReplay as well as an functions open and replayWAL which take a WALReplay instance as a parameter but I wouldn't trust the code is production ready currently.

If you really want to replay the WAL files then it could be possible by:

1. Using reflection to get the wal variable from db.getStore()
2. Creating an implementation of the WALReplay interface
3. Call replayWAL or (preferably) open on the wal variable, passing in an instance of your implementation of WALReplay.