My Crawl Keeps Failing with a Checksum Error

Issue:

My crawls have stopped working and I have no idea why. After restarting Fusion, the crawl still does not work. In the logs I notice the following error:

2018-03-27T00:09:16,332 - ERROR [MapDb open thread:ApolloGuiceContextListener@41] - {} - Uncaught exception in thread 'Thread[MapDb open thread,5,main]':

java.io.IOError: java.io.IOException: Wrong index checksum, store was not closed properly and could be corrupted.

       at org.mapdb.StoreDirect.checkHeaders(StoreDirect.java:269) ~[mapdb-1.0.9.jar:?]

       at org.mapdb.StoreDirect.<init>(StoreDirect.java:207) ~[mapdb-1.0.9.jar:?]

       at org.mapdb.DBMaker.extendStoreDirect(DBMaker.java:971) ~[mapdb-1.0.9.jar:?]

       at org.mapdb.DBMaker.makeEngine(DBMaker.java:758) ~[mapdb-1.0.9.jar:?]

       at org.mapdb.DBMaker.make(DBMaker.java:701) ~[mapdb-1.0.9.jar:?]

       at com.lucidworks.connectors.db.impl.MapDbConnectorDb$1.run(MapDbConnectorDb.java:56) ~[lucidworks-shared-3.1.0.jar:?]

Caused by: java.io.IOException: Wrong index checksum, store was not closed properly and could be corrupted.

       ... 6 more

Environment:

Fusion

Resolution:

The only way to recover from this issue, is the following:

1. Navigate to the crawldb for the affected collection in the error.  In Fusion 3.1.x, you’d locate those files here: FUSION$/3.1.x/data/connectors/crawldb/lucid.web/<datasource-Name>
2. Delete only those for the affected collection.
3. Re-run your datasource.

Cause: