MongoDB Won't Start: Fix 'IllegalOperation: unsupported format characteristic: data'

You restarted the server after a power flicker, and now MongoDB refuses to come up with IllegalOperation: unsupported format characteristic: data. Annoying, but fixable.

The error almost always points to a corrupted WiredTiger metadata file, usually WiredTiger.wt inside your data directory. The unclean shutdown left that file half-written, and now mongod can't parse it.

Stop MongoDB first

Make sure mongod isn't running. If it's stuck in a crash loop, kill it:

sudo systemctl stop mongod
sudo pkill -9 mongod

Back up your data directory

You're about to touch files that hold your actual data. Copy the whole dbPath somewhere safe before you do anything else. If your dbPath is /var/lib/mongodb:

sudo cp -a /var/lib/mongodb /var/lib/mongodb.backup.$(date +%F)

Don't skip this. If the repair goes sideways, this backup is the only thing between you and a very bad day.

Run WiredTiger salvage

MongoDB ships a WiredTiger salvage tool that can rebuild the metadata from the raw data files. Run it against your dbPath:

sudo -u mongodb wt -h /var/lib/mongodb salvage

On Ubuntu/Debian the binary is usually /usr/bin/wt. On RHEL-based systems it lives in the MongoDB server package too. If you get command not found, find it with:

find / -name wt -type f 2>/dev/null

You should see output ending with salvage: successfully salvaged or similar. If it complains about a specific file, note the filename — that's your corrupted one.

If salvage doesn't help, delete the damaged metadata file

Salvage sometimes can't fix WiredTiger.wt itself. In that case, move it out of the way and let mongod rebuild it from the .wt data files:

sudo mv /var/lib/mongodb/WiredTiger.wt /var/lib/mongodb/WiredTiger.wt.broken
sudo chown -R mongodb:mongodb /var/lib/mongodb

Now start mongod with --repair once:

sudo -u mongodb mongod --dbpath /var/lib/mongodb --repair

Wait for it to finish. It can take a while on large datasets — you'll see progress lines scrolling by. When it exits cleanly, start the service normally:

sudo systemctl start mongod
sudo systemctl status mongod

After this, you should see waiting for connections on port 27017 in the log. Open a shell and confirm your databases are intact:

mongosh --eval 'db.adminCommand({listDatabases:1})'

Why this works

WiredTiger stores two kinds of files: .wt files that hold the actual collection and index data, and WiredTiger.wt, which is the metadata catalog describing what's in those files. During a clean shutdown, MongoDB flushes the metadata in one atomic step. During a crash, that flush can be interrupted mid-write.

The data files themselves are usually fine — WiredTiger writes them with checksums and checkpoints, so partial writes get caught. The metadata file is the fragile one. When mongod reads it and finds bytes that don't match the expected format, it bails with IllegalOperation: unsupported format characteristic: data.

Salvage rebuilds the metadata by scanning the data files themselves. Deleting WiredTiger.wt and running --repair does the same thing from scratch: mongod walks every .wt file and reconstructs the catalog. As long as the underlying data files survived the crash, your collections come back.

Variations you might hit

Error mentions a specific collection file

If the error names something like collection-12-3456789.wt instead of WiredTiger.wt, that individual file is bad. You've got two options:

1. Run salvage first — it can often rebuild a single file.
2. If salvage fails, move only that file aside. You'll lose the collection it belongs to, but the rest of the database comes up. You can then restore that collection from a backup.

Error appears during replica set member startup

On a secondary, you don't need to repair at all. Wipe the data directory and resync from the primary:

sudo systemctl stop mongod
sudo rm -rf /var/lib/mongodb/*
sudo systemctl start mongod

mongod will initial sync from the primary automatically. This is faster and safer than repairing a secondary.

Error on mongos or config server

Config servers hold critical cluster metadata. Don't try to repair them blindly. Restore from a config server backup, or if you have three config servers, wipe the bad one and let it resync from the other two.

All WiredTiger tools are missing

Some minimal installs don't include the wt binary. Install mongodb-org-server for your version, or download the server tarball and pull wt out of it. Don't try to fix corruption with random third-party tools.

Preventing this next time

You can't stop a server from losing power. You can make the recovery painless.

• Use a journal. WiredTiger journaling is on by default in MongoDB 5.0+. Don't disable it to save disk space — that's exactly what turns a crash into corruption.
• Set storage.wiredTiger.engineConfig.journalCompressor. Snappy is fine. Skip zlib unless you're tight on disk — the CPU cost on every write isn't worth it.
• Run a replica set. Even a single-node replica set is better than standalone. You get automatic failover, oplog-based recovery, and secondary resync if a node dies.
• Back up nightly. mongodump to remote storage. Restoring from a backup is a 30-minute job. Recovering from a bad WiredTiger.wt without one is an afternoon you'll never get back.
• Monitor disk health. That 'unclean shutdown' might not have been a crash at all. Failing SSDs produce exactly this kind of silent corruption on writes. Check SMART data regularly.
• Use enterprise storage with battery-backed cache. If you're on cloud VMs, pick volumes with consistent write ordering. Cheap ephemeral disks are a known source of this error.

Fix the metadata, verify your collections, and get a backup running before you close the terminal. The next crash might not be so forgiving.