Is the database size larger than about 950 MB? If you are on a 32bit system (which most Raspberry Pis are), that limits the DB size. You might need to run database compaction. Try running chantools compactdb. If that results in the same error, you need to copy the DB to another host (64bit machine) and run the command there, then copy the DB back. This is a bit risky though so feel free to contact me on Slack if you're not sure what to do exactly.