Issue 328: recollindex freezes the desktop

After upgrading from Ubuntu 16.04 to 16.10 I found that the desktop would freeze at intervals while recollindex was running. This was noticed while I was re-organizing my pdf library and moving around large pdf files. It made all desktop GUI operations unusable. Work in the terminal was unaffected.

The gkrellm disk monitor showed continuous writes to the home disk at 1 Mb per sec.

I did a thorough surface check of the disk using e2fsck and it reported no errors. The Ubuntu disks utility also reported no errors.

I re-installed recoll and moved the .recoll folder, holding the xapian indexes, to a second, spare disk. This completely solved the problem. recollindex now runs with no perceptible effect on the GUI desktop.

The only way that the recoll indexer can really disturb a system is by consuming memory. Most sources of problems have been found and fixed, but I can’t exclude that the indexer might sometimes become huge, forcing the system to swap, with very noticeable effects on interactivity.

You should check the process sizes the next time this kind of thing happens (recollindex itself and the python and pdftotext subprocesses).

Both the indexer’s I/O operations and CPU usage are set at minimum priority, so any issue on this side has to come from the system: kernel scheduler or hardware. 1 MB/s is a very modest amount for a modern disk (we’d need the transaction frequency too for completeness).

It is also possible to lower the system load by disabling multithreading in the indexer (See: https://www.lesbonscomptes.com/recoll/usermanual/webhelp/docs/RCL.INDEXING.CONFIG.THREADS.html)

Thank you for reporting this anyway, but there is not much I can do at this point.