Indexing failures

Indexing may fail for some documents, for a number of reasons: a helper program may be missing, the document may be corrupt, we may fail to uncompress a file because no file system space is available, etc.

Recoll versions prior to 1.21 always retried to index files which had previously caused an error. This guaranteed that anything that may have become indexable (for example because a helper had been installed) would be indexed. However this was bad for performance because some indexing failures may be quite costly (for example failing to uncompress a big file because of insufficient disk space).

The indexer in Recoll versions 1.21 and later does not retry failed files by default. Retrying will only occur if an explicit option (-k) is set on the recollindex command line, or if a script executed when recollindex starts up says so. The script is defined by a configuration variable (checkneedretryindexscript), and makes a rather lame attempt at deciding if a helper command may have been installed, by checking if any of the common bin directories have changed.