Issue 340: Xapian 1.4 on Ubuntu Zesty and index sizes

Hi JF, I have spotted that Ubuntu Zesty, due out in under 3 months, will default to Xapian v1.4 (http://packages.ubuntu.com/source/zesty/xapian-core).

I depend a lot on Recoll, and use the backports PPA to keep to a recent version. I’m considering trying it out with the new Xapian version before the OS upgrade is released, to soften the transition. However I saw your exchange with the dev last year on index sizes with Xapian 1.3.5, and that is a sensitive area for my use cases.

Have you any reports of Recoll users on Xapian 1.4 yet or feedback on index sizes? Does Recoll 1.22.4 include automatic compression after indexing, would this need manual intervention, or has the index size issue subsided?

I’m keen to look into these issues before Zesty arrives, as they could be noticed by a lot of users when they upgrade their OS this spring.

I appreciate all your work on Recoll Thanks, Art

Hi,

As far as I know, nothing major changed in the Xapian index since the April tests, so the index size issue must still be there.

If you are not expecting intensive updates of the index after the initial indexing, I guess that running xapian-compact would be a good thing. recollindex does not do it automatically.

I think that the best approach to test these issues would be to create a set of documents for testing, and create indexes on Xenial and Zesty (or build a separate Xapian 1.4 + recoll version on Xenial). Maybe have a second set to add for updating the index, and compare the times with and without xapian-compact, etc. I’d be interested by your results.

I think that the index size increase will only be an issue in relatively special situations (esp. given the proportion of multimedia data on typical user’s disks these days).

Closing this because it is not directly a recoll issue: I don’t think that it would be realistic to bundle Xapian 1.2 with Recoll, this would be rejected by the distributions. However this will remain feasible for a local build. I’d be interested by your further findings on the subject anyway.