Issue 195: async search

Hello,

I’m using recoll’s python interface for a search tool I’m working on. The tool shows search results as soon as you write a character on the search entry. The problem is that as recoll searches are not asynchronous, I cannot cancel current search whenever the user clicks another key. This is possible with tracker.

Do you have any plans of adding async searches that can be cancelled at any moment?

Regards

Above comment was mine, but I forgot to log in. Just commenting here to get feedback on the email.

Regards

Hi,

Xapian searches are not cancellable. I asked them to add cancellation points (which would be easy to do when you know where to put them as they’re exception safe, just throw when you see the flag) but they won’t budge.

So I think that the only possible approach would be to execute a subprocess. If you go this way, it’s probably simpler to execute the command line query tool "recoll -t" than to write a specific python program for the child process.

Otoh, the Python module for the Unity scope/lens just runs the searches. In most common cases, they’re fast enough that cancelling is not really necessary (I certtainly agree that it would be nice to have though).

Hi,

thanks for the quick and really precise response. I tried with subprocess, but I gave up because I needed shared memory and I needed to change a lot of code (my tool already works with a tracker backend). I will give them another try, but with python. I don’t really want to parse text output of recoll -t :D.

I wanted cancellation because I like the "instant result whenever you type a letter". If I cannot cancel, all the searchers are performed, taking so much time. So far, I have solved this issue using thread.Timer and adding a 0.2s delay to perform the search (timers are cancellable whenever they have not started). In this way, if you type normal speed, you don’t perform all the searchers, but only the last one, or when you stop writing to think. This is good enough, I have to say, but I wanted something more "professional" such as async searchers from tracker. In fact, tracker takes longer than recoll to perform the searchs, but it seems to be faster as you can actually cancel them.

I’m making my tool public in bitbucket in a few weeks, either with subprocess cancellation or 0.2s search delay. Anyway, if you are interested on it, I can provide you read access to the repository.

Regards

I had also something like the 0.2 S delay in an early version of the unity scope, and I just abandoned it, because it made things more complicated with relatively little benefits in most cases.

recoll -t -F <field list > output is really easy to "parse", each entry is a space-separated list of the fields you require, each encoded in base 64, so no escaping or quoting is necessary. This format was added just for this usage.

And yes, I’d like to have a look at the tool whenever you feel ready !

Cheers,

jf

I don’t know for Unity, but in my case is just as simple as threading.Timer(0.2, do_search) instead of do_search. I forgot to mention something important. My tool is intended mainly for file search (it has two modes, actaully, one for file search, and the other for Full Text Search), so when I write something like "manual pd", I actually make the following search: "filename:*manual* AND filename:*pd*". This wildchar expansion takes a time. In my computer, average wildchar file search takes around 0.1s. So, if I write "manual pd" it would require the whole second (0.1 per letter), regardless how quick you wrote. If I do the 0.2 trick (or the cancellation trick), whenever you write the second, third, etc. letter, the previous search is cancelled and you don’t realize they were being performed.

I use my search tool lile 100 times everyday. I access to almost all my files this way. I want to open my workout sheet, I write "full" and I get my fullbody.odt file. I always order results by date, and I only present the first 50 results (this is configurable in the code). In this line, I have a further question. I can tell you, it is really convenient if you like that (it is not perfect, of course). One can see it as a faster and minimalistic version of recoll GUI, using GTK.

I already sent you an invitation (note that default configuration uses tracker, you have to add the --engine=recoll parameter). Regards