Issue 196: Avoid filename duplicate results

aperezmendez writes

When I search for a filename, I obtain a result for each one of the contents of a zip file that matches the search query. Ej. if there is a zip file in my system called "proposal.zip", and I execute recoll -t -f "proposal", I get like 40 identical results. Does this make any sense? Is there any way to avoid it?

medoc writes

Hi,

Currently, the top container file name indexes all subdocuments, and I agree that this does not make sense.

There is no way to avoid it in the configuration, but this is a 1 line fix which I am going to apply, and it will be in future recoll releases.

As part of the problem reporter bounty program, you are eligible for a free binary package :) If you indicate your system/architecture (and if I can build it reasonably simply).

medoc writes

don’t add the top container file name to the subdocs terms. Closes issue #196

→ <<cset bfab8ed23f67 > >

aperezmendez writes

I’m using probably the easiest one. Arch linux :) Just send me the patch, and I can download and build the source myself. Or you might want to have you recoll git packaget on AUR.

medoc writes

Ok, patch here: https://bitbucket.org/medoc/recoll/commits/bfab8ed23f67d34cec86058ae492e665a9f7a8c7?at=RECOLL_1_19_MAINT1

For the moment and quite mysteriously, Arch Linux is not part of my current menagerie of VMs…

aperezmendez writes

Am I supposed to regenerate the index or is it related to the search part of the tool?

medoc writes

You need to reindex

aperezmendez writes

It worked. Thanks!

aperezmendez writes

Hi,

did you had the changed to see my search tool? https://bitbucket.org/aperezmendez/pyneedle

medoc writes

Yes I did, and I sent an email. I just checked and it was a reply to notifications-noreply@bitbucket.org :) Not too smart. I’m out the door right now, but I’ll paste the email here, it needs a bit of formatting probably.

medoc writes

I gave it a try and it’s nice and simple !

Using Python threading is a great idea for delaying the search-while-typing. I can’t remember how I did it in Unity, but it was much more complicated (I’m great at finding complicated solutions to simple problems…).

I tried to use your technique inside the Unity Scope, but no way, the Unity interface does not allow it.

It made me realize by the way that the Python Recoll module is not thread-safe at all. I need to do something about it, but, meanwhile, keep sure of having only one thread inside it…

About pyneedle:

^q would be nice (or is there already another shortcut?)

The tool has a few issues on older/non-standard desktop environments, which you may not care about at all, but anyway:

The columns do not auto-resize at all on Ubuntu 12.04 Precise, and as they can’t be manually resized either, usability suffers a lot.
Also under Precise, the file ops don’t work, here is what it says about "Open"

./pyneedle.py --engine=recoll
Traceback (most recent call last):
  File "./pyneedle.py", line 313, in <module >
    main()
  File "./pyneedle.py", line 306, in main
    win = PyNeedle(launcher=args.launcher, terminal=args.terminal, engine=args.engine, debug=(args.debug is not None))
  File "./pyneedle.py", line 133, in __init__
    self._add_popup_menu_actions(action_group)
  File "./pyneedle.py", line 194, in _add_popup_menu_actions
    ifactory.add('terminal', Gtk.IconSet.new_from_pixbuf(self._get_icon('terminal')))
  File "/usr/lib/python2.7/dist-packages/gi/types.py", line 72, in constructor
    return info.invoke(cls, *args, **kwargs)
TypeError: Argument 0 does not allow None as a value

I guess that’s no big deal as everybody will be switching to Trusty now. I’ll make the switch on my main machine as soon as the upgrade is proposed (I think that’s july).

The tool does not work through remote X11. Here is what it says:

./pyneedle.py --engine=recoll
Traceback (most recent call last):
  File "./pyneedle.py", line 313, in <module >
    main()
  File "./pyneedle.py", line 306, in main
    win = PyNeedle(launcher=args.launcher, terminal=args.terminal, engine=args.engine, debug=(args.debug is not None))
  File "./pyneedle.py", line 133, in __init__
    self._add_popup_menu_actions(action_group)
  File "./pyneedle.py", line 194, in _add_popup_menu_actions
    ifactory.add('terminal', Gtk.IconSet.new_from_pixbuf(self._get_icon('terminal')))
  File "/usr/lib/python2.7/dist-packages/gi/types.py", line 72, in constructor
    return info.invoke(cls, *args, **kwargs)
TypeError: Argument 0 does not allow None as a value

Using X11 through the network may be fading away anyway… There are quite a few features of recent desktops which are not designed to work this way. I sure can’t live without it though, I’m not bringing back the noisy PC from the garage to my desk…

aperezmendez writes

Thanks for the feedback.

I have two different recoll engines:

recoll, based on threading.Timer
recoll_mp, based on multiprocessing and threading

Both of them assure only one thread/process will accesses at the same time to the recoll’s python interface.

I’m not sure which one of them you used, or saw the code. In any case, Gtk has a special dealling with threads. The main program needs to call to the GLib’s threads_init() function before it calls to the main loop Gtk.main(). Probably, the Unity Scope/Lens is not prepared for using threads. You might want to take a look to the GLib’s timeout_add() function (it allows to create a new "tasks" that will call to the specified function after a specific amount of time), and GLib’s source_remove() function (it allows to de-register these callbacks). I’ve committed a new engine (RecollEngineNT) that implements this wait while typing functionality using these functions. If you want to see what’s happening, enable the --debug parameter.

^{w works (as used to close terminal windows, browser windows…). ESC also works. But it should be 5s to add}q as well. Thanks.

Actually, I’m an XFCE user, so I do care about columns. My two users so far make use of Fluxbox and i3, so my guess is that might be a problem of 12.04’s gtk3 version. I’ll take a look, thanks! The file ops problem seems also to be related with icons, probably again a problem of gtk3 API change. They got crazy at some point. I might change to qt eventually. It’s just a window with a list of results, shouldn’t be so difficult.

Remove X11 is more difficult to me. ¿Can I test it with ssh -X or ssh -Y?

Thanks for the replies!

medoc writes

I saw the threading.Timer implementation, it’s really neat.

The problem with Unity is that my object lifetime is exactly one query, I get the string, and I have to answer and I die. So the only place where the timeout could be implemented is their layer, and they don’t seem to do anything of the sort. Or maybe I missed something, I just started from a sample and I’m not going to read their code (there is no doc to speak of).

About remote X11, yes, ssh -X should do it (or you could set up your X server to allow tcp connections, but ssh -X is simpler for trying something). To check that it works, just start an xterm (or another basic X11 app). Then you can test the higher level stuff.

And maybe at some point, email would be more convenient and appropriate than this issue report :) mine is jf at dockes . org

Cheers,