Issue 222: system hangs while indexing

Recoll 1.20.1 + Xapian 1.2.8 on ubuntu 12.04 - 64 bit The last time I ran indexing was about two months. I had no problems. Since last Saturday every indexing attempt drives the whole mashine to stuck, every time after arbitrary amounts of files indexed. It can last for a few hours or a few minutes until the whole mashine hangs. Once stuck only recoll and the system stayed alive, but after attempting to kill recoll, also died the mashine. I attempted to restrict the amount of files to be indexed, but without success. What am I doing wrong?

Hi,

recollindex is a regular userland application with minimum cpu and io priorities. There are two ways which I can think of in which such an app could make a system appear "stuck":

The way to eliminate the first issue is to monitor the system while indexing, for example by using top or any other app displaying/recording the same type of info. You want to have a look at the values for virtual and resident memory for the recollindex process.

If the issue does appear to be memory-related, we’ll need to check what files are triggering this, using the debug log: https://bitbucket.org/medoc/recoll/wiki/ProblemSolvingData

It’s been quite some time since recollindex excessive resource usage reports have been common, so I consider this first reason relatively unlikely.

The second possible cause would be that recoll is exercising hardware elements in a unusual way, for example by accessing archive files containing bad blocks (triggering disk retries, which can hang the system for quite a long time). You could check for this by looking at your system logs for disk errors, or trying to replicate the problem by reading the whole data area, with a command like tar cf - topdir | dd of=/dev/null. The pipe is essential here as tar uses mmap and wont actually access the disk if directly output to /dev/null.

IO issues is the most common way by which a system can appear to get progressively stuck: apps appear perfectly functional until they actually need to read/write storage at which point they stop.

Hi Medoc, Thanks for your answer, I will try it. I am a little bit suspicious because of the diversity of the character encodings in the content of many files and some filenames on my disks. Some filenames are in greek language and there are many documents with greek text. On the other hand I had in the past no problems with indexing of almost the same files.

I’m quite certain that you can’t crash a system by speaking greek to it :)

What is the difference between followLinks and followSymlinks in recoll.conf? In my /usr/share/recoll/examples/recoll.conf there is a section [/usr/share/man] with the value followSymlinks = 1 and another parameter #followLinks = 0, which is commented out.

I still cannot index, I allways experience a total system breakdown.

Top gives me reasonable values for memory consumption, reading the disk with tar gives no errors.

I will give it another try with recollindex tomorrow after our parliament election.

Thanks for pointing this out, the real name of the variable is "followLinks", followSymlinks is a mistake and does nothing.

Please review my config and log files. My log file became very big because of loglevel=6.

Here is my log file https://drive.google.com/file/d/0BwKY42sbmSm0OXdkRXhWaVE1VkU/view?usp=sharing

Here is my /home/banos/.recoll/recoll.conf https://drive.google.com/file/d/0BwKY42sbmSm0Qk5TaUE4dzNBVTg/view?usp=sharing

And here some photos after diverse crashes https://drive.google.com/file/d/0BwKY42sbmSm0TEZHRW0tUVVLbkk/view?usp=sharing https://drive.google.com/file/d/0BwKY42sbmSm0eDlxYU1JYjRxY3M/view?usp=sharing https://drive.google.com/file/d/0BwKY42sbmSm0ZzVVQVBmenpXQ00/view?usp=sharing https://drive.google.com/file/d/0BwKY42sbmSm0QlN0TjBLd3BoVzA/view?usp=sharing https://drive.google.com/file/d/0BwKY42sbmSm0cmhjQTk3ZkhibVE/view?usp=sharing

Thanks for your support

did you check dmesg or /var/log/messages for error messages ?

I’d try to delete ~/.recoll/xapiandb before indexing

and also run single-threaded, just to see (add thrQSizes = -1 -1 -1 to ~/.recoll/recoll.conf)

But really, I have no idea what is going on, there is nothing looking suspicious in the log, I would really suspect a system issue.

Thank you for your advice, single-thread seems to solve the problem. But what is the loss from a single-threaded run? What is the advantage of multi-thread?

Is there a possibility to index (or re-index) to external index-files via the GUI?

I have three disks, a SSD and a HDD in my machine case and a portable external HDD. I have built a main index for the SSD (in ~/.recoll/), an external index for the internal HDD (in that internal HD) and another external index for my portable external HD. To update those indexes I use the recollindex command in a simple script at boot time. Is it possible to update the external indexes via the GUI?

Here is an interesting effect of "recollindex" during single-threated indexing: suddenly the font in console changed to some eastern encoding. The indexing process terminated then normally (I think).

https://drive.google.com/file/d/0BwKY42sbmSm0UWNLWHBtWmNqczA/view?usp=sharing

I still suspect the use of the diverse greek encodings in my files. There are at least one separate microsoft encoding, one ISO encoding and the UTF encoding for greek…

Greetings

Thank you for your advice, single-thread seems to solve the problem. But what is the loss from a single-threaded run? What is the advantage of multi-thread?

Is there a possibility to index (or re-index) to external index-files via the GUI?

I have three disks, a SSD and a HDD in my machine case and a portable external HDD. I have built a main index for the SSD (in ~/.recoll/), an external index for the internal HDD (in that internal HD) and another external index for my portable external HD. To update those indexes I use the recollindex command in a simple script at boot time. Is it possible to update the external indexes via the GUI?

Here is an interesting effect of "recollindex" during single-threated indexing: suddenly the font in console changed to some eastern encoding. The indexing process terminated then normally (I think).

https://drive.google.com/file/d/0BwKY42sbmSm0UWNLWHBtWmNqczA/view?usp=sharing

I still suspect the use of the diverse greek encodings in my files. There are at least one separate microsoft encoding, one ISO encoding and the UTF encoding for greek…

How could I check dmesg? My machine dies completely during indexing and after the cold reset a new dmesg is created. Is there any other log I could check?

Thanks and greetings from Greece V.

Single-threaded indexing is slightly slower, that’s all.

About multiple indexes: please have a look at the doc, ask questions on the mailing list if you still have any after reading. http://www.lesbonscomptes.com/recoll/usermanual/usermanual.html#RCL.INDEXING.STORAGE http://www.lesbonscomptes.com/recoll/usermanual/usermanual.html#RCL.INDEXING.CONFIG

The console font change was probably due to some special character in the log file which was interpreted as an escape sequence command by the terminal. It happens, because the recoll log is actually binary data (no care for escaping or transcoding weird data in the journal). Just use terminal reset from the menu to restore the terminal.

Recoll normally has no problems with multiple encodings, at worse you will index bad data, not crash the system.

dmesg messages mostly go to /var/log/messages. It’s possible to get them in a more real time fashion, but we’re getting in kernel debugging territory, and this is not my competence.

Cheers,

jf

probable hardware issue