Issue 216: Building Recoll gets stuck during configure

philgogear writes

I am trying to build recoll from source a QNAP NAS (TS-470 Pro) I have xapian-core, xapian-omega & xapian-bindings up and running. Now I am try to build recoll from the tarbal (http://www.lesbonscomptes.com/recoll/recoll-1.19.14p2.tar.gz) however when I try to set it up with configure it gets stuck.

This is the command I rund

#!bash

./configure --prefix=/share/CACHEDEV1_DATA/.qpkg/Optware/ --with-inotify --disable-qtgui --disable-x11mon

It gets stuck at this output:

#!bash

checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -march=i686 -mtune=i686 accepts -g... yes
./configure: line 2979: test: too many arguments
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... 64
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking how to run the C++ preprocessor... g++ -march=i686 -mtune=i686 -E
checking tr1/unordered_map usability... yes
checking tr1/unordered_map presence... yes
checking for tr1/unordered_map... yes
checking unordered_map usability... no
checking unordered_map presence... no
checking for unordered_map... no
checking for sys/mount.h... yes
checking for sys/statfs.h... yes
checking for sys/statvfs.h... yes
checking for sys/vfs.h... yes
checking for file... /opt/bin/file
checking for aspell... /opt/bin/aspell
configure: inotify not found, inotify monitoring disabled
configure: FAM library not found, disabling FAM and real time indexing support
checking for mkdtemp... yes
checking for type of inbuf parameter to iconv... checking for type of string parameter to putenv... checking for xapian-config... /opt/bin/xapian-config

any ideas what might be going on?

medoc writes

No, normally the next step is to execute xapian-config a number of times. I’d suggest adding traces (I see that there are a bunch of comment echo statements after the xapian section), and retrying to see where it stops.

philgogear writes

Thanks for you quick reply. Adding traces is indeed what I did, from what I can see, it does not go beyond line 4898. I would expect it to at least reach line 4940, before

#!bash

 XAPIAN_CONFIG=$XAPIAN_CONFIG1

but it doesn’t :(

any clue?

medoc writes

No cue, unfortunately. What system are you running on this machine ? Did you try to execute xapian-config by hand to check that it behaves reasonably ?

philgogear writes

It is running on a qnap TS-470 Pro, Intel based. Any xapian config I can try to get you some more info?

medoc writes

It seems that the configure scripts runs xapian-config --libs and xapian-config --cxxflags

I meant, what kind of Linux are you running on the device ?

philgogear writes

Output from xapian-config --libs

#!bash

 > xapian-config --libs
-L/share/CACHEDEV1_DATA/.qpkg/Optware/lib -lxapian

output from xapian-config --cxxflags

#!bash

 > xapian-config --cxxflags
-I/share/CACHEDEV1_DATA/.qpkg/Optware/include

System info

#!bash

 > uname -a
Linux iLabNAS 3.4.6 #1 SMP Fri Oct 3 14:57:06 CST 2014 x86_64 unknown

 > cat /proc/version
Linux version 3.4.6 (root@BuildServer46-1) (gcc version 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)) #1 SMP Fri Oct 3 14:57:06 CST 2014

it is this device: http://www.qnap.com/i/uk/product/model.php?II=106

thx for your support!

medoc writes

So we now know that xapian-config runs normally…

I’m sorry, I have absolutely no idea of what can be wrong. Maybe you could try to re-run autoconf and see if the new configure works better ? This is just shaking the tree to see what falls, I have no reason to believe that this will change anything…

philgogear writes

so, seems that we do get a bit further than earlier mentioned, we reach this command (around line 4996)

#!bash

LIBXAPIANSTATICEXTRA=`$XAPIAN_CONFIG --static --libs 2 > /dev/null`

and indeed if I run

#!bash

xapian-config --static --libs

it also hangs, to be more precise it does seem to be doing stuff, it takes about 25% of my CPU power (dual core) and a bit of memory.

I found references to this --static being added here to the xapian-config code http://comments.gmane.org/gmane.comp.search.xapian.devel/1532

and you can find it in the code of course http://trac.xapian.org/browser/svn/trunk/xapian-core/xapian-config.in

however I do not understand why it is hanging/taking up so much resources.

you any ideas how to overcome this?

philgogear writes

oh and I also found this reference http://grokbase.com/t/xapian/xapian-discuss/084azashw7/static-library-problem

stating that it might be a zlib issue, but still investigating that option.

philgogear writes

I installed the latest zlib from source, that did not help :(

medoc writes

xapian-config is a shell script too, so maybe you can take a look. If there is nothing obvious, maybe you could ask the question on the xapian mailing list, as this is quite clearly a xapian-config issue.

But I think that you could just comment out the section with xapian-config --static in configure.ac, as it is used only for static builds, probably never used nowadays, this comes from a time when xapian was not widely packaged, so it was a help to the users that the recoll binary did not need local Xapian libs.

philgogear writes

might just do that indeed, thx, will let you know how it ends up!

philgogear writes

Commenting out the line with the static reference did work. New issue though:

#!bash

/share/CACHEDEV1_DATA/.qpkg/Optware/bin/../lib/gcc/i686-unknown-linux-gnu/4.2.1/../../../../include/c++/4.2.1/ext/atomicity.h:51: undefined reference to `__sync_fetch_and_add_4'
/share/CACHEDEV1_DATA/.qpkg/Optware/bin/../lib/gcc/i686-unknown-linux-gnu/4.2.1/../../../../include/c++/4.2.1/ext/atomicity.h:51: undefined reference to `__sync_fetch_and_add_4'
/share/CACHEDEV1_DATA/.qpkg/Optware/bin/../lib/gcc/i686-unknown-linux-gnu/4.2.1/../../../../include/c++/4.2.1/ext/atomicity.h:51: undefined reference to `__sync_fetch_and_add_4'
recollindex.o: In function `__exchange_and_add':
/share/CACHEDEV1_DATA/.qpkg/Optware/bin/../lib/gcc/i686-unknown-linux-gnu/4.2.1/../../../../include/c++/4.2.1/bits/basic_string.h:288: undefined reference to `__sync_fetch_and_add_4'
/share/CACHEDEV1_DATA/.qpkg/Optware/bin/../lib/gcc/i686-unknown-linux-gnu/4.2.1/../../../../include/c++/4.2.1/bits/basic_string.h:288: undefined reference to `__sync_fetch_and_add_4'
recollindex.o:/share/CACHEDEV1_DATA/.qpkg/Optware/bin/../lib/gcc/i686-unknown-linux-gnu/4.2.1/../../../../include/c++/4.2.1/bits/basic_string.h:288: more undefined references to `__sync_fetch_and_add_4' follow
collect2: ld returned 1 exit status
make[1]: *** [recollindex] Error 1
make[1]: Leaving directory `/share/CACHEDEV1_DATA/Download/recoll-1.20.0p2/index'
make: *** [all] Error 2

not sure what to do about this "__sync_fetch_and_add_4" issue. any thoughts?

philgogear writes

well, I think I got 1 step further, by adding the right CFLAGS but new issue. This is the command I am running

#!bash
 ./configure CXX='g++ -march=i686 -mtune=i686' CFLAGS='-g++ -march=i686 -mtune=i686' --prefix=/opt/ --with-inotify --disable-qtgui --disable-x11mon

and getting this nice error

#!bash

g++ -g -O2 -I. -I../aspell -I../bincimapmime -I../common -I../index -I../internfile -I../rcldb -I../unac -I../utils  -Wall -Wno-unused -I/opt/lib/../include -I/opt/include  -DRECOLL_DATADIR=\"/opt/share/recoll\" -DLIBDIR=\"/opt/lib\" -fPIC -DPIC -DHAVE_CONFIG_H -D_GNU_SOURCE  -c ../utils/pathut.cpp
../utils/pathut.cpp: In function 'bool fsocc(const std::string&, int*, long int*)':
../utils/pathut.cpp:70: error: aggregate 'statfs buf' has incomplete type and cannot be defined
../utils/pathut.cpp:71: error: invalid use of incomplete type 'struct statfs'
../utils/pathut.cpp:70: error: forward declaration of 'struct statfs'
make[1]: *** [pathut.o] Error 1

if I add the following to ./utils/pathut.cpp

#!bash
#ifndef HAVE_SYS_STATVFS_H
#define HAVE_SYS_STATFS_H
#endif

I goes further but I get the __sync_fetch_and_add_4 error again…

philgogear writes

Btw I am using gcc 4.2.1

medoc writes

About struct statfs, I’d just check where this is defined under /usr/include and just include the appropriate file.

About __sync_fetch_and_add_4, recoll does not use it directly, but I think that this thread looks interesting: http://gcc.gnu.org/ml/gcc-help/2011-09/msg00195.html

philgogear writes

ok so adding --includedir=/opt/include along with a decent version of python did the trick, and only using cxxflags so configure command now looks like

#!bash
./configure CXXFLAGS='-g -march=i686 -mtune=i686' --prefix=/opt/ --includedir=/opt/include --with-inotify --disable-qtgui --disable-x11mon

once I got it running I needed iconv as well, but building that from source was easy enough.

thanks for your support!

medoc writes

Glad this finally worked. Is there a change that you think should get into the recoll source ? The change about struct statfs above seems a bit too generic and I don’t have a system around to test a more appropriate configure.ac change. Anyway, closing the issue !

medoc writes

Finally worked with the right (and quite specific) configure options.