## Introduction

Organizing and searching a large collection of PDFs as part of a research project can be a demanding task. XMP metadata stored in a PDF, such as journal title, publication year, and user-added keywords, are often useful when searching for a publication. Here, we describe the use of a custom Recoll filter to retrieve this metadata, an indexing configuration to store it, and result paragraph format to display it. See also a related wiki entry, Generating a custom field and using it to sort results, for sorting results on PDF page count.

Bibliographic metadata can be saved in the PDF file itself. In the JabRef bibliography manager, this is done with the "Write XMP-metadata to PDFs" menu item. Note the presence of the keywords in the screenshot below; this field is a good place to tag the PDF with any words of your choosing to describe genre, topic, etc.

## Custom indexing (fields file)

Let's create two fields named "year" and "journal". The prefixes starting with "XY" are extension prefixes that are added to the terms in the Xapian database (Recoll internally does not use prefixes starting with XY). Additionally, the year and journal are stored so they can be displayed in the results list. Some other types of metadata, such as title, author and keywords, are already indexed by Recoll (the default rclpdf finds them using the pdftotext command) so there is no need to add those to the [prefixes] section.

Add this text to the fields file in your Recoll configuration directory (~/.recoll/fields).

[prefixes]
year = XYEAR
journal = XYJOUR

[stored]
year =
journal =


## Custom filter (rclpdf file)

This is where the heavy lifting happens. The filter should create HTML meta elements for each of the named index fields. Below is a diff between the default rclpdf and a customized one. The PDF metadata is gathered using the pdfinfo command. Then, grep and sed are used to extract the publication year and journal name from metadata fields beginning with "bibtex:" (part of the XMP metadata written by JabRef, in XML format). That information is fed to awk, which puts together the output. The crucial part in the customized awk script is the inclusion of the HTML meta elements with the names "year" and "journal".

There is some additional processing carried out by the l2html function. This replaces some LaTeX-style accents (stored in the PDF metadata if the BibTeX file contains them) with HTML entities. Only a few examples are shown here; other LaTeX accents could be processed in a similar manner. If desired, the sed commands could be modified to give UTF-8 characters instead of the HTML entities.

The l2html function also converts the LaTeX \emph{...} (emphasized text) to HTML markup for italics, <i>...</i>. With this, and the markup="html" attribute in the HTML meta elements (given in the awk script for the title and at the end of the filter for the author), italicized text and accented characters represented by HTML entities will be shown in the results.

One other thing to note: the filter changes the "Subject" HTML meta tag (created by pdftotext) to "Abstract"; this is so that the actual abstract - of the journal article, stored in the BibTeX database and written as metadata to the PDF, and reported by pdftotext as the "Subject" - is indexed independently of the title. Otherwise, terms in the "Subject" and "title" meta tags by default get indexed together by Recoll, so a title: query would actually match words appearing in the abstract.

Grab the default rclpdf for Recoll 1.18.1 (most likely /usr/share/recoll/filters/rclpdf) then apply this patch and save the result in ~/.recoll/filters/rclpdf .

104a105,126
>
> l2html()
> {
>   # redirect the stdin so the function can be used in a pipe
>   cat |
>   # use sed to replace some accented (LaTeX format) characters
>   sed -e 's/\\"a/\&auml;/g' |      # a umlaut
>   sed -e "s/\\\'a/\&aacute;/g" |   # a acute
>   sed -e "s/\\\\\a/\&agrave;/g" | # a grave
>   sed -e 's/\\u{a}/\&#x103;/g' |   # a breve
>   # linebreak so multiple \emph{.*} can be replaced
>   sed -e 's/\\emph{/\n&/g' |
>   # \emph{.*} to <i>.*</i>
>   sed -e 's/\\emph{$$.*$$}/\&lt;i\&gt;\1\&lt;\/i\&gt;/g'
> }
>
> PDFINFO=pdfinfo -meta "$infile" 2>/dev/null > # need grep -a (--text) becuase sometimes it treats input as binary > YEAR=echo "$PDFINFO" | grep -a bibtex:year | sed -e 's/<\/.*>//g' | sed -e 's/<.*>//g'
> JOURNAL=echo "$PDFINFO" | grep -a bibtex:journal | sed -e 's/<\/.*>//g' | sed -e 's/<.*>//g' > 107c129 < awk 'BEGIN'\ --- > awk -v year="$YEAR" -v journal="$JOURNAL" 'BEGIN'\ 111a134,136 > yearmeta = "<meta name=\"year\" content=\"" > journalmeta = "<meta name=\"journal\" content=\"" > endmeta = "\">\n" 115a141,146 > if(doescape == 0 &&$0 ~ /<\/head>/) {
>     match($0, /<\/head>/) > part1 = substr($0, 0, RSTART-1)
>     part2 = substr($0, RSTART, length($0))