Implementing a Media Server plugin.

The upmpdcli Media Server subsystem has multiple functions: proxying for external media services (Qobuz, Tidal etc.), or non-UPnP servers (Subsonic), serving local media files, giving access to external radio catalogs or services (bbc, radio-browser, radio-paradise, mother-earth-radio).

The main process does nothing itself, beyond implementing the UPnP Media Server protocol. All the actual work is performed by plugins, running as separate processes and communicating through pipes with upmpdcli.

All current plugins are written in Python, but this is not an absolute requirement as the pipe "protocol" is very simple and can be implemented in another language. However, we will not describe this part in detail, and will concentrate instead on writing a Python plugin.

The Python plugins share a common implementation of the pipe protocol in a shared module. As a plugin writer you will implement a small set of methods which will be called by the protocol handler and dispatcher which manages the dialog with the main process.

Plugin name and activation

Each plugin has a name (e.g. tidal, uprcl, qobuz…​). This name is the name of the subdirectory where the plugin code is stored under $datadir/cdplugins, where $datadir is the place where upmpdcli stores its readonly data (typically /usr/share/upmpdcli).

The main Python program for the plugin must be named $datadir/cdplugins/<plgname>/<plgname>-app.py e.g. /usr/share/upmpdcli/cdplugins/uprcl/uprcl-app.py.

upmpdcli will enable a plugin by creating a container of the same name in the UPnP Media Server root container. This will happen if the above directory and program exist, and if a variable named <plgname>user is set in the configuration file. The plugin process is then later started if and when the container is actually accessed.

In addition, if a variable named <plgname>autostart is set, a process for the plugin will be started immediately so that the plugin can perform its own initialisation before the first access.

Environment

Some environment variables are set by the main process for use by the plugins. None are needed by all plugins, and they will be described with the specific function that they support.

PYTHONPATH is set so that the Python code in the pycommon directory can be imported directly.

Helper methods, the upmplgutils module

This module is directly accessible from a plugin with import upmplgutils.

getOptionValue()

getOptionValue(nm, dflt = None)

Return a parameter value. This usually comes from the upmpdcli configuration file (from the key nm), but also possibly from the environment variable named UPMPDCLI_<NM> where <NM> is the uppercased nm argument. The file is searched first, then the environment. If the key is not found, the dflt value is returned. If dflt is set, and a value is found, the method tries to convert the string value to the type of dflt.

getcachedir()

getcachedir(servicename, forcedpath=None)

Return the name of a cache directory for this plugin. The location will exist and be writable. It is private to the plugin, to store whatever it needs. The information will be persistent across reboots (typically under /var/cache/upmpdcli/<plgname>).

direntry()

direntry(id, pid, title, arturi=None, artist=None, upnpclass=None,
         searchable='1', date=None, description=None)

Create and return a container object with the specified values (see further down about container and item objects).

getConfigObject()

getConfigObject()

Return the upmpdcli configuration as a ConfSimple object (see the pycommon/conftree.py module). Normally not used as getOptionValue() will do a better job.

uplog()

uplog(s, level=3)

Print a message to the log (which is usually stderr at the moment).

getserviceuserpass()

getserviceuserpass(servicename)

Return a pair of values from the configuration, the values for the configuration parameters named <plgname>user and <plgname>password. The password may be empty or undefined in the configuration, but the user name will always be defined (even with a bogus non-null value) because its presence determines if upmpdcli will start a process for the corresponding plugin. This method need not be used by plugins which don’t actually require user credentials.

url_from_trackid()

def url_from_trackid(httphp, pathprefix, trackid):

See the trackuri() plugin method description for a detailed explanation.

Return an URL constructed from the upmpdcli HTTP server host and port (UPMPDCLI_HOSTPORT), the plugin pathprefix (UPMPDCLI_PATHPREFIX), and the service track identifier.

trackid_from_urlpath()

def trackid_from_urlpath(pathprefix, a):

See the trackuri() plugin method description for a detailed explanation.

Return a track identifier extracted from the path part of an URL constructed by url_from_trackid() (or built like them). The a parameter is the one received by trackuri().

The data produced by plugins

The plugins output essentially two types of objects, which reflect closely those in the UPnP Media Server interface:

  • Containers are directory-like objects possibly parent to children objects.

  • Items are file-like objects mostly representing multimedia entities: audio streams, images, etc.

Both kinds of objects have a similar structure and are represented in the interface as Python dictionaries. The parent-child relationships are defined by specific field values.

Some plugins present a tree-like hierarchy of containers holding other containers and items. Others may be as simple as a single root container having a number of child items.

The methods in the plugins can usually return multiple objects, and they do so in the form of JSON-encoded arrays of dictionaries.

    ...
    # Produce a list of entries (dicts), then:
    encoded = json.dumps(entries)
    return {"entries" : encoded, "nocache" : "0"}

Each entry object is a Python dict. See further for the contents.

Common object fields

The following are dictionary keys present in both containers and items. All keys and values are strings.

Mandatory keys

  • "id": Object Id: the choice of object id values is mostly up to the plugin. However all object ids for a given plugin must begin with 0$plgname$ where plgname is the plugin name, and this value is the object id for the plugin root container. The rest is up to the plugin.

  • "pid": Parent Object Id: the id for the parent container of this container or item.

  • "tp": object type: must be literal "ct" for containers or "it" for items

  • "tt": title. This will typically be displayed by a control point as the object name.

  • "upnp:class": UPnP class. Containers must have classes beginning with object.container, items have classes beginning with object.item. Common values: object.container.album.musicAlbum, object.container.person.musicArtist, object.container.album object.container.playlistContainer, object.item.audioItem.musicTrack. See the UPnP Content Directory document, section 7 for a more complete list. The exact values are usually not that important as long as containers are object.container descendants and items object.item ones.

Optional keys

  • "upnp:albumArtURI": The URI for an image to be displayed with this object when appropriate.

  • "upnp:artist": An artist related to the object.

  • "dc:date": in ISO format, e.g. 2015-12-31.

  • "dc:description": Description text.

Container-specific fields

  • "searchable": Possible values: "0", "1". This tells the Control Point if a search operation rooted in the container could be successful.

Item-specific fields

The following attributes relate to the performance itself, independantly of a concrete representation. All fields are largely optional:

  • "upnp:album"

  • "composer"

  • "conductor"

  • "upnp:genre"

  • "upnp:originalTrackNumber"

  • "didlfrag": it is possible to communicate other properties by directly encoding them as an XML fragment in this string, which will be directly incorporated in the UPnP data by the main process.

The following attributes relate to a specific representation, e.g. a flac or mp3 file. Note that renderers will usually override the values by getting them from the actual data. Control Points may display some of them, especially "duration"

  • "uri" the link from which the data maybe retrieved (maybe indirectly, see the trackuri() method description further).

  • "duration" (should have been res:duration). Seconds.

  • "res:bitrate": bits per second (the UPnP document says bytes??).

  • "res:bitsPerSample": bit depth. e.g. 16, 24.

  • "res:channels": channel count.

  • "res:mime": MIME type, e.g. "application/flac".

  • "res:samplefreq": Sample frequency in Hz.

  • "res:size": size in bytes.

Additionally, if a "resources" field is present in the entry, it must be an array of additional objects each describing a specific representation of the performance, as above. Returning several resources with different formats (MIME, sample rate, etc.) may allow the Control Point and Renderer to choose the best one.

The plugin interface

All plugins need to implement two methods, each having a decorator which allows the dispatcher to call them when a request comes over the pipe: browse() and search(). The latter can always return an empty list if search is not supported, and should not be called if the searchable attribute is never set, so for plugins with no search support, it’s mostly there as a precaution.

@dispatcher.record("browse")
def browse(a):
    """Browse the container defined in a (a dict), and return the results"""
    ...

@dispatcher.record("search")
def search(a):
    """Search the container defined in a (a dict), and return the results"""
    return {"entries" : "[]", "nocache" : "0"}

A third method trackuri() exists and needs only be implemented by plugins with special needs about the track URIs (see further).

The browse() method

The input is single Python dict parameter with the following keys:

  • "objid": the object ID for the target object.

  • "flag": can be set to either literal "meta" or "children". If set to "meta" the data wanted is the metadata for the object itself (which should be returned as a list with one element anyway). If set to "children", the metadata for the container direct children is wanted.

  • "offset": the offset from 0 of the first returned entry.

  • "count": the entry count desired.

Note that a plugin can always chose to ignore offset and count and return all its data (indicating it does so in the return values).

Return value: this must return a Python dict with the following keys:

  • "entries": a JSON-encoded array of container or item objects as described above.

  • "nocache": (optional, default false) a flag set to "0" or "1" indicating if the result can be cached or not.

  • "offset": (optional, default 0) the offset of the first entry (default: 0)

  • "total": (optional, default -1) the total number of entries in the container.

The search() method

This gets a single Python dict parameter with the following keys:

  • "objid": the object ID for the container to be (recursively) searched.

  • "value": the string to be matched.

  • "objkind": the type of object which is looked for. This can be one of "track", "artist", "album", "playlist".

  • "field": the object field to be used for searching the input value. Can be one of "artist", "album", "track".

  • "origsearch": the actual UPnP search string, in case the plugin can make better sense of it than what is conveyed by "objkind", "field" and "value".

  • "offset": the offset of the first value in the return list.

  • "count": the desired entry count in the result.

The return value is identical to the one for the browse() method.

The trackuri() method

Some plugins act as proxies for streaming music services like Qobuz or Tidal. The streams supplied by most of these services have no permanent URLs. Rather, a temporary URL with a limited life time is used to get the data. Consequently the actual stream URLs cannot be directly used in the UPnP data, because, for example, they would become stale while sitting in the playlist waiting to be retrieved. Instead, the URLs initially supplied by the upmpdcli plugins in the return lists of browse() or search() just contain the permanent track identifier, and point to the internal upmpdcli HTTP server.

When the renderer actually requests the URL, the upmpdcli server obtains the actual service stream URL from the plugin, and either redirects the renderer to it, or, for renderers which would not support redirection, proxies the stream data. The choice between proxying or redirecting is currently done globally, for all plugins and renderers, but this could be changed. By default, redirection is used, being much more efficient.

This mechanism is implemented in the plugin by the trackuri() method, which receives the permanent URL, extracts the track id, and requests the temporary URL from which the audio data can actually be fetched from the streaming service.

Example, the Qobuz plugin trackuri() method:

@dispatcher.record("trackuri")
def trackuri(a):
    msgproc.log(f"trackuri: [{a}]")
    maybelogin()

    trackid = trackid_from_urlpath(pathprefix, a)
    media_url = session.get_media_url(trackid)

    return {"media_url" : media_url}

The trackuri() method receives a single parameter which is the URL path part, and should extract the track identifier and request a temporary URL from the service.

Return value: the method must return a Python dictionary with a single key: "media_url", the temporary service URL.

The permanent URLs are constructed in the following way: the host and port are obtained from the UPMPDCLI_HOSTPORT configuration variable (e.g. "192.168.1.1:9090"), and the beginning of the path part is obtained from UPMPDCLI_PATHPREFIX (in practise "/" + <plgname>, e.g. "/tidal"). The host and port is of course necessary so that the renderer goes to the right server. The path prefix tells the server from what plugin it should request the actual URL. The rest of the URL is up to the plugin, but some common code (not of mandatory use) assumes the following form: <pathprefix>/track/version/1/trackId/<trackid>

Two helper methods in the upmplgutil module support the mechanism: url_from_trackid() constructs an appropriate URL, and trackid_from_urlpath() extracts the trackid from the path part of an incoming URL.

WEB server

By default, the upmpdcli UPnP (from libnpupnp) HTTP server does not serve any data from the file system. It is possible to instruct it to do so by setting the webserverdocumentroot parameter in the configuration file. It will then be possible to access all documents under the designated file system directory with an HTTP GET (as long as the filesystem permissions allow of course).

Only some plugins have a use for this, and the variable is not set in the default configuration.

If the facility is enabled, its parameters are communicated to the plugins through two environment variables: UPMPD_UPNPHOSTPORT and UPMPD_UPNPDOCROOT which respectively define the host and port to use in the URLs and the filesystem directory used as root for the Web server.