upabx is a tool for performing ABX listening tests using one or several UPnP renderers, and, possibly, a hardware switchbox.

It is useless because all reasonable hardware sound the same (to me), except for speakers and listening rooms. upabx does not support switching the latter (bad bad lazy developper), and there are better tools for comparing encodings and other software factors.

In other terms: you could use upabx to compare your extremely expensive DAC with a Raspberry PI Hifiberry board, but they sound the same. If you want to compare 128 vs 320 Kb/S MP3, use the foobar2000 ABX comparator plugin or something like it, it will do a better job.

See at the end for a practical example, comparing two DACs (none of the very expensive kind).

If you are still with me though…

What the software does

The program controls:

  • What is played (maybe different for A/B if comparing e.g. compression methods). This is set as UPnP URL, offset, duration.

  • The device it is played on (for comparing streamers): UPnP Renderer name or uuid.

  • The line-level switch setting (for comparing preamps or sources, etc.).

  • The speaker-level switch setting. Speaker refers to the power level, we normally use it to switch amps actually (still, could switch speakers, too).

These are defined in two sets in parameters, which define the test. The definition file also supports text describing what is connected to what and what it is we try to test.

A companion program provides an easy GUI for creating test definition files, including retrieving the URL from any renderer on which it is playing.

The ABX test Graphical User Interface

upabxgui-1.png

This should be largely self-explanatory. The program should be given a test definition file as parameter on the command line:

/path/to/upabx/bin/upabx /path/to/test_definition_file

You can use the Player section to play samples A, B and X as you like.

The Range section allows you to restrict the part which is played (start and end). Just click the buttons while one of the samples is playing.

Once you’re tired of comparing, use the Choice section to tell the application if you think that the X sample is the same as A or B. Then click Confirm, which will store your choice, and increment the test number.

Repeat until the end of the test sequence.

The A and B samples can be the same URL playing on different renderers (needs switch or mixer), or different URLs (for comparing encodings or performances), or the same renderer and URL if you are using a hardware switch to compare preamplifiers or amplifiers. More on the sample parameters further down.

Building, operating environment

The program only runs on Linux/X11 at the moment. It should not be hard to port it to Windows (or Mac OS X). As far as I can see it should mostly be a matter of adjusting the configuration file locations. libupnpp is already ported to both systems, and its Python bindings should not be a major issue. Please contact me if there is an interest: jf@dockes.org

All the code lives here. The GUI code is in the abx directory, the test creation code is in abxmktest.

The Makefile will bundle the Python code into two self-contained Python zipped scripts inside the bin directory: bin/upabx and bin/upabxmktest. Just run make:

cd /path/to/upabx
make

You need to have libupnpp and libupnpp-bindings installed, and you can then start upabx or upabxmktest like any Linux command: copy them somewhere in your PATH, or specify their absolute path, e.g.:

/path/to/upabx/bin/upabxmktest
/path/to/upabx/bin/upabx /path/to/test_definition_file

You will need a hardware switch (or at least a mixer) if you want to compare electronics (from streamers/DACs to power amps). upabx interfaces with a small local web application, typically running on a Raspberry Pi, for controlling the switch (the GUI does not need to run on the same host as the switch controller). Any kind of relay which can interface more or less directly with Raspberry Pi GPIO pins should be workable.

The upabxmktest test creation utility

upabxmktest-1.png

The utility makes it easy to create test files. Especially, it will find out the list of renderers present on the network, and it will retrieve an URL and the associated metadata from a Renderer, so you can set the track to play for each of the A/B sections just by setting it to play (or pause) somewhere, and fetching it from the GUI.

Set the parameters to use in the A section, then set the changed parameters in the B section.

The URL Choose buttons will open a popup, where you will be able to select a renderer from a list. The utility will then fetch the URL and metadata for the track currently playing or paused on the Renderer.

When you are done, use the File menu to save the test definition. The file format details are described in an annex.

Test results

The test results are stored in ~/.local/share/upabx and are named like date-time-result. They look like the following:

[ Contents of the test parameters file included here]


Mon Nov 13 17:45:37 2017 Choice: B : True
Mon Nov 13 17:45:59 2017 Choice: A : True
Mon Nov 13 17:46:16 2017 Choice: A : True
Mon Nov 13 17:46:36 2017 Choice: B : True
Mon Nov 13 17:46:55 2017 Choice: A : True
Mon Nov 13 17:47:11 2017 Choice: B : True
Mon Nov 13 17:47:26 2017 Choice: A : True
Mon Nov 13 17:47:42 2017 Choice: A : True
Mon Nov 13 17:47:52 2017 Choice: A : True
Mon Nov 13 17:48:00 2017 Choice: B : True

Mon Nov 13 17:48:00 2017 Test completed: 10/10

The global configuration

Most of the program parameters are found in the test parameter files (see next). However, some defaults are defined in a global configuration file (~/.config/upabx/upabx.conf). The file just contains name = value lines.

seqlen

Default test sequence length.

switchhost

Host name or IP address for the switch interface.

switchport

TCP port for the switch interface.

The hardware switch interface application

The switch interface application is a trivial WEB interface based on the Python Bottle framework, and on the Raspberry PI GPIO Python interface (I only tried it on a Raspberry PI, I guess that it should be easy to port to another GPIO-equipped computer if an appropriate Python module is available).

The application should be able to control any relay which can interface with the Pi On/Off GPIO lines (details on my crappy hardware implementation follow further down).

I was too lazy to package the code, which does not really need an installation (can run from anywhere). It is found under the hwctl directory in the upabx source tree.

The switch application configuration

The configuration file is expected to be found in /etc/audioswitch.conf. It defines the GPIO pin numbers used to control the line and speaker relays. The contents are as follows:

# GPIO pin numbers for the speaker relay and line input relay
speaker_channel = 4
line_channel = 17

To be accessible from the network, the app should be started as:

switcherapp-run.py -a 0.0.0.0

The default port is 8080. You can change it by passing -p portnum on the command line.

Example systemd service file (change mylogin to the real user name):

[Unit]
Description=HW Switch manager HTTP server
After=network.target

[Service]
User=mylogin
ExecStart=/usr/bin/python2 /home/mylogin/hwctl/switcherapp-run.py -a 0.0.0.0
Restart=always

[Install]
WantedBy=multi-user.target

The switch application interface

The server presents four URLs on the network: /spka, /spkb, /linea, /lineb. I’d wager that you can guess what they do. Example command line usage from another machine (upabx uses Python code to do the same):

wget -o /dev/null http://swhost:swport/spkb

When using the application from upabx, the host and port are set in the upabx global configuration file.

The hardware switch

I am not a hardware engineer, and I’m sure that there is not much to like in my switchbox :)

The big relays need too much command power to be controlled directly from the GPIO, so there is a small interface relay between them and the Pi.

Also they need 12V, (which is ennoying, it should be possible to arrange to run everything on 5V), in consequence, the whole thing is powered by a small external 5/12v PC power supply.

I am using pre-integrated modules for the low power relays (line control and speaker interface relay), they have a transistor for GPIO-compatible control levels, a flyback diode and maybe an indicator LED, all on a small circuit board, and DIY them would not save a lot of money…

Most expensive section of the whole thing ? Line and speaker connectors.

The box looks complicated because there are many wires. But really, it is very simple:

  • The chosen GPIO pins are connected to the small relay inputs.

  • 5V (and possibly 12 V) are distributed where needed (Raspberry PI, relays VCC).

  • The intermediate relay output goes to the big relay command input.

  • The switched lines to the relay contacts.


The switch box

The switch box. On the left, the Raspberry PI presents its ethernet port through the back plate. And the power supply terminal. In the centre, the big speaker/power amp relays. On the right, the line relay (will be replaced by a pair so that I can switch both signals and earths):



The speaker relays

One of the big speaker/power line relays. The small interface relay controlled by the GPIO piggy-backs on it, fixed by double-sided tape. One can also see the main relay flyback diode.

supplier page for the big relays (in French, sorry, but I’m sure that you can get an idea anyway and find an equivalent in your country). Finding one which can be controlled with 5V will make your life easier.



The line-level relay

The line-level relay module. For example. I want to replace it with a double one to avoid having to link all the signal earths.


Battle of the DACs

Rpi+hifiberry odroid+explorer

And now for a real example. Comparing a HifiBerry DAC+ pro mounted on an rpi3, with the Meridian Explorer 2, connected to the USB output of an Odroid C2. Both systems are running mpd + upmpdcli. The Headphone amp is an O2.

I am using a 96/24 FLAC 2L demo track for the comparison: "Vivaldi Cantata Rv 679. Aria Cupido te Vedi"


The test configuration file:

description = Comparison of hifiberry dac+pro with Meridian Explorer 2, \
using the 2L Vivaldi 96/24 FLAC sample

# Hifiberry DAC+ pro
renderer = UpMpd-r31
line = A

url = http://192.168.4.4:9790/minimserver/*/mp3/test-tracks/2L/VIVALDI_Cantata_Rv_679__Che_Giova_Il_Sospirar,_Povero_Core_-_Aria__Cupido,_Tu_Vedi-flac-96-24.flac


[test]

# Explorer
renderer = odroid32
line = B
Rpi+hifiberry odroid+explorer

It is very important to match the output levels. Conveniently, my Sennheiser has pluggable connections with nice bare contacts which I can easily hook up to the multimeter. Otherwise, a custom mini-jack to banana plug cable would be handy.

Both DACS have hardware volume controls, which I use. The Hifiberry is set at 100/100, the Explorer 2 at 96/100 (in MPD terms). Testing with a 50 Hz tone, I could not adjust better than 0.43 V vs 0.45 V, which should be roughly equivalent to a 0.6 dB difference ? This might need a little more attention, also measuring at other frequencies (hopefully, these 2 should be flat at 50hz), but it’s all moot because anyway…:


I did not even bother to run the ABX sequence: both DACs sound exactly the same to me.

Useless ABX tool :)

I am not claiming any absolute result though: I do have very bad hearing, and, as you know, we can’t prove a negative :)

And I’d be quite interested by your comments (see the Disqus section at the end).

Annex: test definition file details

Tests are defined in simple configuration files. The configuration file name for a test is passed to the GUI on the command line.

This section describes the file contents in detail, but you will usually create the tests with the upabxmktest program, so you should not need to care about the details.

The parameter values are set as:

paramname = some value

Anything which does not look like this will be a comment.

There are two sections in the file: the first one defines the base parameters (A), the second one ([test]) defines only the parameters which change for B.

# A parameters
param = somevalue
otherparam = othervalue

[test]
# B parameters. Only otherparam changes

otherparam = yetanothervalue-for-the-B-sample

Test parameters

The following values can be set. Except if mentionned otherwise, they all can be changed in the [test] section.

renderer

renderer name The UPnP friendly name or UUID for the Media Renderer device to use.

url

The URL used by the Media Server for the track to play. upabxmktest will find this for you. Else, this is not always obvious to determine. upplay has a handy Copy URL entry in the directory listing popup menu. Also, upnp-inspector.

meta

The metadata for the track to play. This will be set by upabxmktest, but not all renderers require it, so you may not have to bother about it if you create a test by hand. Some renderers don’t like URLs though. For example, upmpdcli needs checkcontentformat = 0 if there is no MIME format in the metadata.

seek

start of play position in seconds. The player will seek to this position before starting play. This can differ in the second section, which can be handy if you are comparing interpretations and not audio quality.

dursecs

sample duration in seconds. Set to 0 to let the complete track play. Can’t change for the B section (what for ?).

seqlen

number of tests to perform. You won’t be able to save a result before the sequence is complete. The default is defined in the global configuration.

line

line-level switch position. Set to A or B. The value is not related to the A/B choices, A and B are the values used by the hardware switch control application for turning the relay on or off.

spk

speaker-level switch position. Set to A or B. Ditto no relationship to the A/B sections.

If neither line nor spk are set, upabx will not try to access the hardware switch interface.

comments powered by Disqus