Minuet 0.2: massive refactoring and Android version available

Hi there,

It's been a while since my last blog post about Minuet but that doesn't mean we aren't moving it forward. Actually a lot of work has been done lately, mostly related to architecture improvements, UX revamping, refactoring, code convergence, and its availability on Android devices. Minuet is a quite recent KDE project (it's been developed since November, 2015) and I'm really delighted with what we achieved so far, given we are a small team made up of only two developers (including a GSoC student) and a designer.

So, keep reading for an overview on the improvements delivered in Minuet 0.2 (desktop version) and our journey towards the very first release of Minuet for Android devices :).

Architectural improvements and general refactoring

Minuet 0.1 already presented a somehow nice architecture, where all ear training exercises are defined in multiple JSON files, which are automatically merged by Minuet's core to make up the navigation menu. That makes it easy to maintain a huge number of exercises and add new ones with no changes in source code. Some technical debt regarding QML source code was identified though and, of course, challenges introduced by the advanced features we expect to address and the need to have Minuet running on other platforms (such as Android, iOS, and Windows) had to be properly tackled with a stronger and more flexible architecture.

The architecture improvements released in Minuet 0.2 address three fundamental aspects: JSON specification of ear training exercises, sound infrastructure, and UI/UX improvements.

New JSON structure for exercises specification

In Minuet 0.1, exercises were defined in multiple JSON files, where intervals/chords/scales/rhythms were defined alongwith the exercises where they appeared as possible answers:

{"exercises":[{
    "name":"Intervals",
    "children":[{
        "name":"Ascending Melodic Intervals",
        "root":"21..104",
        "children":[{
            "name":"Seconds", "options":[{
                "name":"Minor Second", "sequenceFromRoot":"1"
            },{
                "name":"Major Second", "sequenceFromRoot":"2"
            }]
        },
        ...
        ]
    }]}
    ]
}

Although that allowed for defining new exercises with no changes in source code, music concepts definitions (e.g. the "Minor Second" and "Major Second" intervals) had to be duplicated in any other exercise category where they appear (e.g. in "Second and Thirds" and "Second to Octave" categories). That was a burden since it caused a lot of duplicated entries for those concepts appearing in multiple categories.

In Minuet 0.2, exercise specification JSON files were splitted in two different types: definitions JSON files and exercises JSON files. Definitions JSON files specify music content (scales, intercals, chords, and rhythm patterns) regardless of the exercise categories where they appear in:

{
  "definitions": [
    {
      "tags": ["interval", "ascending", "2", "minor"],
      "name": "Minor Second",
      "sequence": "1"
    },
    {
      "tags": ["interval", "ascending", "2", "major"],
      "name": "Major Second",
      "sequence": "2"
    },
    ...
  ]
}

In the new architecture, music content definitions are marked with any number of tags. Those tags are used by exercises JSON files to collect the definitions which will make up a given exercise category. That makes the definition of new exercises as simple as querying definitions by the tags they were marked with:

{
  "exercises": [
    {
      "name": "Intervals",
      ...
      "children": [
        {
          "name": "Ascending Melodic Intervals",
          "and-tags": ["interval", "ascending"],
          "children": [
            {
              "name": "Seconds",
              "or-tags": ["2"]
            },
            ...
            {
              "name": "Seconds and Thirds",
              "or-tags": ["2", "3"]
            },
            ...
            {
              "name": "Second to Octave",
              "or-tags": ["2", "3", "4", "tritone", "5", "6", "7", "8"]
            },
            ...
          ]
        }
      ]
    }
  ]
}

Note how any (sub-)category uses and-tags and/or or-tags to select the definitions marked with, respectively, all and/or any of the provided tags. Now, changes in definitions JSON files are propagated to all exercises JSON files. You can define any number of definitions and exercises JSON files, since both are merged into a single JSON file for each type.

Sound infrastructure

Minuet 0.1 relied on Drumstick library to implement the required MIDI capabilities to play exercises. That not only yielded a high coupling between Minuet's core and Drumstick but also added a number of run-time dependencies, such as TiMidity++ and freepats. As a consequence, and in spite of a basic system sanity check executed at first Minuet's run, we still got some broken audio infrastructure in some installations.

The new architecture released with Minuet 0.2 totally decouples the sound infrastructure from Minuet's core and sound backends for different platforms are now implemented as Qt plugins. That enabled the move to using Fluidsynth + GeneralUser GS soundfont as Minuet Desktop's sound backend, with no run-time dependencies. Minuet Android's sound backend was built on top of CSound + sf_GMbank soundfont.

New Minuet sound backends can be easily created by implementing the Minuet::ISoundBackend interface:

class MINUETINTERFACES_EXPORT ISoundBackend : public IPlugin
{
    Q_OBJECT
...

public:

...
public Q_SLOTS:

virtual void setPitch(qint8 pitch) = 0;

virtual void setVolume(quint8 volume) = 0;

virtual void setTempo(quint8 tempo) = 0;
virtual void prepareFromExerciseOptions(QJsonArray selectedExerciseOptions) = 0;
virtual void prepareFromMidiFile(const QString &fileName) = 0;

virtual void play() = 0;
virtual void pause() = 0;
virtual void stop() = 0;
virtual void reset() = 0;

...

};

The API is quite straightforward. The most important service is provided by prepareFromExerciseOptions(), where all required steps to initialize/create the audio representation of a given exercise option (a specific interval, scale, chord, or rhythm pattern) should be executed. After that, the exercise can be played by invoking the play() method. The methods setPitch(), setVolume(), and setTempo() should be also implemented to adjust, respectively, the overall pitch deviation, the playing volume, and the playing speed.

UI/UX improvements

Minuet 0.2 is now totally based on QtQuickControls2 and, therefore, requires Qt 5.7 to build. The move to QtQuickControl2 is justified not only by its clean and simple API, but also because it allows for enhanced productivity and leverages code convergence. Enhanced UX and code convergence are ongoing efforts in Minuet, even though the codebase for Minuet Desktop and Minuet Android are already quite similar, differing only on the adopted sound backends.

Minuet for Android available in Google Play Store

Minuet for Android is the result of the nice work performed by Ayush Shash in Google Summer of Code 2016. After three months of intense work, struggling with different sound libraries for Android and diverse UX patterns for mobile applications, we are happy in making Minuet for Android available with all features already presented in its desktop counterpart.

The sound backend in Minuet for Android was implemented on top of CSound: a powerful domain-specific language for sound synthesis which works on Windows, OS X, Linux, Android, and iOS. Instead on using all those oscilators combinations and other complicated sound synthesis stuff, we again adopted soundfonts as audio samplers for CSound in Android.

A typical navigation drawer allows for the user to navigate through all exercise categories resulted from merging the available exercises JSON files. The source code responsible for loading and merging JSON files, creating the navigation menu, and dynamically exhibiting the exercise screen is 100% shared with Minuet Desktop.

Adapting the piano virtual keyboard to small form factors was a particularly challenging task. After some unsuccessful attempts, we ended up using some labels to identify the piano octaves, we constrained the visualization to a single octave and then implemented an automatic horizontal scroll to the keyboard region which encompasses the exercise being currently played.

The UI for running intervals, chords, scales, and rhythm patterns exercises is quite similar to Minuet Desktop's one. In order to keep it scalable for exercises with many available answers we implemented a vertical scroll inside "Available Answers" groupbox. The answer(s) selected by the user for a given exercise is(are) presented in the "Your Answer(s)" groupbox. Incorrect answers are shown with a red rectangle and can be clicked to have the associated right answer revealed.

As I mentioned before, Minuet is still on its infancy but I guess it's looking quite promising already :). Now, we'll concentrate our energies in merging remaining divergent code, stabilizing the architecture, making the last UI polishments and then work hard on providing really amazing music content. Addressing new platforms? Yes, that's also in our roadmap, hopefully :).

Minuet for Android wouldn't be possible without the support of the KDE community. Many thanks to Ayush Shah for the courage to brave this road, to Alessandro Longo for the amazing category icons, to the VDG team for valuable UI feedback, and to Aleix Pol for helping with Android cmake buildsystem.

Yeah! We are nearly one week away from QtCon. Dude, I'm excited to meet old friends and make new ones :) If you are heading Berlin and want to learn more about Minuet, I'll present a talk about it on Day 3 (saturday, 3rd September), at 3pm, in room A08.

See you!