Using your guitar (or other instrument) in Ubuntu via Jack

When I'm not working on JotPot, I often play the guitar (poorly).

Something that took me quite some time to figure out and setup was connecting my guitar to my computer running Ubuntu. And so I thought I would write a post about it.

So. How do you connect your guitar (or other instrument) to your Ubuntu system?
I will be assuming that you have Ubuntu 12.10 (not Ubuntu Studio), but the same guidelines could apply to other Linux distros as well.

Plugging my guitar into the microphone jack

Well, I first tried the most obvious thing - connect it through my microphone jack on my on-board sound card. At first, the clean guitar sound was rather... metallic. So I played with the microphone boost setting.

Typed alsamixer in terminal, pressed F4 for Capture controls and found a setting called 'Front Mic Boost' on my card. Setting that to 0 made the metallic effect go away and I had a nice clean sound. The only problem was that whenever I would change the volume (or other setting) on my guitar, the sound would mute for a few seconds. I suspect that's due to some filtering that ordinary cards do on microphones to cut out crackling. That probably picked up some high-pitched sound when tuning settings on the guitar.

So all in all - it worked reasonably well, except I couldn't play with the settings on the guitar live (while playing it).

A real (but cheap) guitar card

So I bought this very basic guitar connection kit, which worked like a charm. Be wary when buying sound cards for Linux, because the drivers might not work. Check Ardour's FAQ page or do a Google search when you make a decision.

Low latency

Now, to do any proper recording or live sound processing in Linux you need to have the sound processed in real time. That means you can have at most 10-20 milliseconds sound delay. Ubuntu's default sound system, pulseaudio, was not designed for this - it has a latency that can get to a few hundred milliseconds, which is huge. Instead, we need to use Jack.

$ sudo apt-get install jackd qjackctl

jackd is the Jack sound daemon and qjackctl is the easiest way to control Jack (but you could also use the command line as well). I will go into actually configuring Jack a bit later.

The next thing you need to do, is install a more appropriate kernel. The default Ubuntu kernel is designed for all-around desktop use, and in general favors throughput and power saving over latency (because that's what makes things more efficient in servers and in common desktop applications). Changing the kernel might sound a bit scary, but thanks to good community support this is in fact pretty easy (short of recompiling the kernel yourself).

Ideally you would want a hard real-time kernel (see this wikipedia article on real-time kernels and this list of kernel types in Ubuntu Studio) for the best latency. But that would actually require you to recompile the kernel yourself and might cause your proprietary graphics driver to stop working. If you want to do other things on your Ubuntu system, other than recording I would advise against this.

Instead, you can install a soft real-time kernel, which probably provides ~90% of the latency benefit (from 60-100 ms latency to 5-20 ms) at no other noticeable cost. To get it, all you have to do is

$ sudo apt-get install linux-lowlatency

and reboot into the new kernel. You might have to reinstall your proprietary graphics driver if you have problems, but mine worked straight away. The lowlatency kernel works so seamlessly that I removed my generic kernel after a few days of testing and I'm using the lowlatency version all the time.

Introduction to Jack

Right. So now, how do you use Jack?

First of all, get rid of pulseaudio (for the time being). You need to tell pulseaudio not to restart when it is killed. For that, create a file ~/.pulse/client.conf and add the line

autospawn=no

to it. Now you can use the following commands for stopping / starting pulseaudio

$ pulseaudio -k  # stop pulseaudio daemon
$ pulseaudio -D  # start pulseaudio daemon

The reason you need control over this is that PulseAudio takes over the sound device and then Jack might have problems starting.

After you have killed PulseAudio, you can start qjackctl. Go into Setup and select the card you want to use Jack with (I use my USB guitar card). Note: don't use multiple cards at once with Jack (via Input / Output device) - more on this later. Click OK and then click Start.

The default settings should be OK to start off. But you might want to reduce the latency by changing Frames/Period and Periods/Buffer to adjust the buffer size - this is the main thing to look after. If the buffer is too small, you get frame drops. If the buffer is too big, you get high latency. Try out different values and see what works best for you. The values I use for my system are 64 frames and 3 periods.

To actually hear something after starting Jack, you need to use the Connect button and play around with connecting different programs (Jack clients) or devices to each other. The Jack connections make Jack an incredibly flexible sound system, without compromising latency.

If you have problems hearing sound after connecting your guitar to your output, use

$ alsamixer

to see if the volume is OK (you need to run this in a terminal).

Next, you can use programs like rakarrack (guitar effects) or ardour (recording) to start off - both available through Synaptic. You will find that some audio players (Spotify, Rhythmbox) have Jack bindings too - although you'll need to find out how to keep them from using PulseAudio by default. Or if you want to do some video-audio track syncing, there are some video players (like xjadeo) that can output in sync with Jack.

Also awesome about Jack is its transport syncing capability - meaning that you can have multiple programs using Jack and have their transport sync'd up with frame-by-frame precision - also useful for video-audio track syncing (or what not). See this article for a primer on transport syncing.

Advanced Jack configuration

People often use Jack only when they need it - and use PulseAudio the rest of the time. But I configured it such that I use it for everyday things too. Let's see how.

My requirements for the job are: I want to plug my guitar via my USB guitar card, hear the sound through my speakers, which are plugged into my on-board card and still be able to hear sound from any regular application.

So here's the challenge. Jack is (mostly) unable to use two cards at once, because of time skew issues. If you do try to use it with two cards, you start getting crackling very soon, due to de-syncing between cards. Also, regular applications (like, Skype, Chrome, most music and video players) cannot use Jack for sound output - they use PulseAudio. But PulseAudio cannot use a card if Jack is already using it. Moreover, the sound coming from my guitar and going to my speakers may never go through PulseAudio, because that would impose a high latency penalty. And finally, it would be nice if I could use the microphone in my webcam for Skype-ing.

After many, many (many!!) different tries, I came up with this setup: I use Jack just on my USB guitar card and alsa_in and alsa_out for my on-board card and my webcam (which acts partially as yet another sound card for its microphone). alsa_in and alsa_out are Jack clients that can send / receive audio straight to / from an Alsa device. These programs don't have sync issues, because they do another (completely independent from Jack) resampling of the sound. And note that they take over a sound device the same way Jack or PulseAudio do.

Next, in order to use PulseAudio for other programs, I needed a way to make PulseAudio output sound via Jack itself. For this, I used the PulseAudio Jack module.

$ sudo apt-get install pulseaudio-module-jack

This essentially makes PulseAudio believe that there is a device (called Jack Sink) that it can output sound to. But this actually shows up in Jack connections as a sound source, which you can connect to your output. Neat! You can also use this for input - via Jack Source, which shows up as an output device in Jack.

Be wary of these gotchas though. 1. Start PulseAudio only after starting Jack, alsa_in or alsa_out, so it would not take over the device(s) Jack wants to use; and 2. If you can't see the Jack Sink in PulseAudio settings (System Settings -> Sound), you need to do


$ pacmd load-module module-jack-source channels=2
$ pacmd load-module module-jack-sink channels=2

to load the module manually.

So finally, I used some scripts to keep everything running automatically. Here's my complete setup:

The reported Latency is 4.35 ms. Note: alsa_out adds another ~5 ms to this.

I'm using Soft Mode so it would not crash in case of occasional (but rare) frame drops. The qjackctl tray icon turns red when that happens - don't panic if you get that from time to time (so long as it doesn't affect your recording).

Note: Use a persistent Patchbay, to keep the jack connections between Jack restarts.

Notice that I'm using a custom script started asynchronously (might need to sudo apt-get install daemon). That takes care of starting alsa_in and alsa_out processes and setting up PulseAudio.

jack-after-startup.sh:


#!/bin/bash

sleep 3

# On-board card output.
daemon -- alsa_out -j 'alsa_out_PCH' -d hw:PCH -p 64 -n 4 -r 44100

sleep 2

# On-board card input (disabled).
#daemon -- alsa_in -j 'alsa_in_PCH' -d hw:PCH -p 1024 -n 4 -r 44100

# Webcam input.
daemon -- alsa_in -j 'alsa_in_webcam' -d hw:U0x46d0x9a4 -p 372 -n 4 -r 16000 -c 1

sleep 3

# Start PulseAudio. Note, we are starting this AFTER alsa_in and alsa_out have taken over the devices.
pulseaudio -D

sleep 3

# Load PulseAudio Jack modules.
pacmd load-module module-jack-source channels=2
pacmd load-module module-jack-sink channels=2
sleep 1

# Set the default sink and source in PulseAudio, to use Jack.
# Also, set the volume to 25% and 100% for sink and source, respectively (I had problems with these getting reset to random values every now and then).

pacmd set-default-sink jack_out
pacmd set-default-source jack_in
pacmd set-sink-volume jack_out 16384
pacmd set-source-volume jack_in 65536

sleep 1

# Set some ALSA sound volumes as well. Look in alsamixer for the correct names for your channels (Master, Speaker etc.)

amixer -D hw:PCH sset Master 85% unmute
amixer -D hw:PCH sset Speaker 100% unmute
amixer -D hw:PCH sset Headphone 100% unmute
amixer -D hw:PCH sset PCM 100% unmute

Note that I name devices by hw:PCH (on-board card), hw:CODEC (USB guitar) and hw:U0x46d0x9a4 (webcam) - to avoid confusion. Type in

$ aplay -l

$ arecord -l

to find out what your devices are called. Using their names rather than hw:0,0 is much better especially when using USB cards (my devices occasionally switch up).

Auto start, start minimized, show up in system tray. Note: You might not see it in the tray with Ubuntu's default settings (see this).
You will also need to add qjackctl to the Startup Applications.

Note: system is Jack's default output right to the device (the USB guitar card). But I use alsa_out_PCH to hear sound in the speakers plugged into my on-board card.

Extra points if you get jackd, alsa_in and alsa_out to start with a higher process priority (nice -8 perhaps?). But you need root permission for that, which means you would need to start jack in an entirely different way (on system startup).

Some more hackery (fixing suspend)

The only thing I still haven't fixed is having Jack survive system sleep (suspend). Every time it wakes up, I have to stop Jack from qjackctl (important - don't kill, stop first), do a sudo alsa force-reload and start qjackctl again. Let me know if you find a better solution to this.

EDIT: Woohoo! Managed to fix suspend/wake as well.

You need to create a file (note, the 00_ in front of the file name is important)

$ sudo touch /etc/pm/sleep.d/00_reset-jack
$ sudo chmod +x /etc/pm/sleep.d/00_reset-jack
$ sudo gedit /etc/pm/sleep.d/00_reset-jack

And add the following to it


#!/bin/bash

my_user="aether"

onSuspend() {
    # Note that it's easier to stop all sound-related stuff on Suspend,
    # before Jack enters in error state (it takes a really long time to
    # recover on resume otherwise).

    sudo -u $my_user -- pulseaudio -k 2>&1 > /dev/null

    # Send SIGTERM to applications and wait for 10 seconds. If they haven't closed
    # within 10 seconds, send SIGKILL.
    applications_to_kill="qjackctl qjackctl.real alsa_in alsa_out pulseaudio"
    echo "SIGTERM-ing $applications_to_kill"
    killall $applications_to_kill 2>&1 > /dev/null
    start_time="$(date +'%s')"
    while pidof $applications_to_kill 2>&1 > /dev/null ; do
        echo -n "."
        sleep 1
        now="$(date +'%s')"
        dt=$((now - start_time))
        if [[ $dt -gt "10" ]]; then
            echo
            echo "SIGKILL-ing $applications_to_kill"
            killall -9 $applications_to_kill 2>&1 > /dev/null
            break
        fi
    done
    echo

    # Tell jack server to stop and wait until its status is reported as stopped.
    echo "Stopping jack"
    status=0
    while [ $status -ne 1 ]; do
        echo -n "."
        DISPLAY=:0 sudo -u $my_user -- jack_control stop 2>&1 > /dev/null
        sleep 1
        DISPLAY=:0 sudo -u $my_user -- jack_control status 2>&1 > /dev/null
        status=$?
    done
    echo
}

onResume() {
    sudo -u $my_user -- pulseaudio -k 2>&1 > /dev/null

    sleep 1

    # Start qjackctl again.
    DISPLAY=:0 daemon -- sudo -u $my_user qjackctl
}


case "$1" in
    hibernate|suspend)  onSuspend;;
    resume|thaw)        onResume;;
esac

But make sure you change my_user at the top with the user you are running qjackctl as. This script basically kills and restarts jack et al before the computer enters sleep and then restarts it when it resumes (assuming qjackctl takes care of restarting pulseaudio, alsa_in and alsa_out as per jack-after-startup.sh).

If you feel uncomfortable with this kind of scripts, you can also just do

$ jack_control stop

and then quit and start qjackctl again every time you resume from suspend and you should be good to go. The above command may take a long time to complete though, because Jack is in an error state.

That's pretty much it!

You might find some more programs to play with in this Ubuntu Studio preparation guide - but note that not everything applies to regular Ubuntu as well.

But all in all, have fun playing!

JavaScript memory leaks

"What? Memory leaks? In JavaScript?!?"

Yes.

Modern browsers nowadays do have garbage collectors that can successfully collect JavaScript structures with circular references. But let's see what could actually create memory leaks then.

The anatomy of a leak

As with all GC'ed languages, memory leak problems can appear at the interface with another language. Any ideas yet?

Well, in modern browsers, the JavaScript world and the DOM world are different languages and therefore, they are GC'd in different ways. So when you make references between JavaScript and HTML elements, you actually create a reference between these two worlds. Now, because these two worlds are different, there are problems when we come to detecting whether things are referenced or not.

While it is possible to detect that JavaScript objects or DOM elements are not referenced anymore by anything, detecting that there's a circular reference between JavaScript and the DOM is problematic. As of the time of writing, for some reason, none of the browsers out there can do that.

(EDIT: Nope, I was wrong. When I tested these cases in isolation in Chrome, they did not leak. But, I'm pretty sure I have seen at least one of these patterns leak in Chrome in a more complicated scenario. So it might be that the patterns discussed here - in this simple form - are somehow optimized by the compiler, which prevents them to leak. Or I am entirely wrong about Chrome, and it does cover for all these patterns gracefully.)

So, with no further ado, I would like to introduce you to...

Leak Pattern #1 (The Leaking Closure)

function callMeAndIWillLeak() {
  var div = document.createElement("div");
  div.onclick = function () {
    div.innerText = "Hello, leak!";
  };
}


This is the most obvious leak possible. The created DOM element references an anonymous function, which references div, which references the DOM element.

So DOM element -> (anonymous function) -> div -> DOM element.

The problem is that the anonymous function is a closure, which in JavaScript terms, means that it contains references to otherwise local variables (in this case, the variable div). The takeaway is that you should be very careful when giving closures to the DOM. Also, it is a good idea to use a battle-tested event system like JQuery's, which works around these leaks.

Leak Pattern #2 (The Leaking Inline Event Handler)

You might be surprised that older browsers (Ahem! You know who you are!!) leak in other situations too.

function callMeAndIWillLeak() {
  var div = document.createElement("<div onclick='foo()'>");

}

Yup, that's it! The function foo doesn't even have to exist!
At least, this doesn't affect modern browsers, I believe. Though I wouldn't be too surprised if I saw IE8 still reproducing this.

Leak Pattern #3 (The Leaking Expando)

Expando properties are additional JavaScript properties that you assign to DOM elements. Here is an example of a leak:


function callMeAndIWillLeak() {
  var div = document.createElement("div");
  var myObjectWichReferencesDiv = {
    /* ... */
    myDiv: div,
    /* ... */
  };
  div.myCustomProperty = myObjectWhichReferencesDiv;
}


This leaks due to the following circular reference: DOM element -> expando property -> complicated object -> DOM element.

Bonus. Leak Pattern #4 (The Leaking Spaghetti)

In practice, the leaks above won't be really sitting all that obviously, though. In most cases you will have no idea why your web page keeps growing in memory.

So in general, I have found that if I keep very well track of where I reference the DOM elements (especially the ones created programatically) and if I clear out with null when I'm done with them, I get away with it.

You can think of DOM elements as the only things in JavaScript that are not automatically garbage collected and which need manual clearing of all direct references to it - just to make sure.

Leak patterns summary

DO use a good event system (like JQuery's) to attach events.
AVOID using closure variables in event handlers.
DON'T use inline event handlers (<div onclick='foo()'>).
AVOID using expando properties.
If you do use expando properties, AVOID using non-primitive JavaScript types.
And if you do use expando properties with non-primitive objects, DO nullify those properties when they are not needed anymore.
DO nullify all direct references to DOM elements when not needed anymore, just to make sure.

Finding leaks

Now that I talked about what a leak is, we might as well look at how to find them.
I will show you how to use Chrome's developer tools for this one, although other browsers have similar features too.

EDIT: This assumes that you are able to reproduce the leak in Chrome.

The leaking action

First, use Chrome's task manager (Shift+Esc) to look at the per-tab memory usage. While keeping an eye on the memory usage try to find the user action which triggers an increase of memory every time you do it.

Alternatively, open Chrome's developers tools -> Timeline -> Memory and then click the record button. That will give you a more detailed view of where the memory gets allocated.

You will notice that the memory shown in the task manager is different from the memory shown in the timeline. This is because a lot of the memory is kept natively and is not shown in the timeline. Canvas or image data are such examples. When hardware acceleration is used, your web page could occupy significant amounts of memory in the GPU Process (shown separately in Chrome's task manager) as well. I'm guessing this memory comes from canvas and image data, mostly, as well.

Pinpointing the leaking objects

After you know which action leaks, go to Chrome's developer tools -> Profiles. And use the Take Heap Snapshot command. Take a snapshot before and after the action. You should return to the same state as before the action before taking the "after" snapshot on your web page (i.e.: the state in which you would normally expect the memory usage to drop to the initial value). After that, click on Snapshot 2 and in the bottom, where it says Summary, select Comparison.

That will show you all the objects that were created or garbage collected between the two snapshots. Note that not all the objects shown there are memory leaks. Some objects may have been replaced. Those will show up as well, since they are different objects now.

Note that Chrome does a GC just before taking the snapshot, in order to get rid of all the objects that would be GC'd anyway.

As we have seen, the most common source of leak are DOM elements. So a good place to start is looking for "Detached DOM tree" entries. Look through their properties and try to find out where they come from.

The memory leak is usually noticeable when there are "Detached DOM tree" entries of the order of hundreds. But if you are leaking images or canvases, then the memory could burn even with a few entries. (The snapshot doesn't report high memory usage for those entries, though! The image and canvas data is stored natively.) So keep that in mind when handling those.

Super.js

I wrote this small JavaScript Object-Oriented library while working on JotPot.

I know there are lots of frameworks/libraries out there that do something similar, but

1. I couldn't find one that did exactly what I was looking for
2. I wanted to learn more of JavaScript

... and so, I built my own.

This is the result:

https://github.com/vlad-alexandru-ionescu/super.js#superjs


Super.js is a helper library for cleaner OOP JavaScript. It allows you to use classes in a way closer to what you are familliar to in Python or Java.

Here is a usage example:

declareClass(
    function Mammal(hasHair, legs, arms, canSee) {
        this._hasHair = hasHaid;
        this._legs = legs;
        this._arms = arms;
        this._canSee = canSee;
    },

    function eat(food) {
        alert("Om nom nom...");
    },

    function sleep() {
        alert("Z Z Z z z z . . .");
    }
);

declareClass(
    function Dog(onLeash) {
        this._super.constructor(true, 4, 0, true);
        this._onLeash = onLeash;
    },

    extend(Mammal),

    function eat(food) {
        if (food.isDogFood()) {
            return this._super.eat(food);
        } else {
            throw "up";
        }
    }
);

declareClass(
    function Cat(canSee) {
        this._super.constructor(true, 4, 0, canSee);
    },

    extend(Mammal)
);

declarClass(
    function BabyCat(mommy) {
        this._super.constructor(false);
        this._mommy = mommy;
    },

    extend(Cat),

    function eat(food) {
        if (food.isFromMotherCat()) {
            alert("Meaaaw nom nom...");
        } else {
            throw "NO!"
        }
    }
);

And you use the classes as you would expect:

var myDog = new Dog(true);
myDog.eat(someFood);

var myCat = new Cat(true);
var myBabyCat = new BabyCat(myCat);
while(isDay()) {
    myBabyCat.sleep();
}