Reverse Conducting Example

Sections: [ Data acquisition | Raw data | Data analysis | Score alignment | Evaluation ]

This webpage demonstrates data acquisition and analysis results for reverse conducting of the Chopin Mazurka in F minor, Op. 7, No. 3. Reverse conducting means tapping along to an audio recording of a performance by following the beats of the performer. The data will not be completely reliable since the person following the recording will be reacting to the performer by ear, although this is remedied somewhat by repeated listening to a single performance.

The performance used in this example is by Ignaz Friedman from the compact-disc re-release of a 1930 recording. The reissued recording is volume 30 of the Great Pianists of the 20th Century series published by Phillips (456 784-2) which was released in 1999. The example mazurka is track 10 on the first CD of the two-disc volume.

Data Acquisition

Timing data was acquired by tapping on the computer keyboard of a Sony Vaio PCG-R505GC laptop running Windows XP. The acquisition program was a command-line program called backbeat.exe written in the C++ programming language. Here is the source code for the program (excluding the necessary library for compiling) as well as a compiled version of the program.

• Source Code: backbeat.cpp
• Compiled Program: backbeat.exe

Type this number (1193 in the example) after pressing "C" when you start up the backbeat.exe program. The CPU speed was measured more carefully, and found to be 1192.981 rather than 1193. So, if you can tolerate a drift of 1 or 2 milliseconds per minute, then the approximation report in the system information window is OK. Otherwise, take a watch with a second hand, and use the backbeat program to time a known duration, such as 24 hours to fine-tune the CPU speed value.

If your computer has a variable-speed CPU (perhaps Mobile Pentiums) then you probably cannot use the backbeat.exe program. The backbeat program uses a 64 bit register on the CPU chip which stored the number of cycles that the CPU has gone through since the computer was turned on. For example, on 1 gigahertz computer computer, this register will be updated a billion times each second. After the computer has been turned on for one minute, the register will contain the number 60 billion. This timer is very accurate (since it is derived from a crystal oscillator as in digital watches) and is independent of the operating system; however, you need to also know the speed of the CPU in order to measure time accurately with it. For those who want implementation specifics, here is the function I use to access the CPU clock cycle count in Windows and Linux on Intel CPUs which are faster than 75 MHz.

While running the backbeat.exe program, tapping on the space bar indicates a beat, and tapping on any other lowercase key will generate a barline as well as a beat marker. The program is hard-wired to cycle through a 3/4 meter (since all of the mazurkas are in 3/4). So if you come to the end of a data acquisition session and beat one does not start after a barline, then you know that there was a missing or extra beat somewhere in the data, and you should probably throw that run out and start over.

The audio playback was using the Microsoft Media player, version 10. The audio playback was done from this program while another window ran the backbeat.exe program in parallel. First the backbeat program was started and configured, then the audio playback was started, and finally the backbeat window was selected and reverse conducting commenced.

Raw Data

Raw data from the backbeat program is displayed on the screen and written to a file at the same time so that accuracy can be monitored while tapping to a performance. The basic data acquisition is recorded in four columns, in a Humdrum file format compliant form, where the data columns are separated by a single tab character:

1. The first column indicates the rhythm for the beat. For the mazurkas, this will always be "4" which indicates a quarter note duration for the beat. This is not very exciting data, but is useful for error checking.
2. The second column indicates the beat in the measure on which the data was acquired. The options are "1", "2" or "3" for the first, second or third beat of the measure respectively.
3. The third column indicates the time in milliseconds (thousandths of a second) which is when the reverse conductor pressed a key on the computer keyboard to indicate a beat. This is the absolute time in milliseconds since the very first time of the first conducted beat. Note that there will be a slight offset between takes of the reverse conducting due to the zero time measurement. Time 0 is defined as the first reverse conducted beat rather than the first sonic event. Since the is no preparation for the first event, there will be a delay due to the reaction time of the reverse conductor.
4. The final column of raw data is the delta time in milliseconds between the current beat and the previous beat. This value can also be thought of as the duration of the previous beat, and can be used to calculate an instantaneous tempo value for the previous beat.

Here is raw data for the first reverse conducting trial run:

A total of 20 reverse conductings of the performance were done for this example mazurka performance. Here are links to each of the trials stored in separate files:

These individual files are combined into a single file, which is given below, by this PERL program. This file is then imported into a Mathematica notebook for analysis.

Data Analysis

The reason why 20 separate reverse conducting sessions were collected is due to the inaccuracy of conducting the beat along with a performance recording. This is especially true of solo instrument performance where the tempo can change in an erratic manner since there is no need for coordination between musicians. Reverse conducting an orchestral recording may be easier since the real conductor is contrained by the reaction times his performers, although they are usually prepared with visual cues as well as listening to each other.

Basic analysis and plotting of the reverse conducting data was done in Mathematica. Here is the Mathematica notebook used to examine the data:

• PDF Version
• HTML Version
• Mathematica Notebook

The Mathematica notebook outputs a file which contains average information about each beat in the music. The seven numbers on each line of the output file represent:

1. The expected position of the beat in the audio file in milliseconds.
2. The average beat duration for all trials.
3. The minimum beat duration in all trials.
4. The maximum beat duration in all trials.
5. The minimum duration of the true average beat with 95% confidence.
6. The maximum duration of the true average beat with 95% confidence.
7. The standard deviation of beat durations for all trials for the given beat.

This raw output file is then converted into a Humdrum compliant file with this PERL program by changing the spaces into tabs and adding useful score information such as barlines, beat information, and beat duration information.

Aligning the expected beat locations to the score

Now that the absolute times of the beats have ben determined, they can be incorporated into the score for the mazurka. This can be done by using a few Humdrum Toolkit commands as illustrated below.

First, start with a score which has the same number of beats as the performance (i.e., you will need to expand repeated sections in the score as necessary. The Mazurka in F minor, Op. 7, No. 3 does not have repeats, so the score can be used without modification. Here is the original score:

Next, use the minrhy program to determine the minimum rhythmic unit which is common between the score and the beat data:

# minrhy  mazurka07-3.krn  id5667230-10.dat
     mazurka07-3.krn:        48
     id5667230-10.dat:       4
     all:    48

Therefore, a 48 rhythm (triplet 32nd note) is the smallest rhythmic unit with which all other rhythms in any combination can be generated with an integral number of 48 rhythms. Now, extract only the absolute beat times (spine 3) from the Mathematica analysis data along with the beat rhythm information (spine 1):

# extract -f1,3 id5667230-10.dat > temp.dat

Now, coordinate the score and the Mathematica output data by making each line in the files equivalent to a 48 rhythm using the timebase program:

# timebase -t 48 mazurka07-3.krn > scoretb.krn
# timebase -t 48 temp.dat | extract -f2 >  temp2.dat

And then, use the assemble command to join the timing data to the score. The rid command is used to remove extra lines used for alignment which are no longer necessary.

# assemble temp2.dat scoretb.krn | rid -d > combined.krn

Now the timing data for each beat is aligned with the original score. Here is combined score which contains the timing data in the first spine (column):

Notice that each beat in the score is assigned an absolute time, but the off beats are not given any timing information. Be careful about aligning the timing information and the score in this manner because the timebase program will remove grace notes from the score. Since the score for this mazurka does not have grace notes encoded in it, there is no problem in this case.

Now absolute time data needs to be assigned to the offbeats. As an approximation, it will be assumed that the tempo of the music inside of the beat is constant. Use the gettime program to calculate the expected absolute time positions of the offbeats in the score:

# gettime -i combined.krn > fulltime.krn

The timing information from the above output is sent through the time2matlab program to generate the following output for analysis in matlab.

Evaluation

To get feedback on how accurate the reverse conductor was, it is now useful to take the absolute millisecond timings of each event in the score and add a click track to the actual performance. Adding the click track can be done in many ways. For the following examples, a command-line program called clicktrack was written:

• clicktract.exe -- Windows executable program
• clicktract.osx -- Mac OSX executable program
• clicktrack.cpp -- source code

This program takes two input files (the original audio, and a list of the clicks in milliseconds as a text file) and the output file name. An output file will be generated which contains the new click track in the left channel, and a mixture of all of the original channels (unless originally mono) in the right channel. The click track is generated by a one millisecond burst of white noise.

For example, here is how to create a click-tracked performance:

# clicktrack.exe input.wav clicktrack.txt output.wav

The click track from the average reverse conducting beats were subsequently corrected by ear in a sound editor, and the resulting positions were extracted from the audio click track with the following program:

• extractclicks.exe -- Windows executable program
• extractclicks.osx -- Mac OSX executable program
• extractclicks.cpp -- source code

Numerical Evaluation

• PDF Version
• HTML Version
• Mathematica Notebook

A side-by-side comparison of the two test performances are shown on this page.