Note Onset Localization Experiment

This webpage describes an experiment to determine the perception time range of note onset.

Experimental setup

The test material consists of a recording of a piano note on middle C. The soundfile was provided by the Musical Instrument Samples collection at the University of Iowa Electronic Music Studios. The piano note is played in the right channel of a stereo soundfile with the attack starting 500 milliseconds from the start of the file.

In the left channel, a 1 millisecond sample of white noise is played with approximately the same loudness as the attack of the piano in the right channel. This click is moved before and after the attack of the piano note in a random location between -100 and +100 milliseconds around the note attack in increments of 5 milliseconds. Here is a plot showing the 21 possible click locations in the left channel, along with the piano note in the right channel with a zoom onto the interesting region shown on the right (click to enlarge):

Listeners are asked to identify whether the click in the left channel comes before or after the note attack of the piano sound in the right channel.

The conditions for acquiring the data for the experiment are:

• The listener uses headphones.
• The listener listens to a randomly selected click offset typically in the range from -100 to +100 milliseconds in increments of 5 milliseconds.
• The listener writes down whether the click comes before or after the piano note attack.
• The correct answer is not revealed to the listener.
• The soundfile is played only once and the listener then gives an answer. If the listener cannot decide on an answer (for example, they are not paying attention or an external noise occurs) they can skip that particular test.

Results

The following plot shows the identification rates for each time offset from the note attack. The orange line shows the fraction of responses for the click occurring before the piano note, and the blue line shows the responses for the click occurring after the piano note.

When the click is further than 50 milliseconds from the note attack start, then the accuracy rate for distinguishing between before and after is nearly 100%. The click can reliably be identified (at a 75% rate) before or after the beat outside of the range from -21.5 to +22.0 milliseconds based on the current data and approximate fit.

The 67% accuracy points (getting 2 out of 3 test correct) are -14.0 to 14.7 milliseconds. Any clicks inside of this range cannot be accurate identified as being before or after the piano note attack.

The 50% accuracy point measured from the fitted curve occurs at 0.4 milliseconds from the start of the piano note attack.

In terms of cycles of the fundamental frequency (middle C), the click orientation ambiguity can occur anywhere within about +/- 6 cycles of the fundamental from the start of the attack; however, this region is probably independent of the fundamental frequency.

The following plot superimposes the predetermined alignment quality categories used for evaluating the accuracy of reverse conducting and subsequent automatic alignment. Region A is anything within 20 milliseconds of the true location of an event, region B is anything within 20-40 ms, region C is anything within 40-80 ms, and region D is anything within 80-160 ms of the true event time.

Redefining the regions to be doublings of 15 ms might work well because anything within 15 ms of the true event cannot be heard as distinct events. 15-30 can be somewhat reliably identified audible. 30-60 is fairly reliable for distinguishing event order, and greater than 60 is 100% reliable. Of course, this was a laboratory test, so keeping the range slightly wider at doublings of 20 ms is probably more representative in real situations.

Raw Material

• responses.dat -- Raw answers for various click offsets.
• countdata -- a PERL program to read the responses.dat file and generate a summary.
• pianonote.wav -- a monophonic soundfile which contains a piano note, middle C, with the attack starting at 1000 milliseconds into the soundfile.
• runonsettest2 -- a PERL program to run the experiment.
• sounder.exe -- a simple command-line soundplayer for Windows used by the runonsettest2 program.
• clicktrack.exe -- a Windows command-line program for generating a stereo clicktrack.
• noteonsetlocalization.nb -- Mathematica notebook for plots.

Further Work

Further Reading