Q: How to stream audio data to/from a file in real-time without glitching?
It’s a question that comes up on the PortAudio mailing list from time to time.
I’ve just published a paper on the use of message passing and lock-free queues to implement real-time-safe asynchronous file I/O. The paper presents one method for interfacing between real-time client audio streams and an I/O server thread. The paper will be of interest to developers wanting to implement their own real-time file streaming code. It also describes a number of lock-free tricks that are generally useful for real-time audio programming. In addition to the paper there are slides, an animation, and example source code linked below. Here’s the abstract:
Programming a computer to record or play a sound file in real-time is not as easy as it may seem. The naive approach is to call file I/O APIs from within the routine that handles real-time audio I/O. This leads to audible glitches whenever the time taken to access a file exceeds the time available to deliver a buffer of real-time audio. This paper describes an approach to streaming file playback and recording that operates correctly under these conditions. It performs file I/O in a separate thread, buffers audio data to mask file I/O delays, and uses asynchronous message passing and lock-free queues for inter-thread communication.
The full reference for the paper is:
Bencina, R. (2014) “Interfacing Real-Time Audio and File I/O,” Proceedings of the 2014 Australasian Computer Music Conference, ACMC 2014. Melbourne, Australia. July 9-11 2014.
A number of people generously gave feedback on earlier drafts of the paper: Tony Holzner, Scott Brewer, Phil Burk, Andrew Bencina and the anonymous reviewers — thanks guys, I appreciate it, your feedback was invaluable.
Here are links to the paper and supporting materials:
- The paper (pdf)
- The slides from my talk at ACMC 2014 (pdf)
- The animation used in the talk (html 5 using paper.js)
- The example code: github.com/RossBencina/RealTimeFileStreaming
- The lock-free queue data structures: github.com/RossBencina/QueueWorld
For background reading, you might be interested in these related blog posts:
- Real-time audio programming 101: time waits for nothing
- Programming with lightweight asynchronous messages: some basic patterns
- A brief meditation on two-party message exchange
- Asynchronous cancellation, abort and clean-up
If you have any questions or comments please feel free to get in touch, either by email, or in the comments below.
One comment
Hi Ross,
This is a fascinating article that I’ve been thinking about for a while. The lock-free queue implementation is very clever and I can see where this can definitely guarantee a non-blocking scenario between threads. I’m going to experiment with your RealTimeFileStreaming code so thank you for providing it to the community.
I have one question: Are there test cases where this method performs better than more traditional threading with condition variables and mutexes? For example Pure Data’s readsf~ uses locks and signaling via pthread, so does STK’s streaming classes I believe and probably Max’s sfread~ too. These seem to perform rather robustly and I have yet to experience glitches. Have we all just gotten lucky? — or has thread prioritization on modern OSes become more reliable? — or perhaps is the code so lightweight that a mutex has never failed to unlock before the parent thread needs it?
After reading your article I tried to “break” these objects to no avail. I’ll try harder. Thanks again for all the code, the article, portaudio, AudioMulch….etc.