High Bandwidth and the rapid development of social networks introduce a new way of making music together. The users of this new community are ambitious musicians and can log into online jam sessions to play together, overcoming the distance between them. Remote collaborative musical performances over the internet are already possible and there are numerous solutions out there. But all these solutions are dealing with one problem: Latency. This means, the audio signal of one user reaches the other user not in real time but lagged, which makes it impractical to make music together. It’s not feasible to eliminate Latency but it is feasible to reduce it. In this work I will define tolerable latency and I will outline possible problems which create latency in the audio conduction and recommend solutions, for keeping latency as low as possible.
This includes both client-side difficulties like bandwidth, real time audio compression, buffer times, and problems in the server-side streaming architecture like transport protocols, synchronization, packet loss, and delay times.
Table of Contents
Introduction
Chapter 1: A Short Market Review
Chapter 2: Maximum Tolerated Delay Times in Musical Collaboration
Chapter 3: How to reach Minimum Delay
3.1: End-System Delay
3.1.1: Digital Audio Processing
3.1.2: Real Time Audio Compression
3.1.3: Minimum Bandwidth
3.2: Network Delay
3.2.1: Casting Concepts
3.2.2: Transport Protocols
3.2.3: Latency
3.2.4: Audio Optimizing
Chapter 4: Conclusion
Research Objectives and Core Themes
The primary objective of this thesis is to identify and overcome the technological challenges inherent in networked collaborative musical performances, specifically focusing on the critical issue of latency. The paper seeks to establish the maximum tolerable delay thresholds for successful remote musical collaboration and to propose architectural and software-based solutions that maintain latency within these manageable limits.
- Analysis of existing market solutions for online jam sessions and their latency characteristics.
- Determination of psychoacoustic and technical boundaries for acceptable audio transmission delay.
- Evaluation of client-side factors, including audio hardware interfaces, buffer settings, and real-time compression codecs.
- Investigation of network-side factors such as casting concepts, transport protocols, and packet management.
- Development of best-practice strategies for minimizing end-to-end delay in high-fidelity audio streams.
Excerpt from the Thesis
CHAPTER 2: MAXIMUM TOLERATED DELAY TIMES IN MUSICAL COLLABORATION
In order to understand the basic principles of delay times in musical collaboration, first of all, we have to define the term “audio delay”. Delay or latency is the time lag between the incurrence of a sound and its actual reaching of our ears. (cf. Raffaseder, 2002, p. 218) The sound of a created sound event is spreading out as sound waves in the air. The dissemination via air takes a defined amount of time. Sound waves in the air have a speed of 340m/s. As a consequence, a sound wave lays back 1m in 1/340s, or 0.0029s, which is about 3msec. Lightning and thunder is a practical model illustrating delay time. Thunder is created by lightning. You see lightning and seconds after the incurrence of lightning you hear thunder. In other words, sound is much slower than light.
The impact of delay times in musical collaboration is exemplified in the following paragraph. Note that in a huge orchestral performance, musicians are positioned at a certain distance to each other, usually based on instrumental arrangement. A violinist and a trumpeter, for instance, are located at a distance of 15 meter; they hear the sound from each other in a time lag of 15m x 3msec, or 45msec. Consequently, musicians in an orchestral performance have to be experienced to overcome relatively high delay times and a conductor is absolutely indispensable for a synchronized performance in the right tempo.
Summary of Chapters
Introduction: Outlines the research question regarding the technical challenges of real-time musical collaboration over the Internet and provides a strategic overview of the thesis.
Chapter 1: A Short Market Review: Examines current software and hardware solutions for online jamming, discussing their technological approaches to latency and user experiences.
Chapter 2: Maximum Tolerated Delay Times in Musical Collaboration: Establishes the definition of audio delay and investigates psychoacoustic research to determine the maximum tolerable latency for musicians.
Chapter 3: How to reach Minimum Delay: Analyzes the dual sources of latency in client-side processing and network transmission, proposing hardware and software optimizations.
Chapter 4: Conclusion: Summarizes findings on end-to-end delay, concluding that distance and architectural constraints significantly limit current possibilities for global, low-latency networked jam sessions.
Keywords
Networked Collaborative Musical Performances, Latency, Audio Delay, Real Time, Buffer, Jitter, Packet Loss, CELT Codec, Streaming Protocols, End-System Delay, Propagation Delay, Audio Compression, ASIO, Peer-to-Peer, Synchronization
Frequently Asked Questions
What is the core problem explored in this thesis?
The thesis addresses the issue of latency in networked music performances, which prevents real-time musical collaboration because audio signals reach participants with a noticeable, impractical delay.
Which factors contribute most significantly to latency?
Latency is caused by two main areas: end-system processing (hardware buffers, CPU overhead, and audio compression) and network-related factors (propagation delay, routing, and switching).
What is the maximum tolerable delay for musicians?
The author identifies 50 milliseconds as the generally accepted limit for non-synchronized environments, while 75 milliseconds may be tolerable in specific synchronized setups.
What role does the CELT codec play in this research?
The CELT codec is highlighted as a highly efficient compression algorithm designed for low-delay, high-quality audio, offering performance superior to many other standard codecs in this context.
How does the author evaluate network casting concepts?
The author analyzes Peer-to-Peer, Unicast, Multicast, and Broadcast, concluding that while Peer-to-Peer helps avoid central server delays, it faces significant bandwidth and synchronization challenges.
What is the main takeaway for someone building a platform for online jamming?
Successful platforms must rigorously minimize end-system delay through high-quality audio hardware, optimized buffer settings, and low-latency codecs, while acknowledging that physical distance remains a hard constraint on performance.
Does the paper recommend specific audio drivers?
Yes, the author recommends the use of ASIO drivers, specifically the 'ASIO DirectX Full Duplex Driver' for Windows, to ensure the lowest possible latencies and minimal CPU overhead.
What are "Attack and release times" in the context of audio optimizing?
These refer to parameters in dynamic range control; improper configuration can lead to audible distortions or "pumping" effects, which are detrimental to the quality of a live audio stream.
- Quote paper
- Stephan Winkler (Author), 2010, How to Reach Minimum Delay Times in Networked Collaborative Musical Performances, Munich, GRIN Verlag, https://www.hausarbeiten.de/document/174676
