EP2899997A1

EP2899997A1 - Sound system calibration

Info

Publication number: EP2899997A1
Application number: EP14305092.0A
Authority: EP
Inventors: Michael Arnold; Peter Georg Baum; Xiaoming Chen; Ulrich Gries
Original assignee: Thomson Licensing SAS
Current assignee: Thomson Licensing SAS
Priority date: 2014-01-22
Filing date: 2014-01-22
Publication date: 2015-07-29

Abstract

A method, an apparatus and a system for efficient calibrating a multi loudspeaker sound system like a 5.1, 3D audio or home cinema system are disclosed, wherein watermarks distinct for each loudspeaker are transmitted in audio signals from the loudspeakers of the multi loudspeaker sound system and are used for calibrating the multi loudspeaker sound system. The watermarks are used for calibrating the multi loudspeaker sound system by adjusting sound pressure levels with respect to relative distance differences between the loudspeakers and the listening position. Relative distance differences between the loudspeakers and the listening position are determined by evaluating a relative propagation delay of the sound originating from the loudspeakers with correlation peaks in the received audio signal. In one embodiment, the watermarks are e.g. also used to calculate a relative propagation delay of the sound originating from each loudspeaker of the multi loudspeaker sound system with regard to a reference loudspeaker. The apparatus for calibrating the multi loudspeaker sound system comprises a receiving device correlating the watermarks und a system for upon request automated or manuell calibrating a multi loudspeaker sound system. An improvement with respect to the conventional adjustment method is disclosed, which no longer requires a sequential measurement and makes the setup or so-called calibration convenient for the user.

Description

FIELD OF THE INVENTION

The invention relates to a method, an apparatus and a system for efficiently calibrating a multi loudspeaker sound system like e.g. a 5.1 or 3D audio or so-called home cinema or even a home 22.2 multichannel sound system with UHDTV.

BACKGROUND OF THE INVENTION

In order to provide a perfect listener experience by a multi loudspeaker sound systems like 5.1 or 3D audio or so-called home cinema sound system, it is necessary to know the arrangement of the loudspeakers relative to the listener, which in general is different to the arrangement of loudspeakers for which the sound has been recorded. Furthermore, it is necessary for compensating a delay stemming from the fact that distances vary between loudspeakers and users at varying locations, the rooms in which these systems are arranged as well as the arrangement of loudspeakers in general differs from a desired arrangement. That means that these systems have to be calibrated in some way after installation. Conventionally, random noise is presented by each loudspeaker individually, and the random noise can be correlated, for the reference loudspeaker and the remaining loudspeakers. In this way, the different distances respectively time delays between each loudspeaker and the microphone at the listening position can be determined. Re-calibration is required after any change in the loudspeaker setup or varying a listening position. The conventional calibration cannot be done during normal audio playback or audio presentations, and must be done sequentially for each loudspeaker. This is time consuming, inconvenient for the user, and error prone.

SUMMARY OF THE INVENTION

It is an aspect of the present invention to propose a method, an apparatus and a system for efficient calibrating a multi loudspeaker sound system like a 5.1 or 3D audio or so-called home cinema sound system, which no longer requires a sequential measurement of the distances between loudspeakers and listening positions, and is executable with a mobile device, to make the setup or so-called calibration convenient for the user.
A method, an apparatus and a system for efficient calibrating a multi loudspeaker sound system with respect to a listening position are disclosed as determined in independent claims. Advantageous embodiments of the invention are disclosed in respective dependent claims. According to an aspect of the invention, a method for calibrating a multi loudspeaker sound system to a listening position is disclosed, wherein watermarks distinct for each loudspeaker are transmitted in audio signals from loudspeakers of the multi loudspeaker sound system and are used for calibrating the multi loudspeaker sound system.
The watermarks are simultaneously transmitted in the audio signals from the loudspeakers, however, according to a further embodiment of the invention, the watermarks distinct for each loudspeaker are transmitted in a serial order with a predetermined shift for calculating a relative propagation delay of sound originating from each loudspeaker of the multi loudspeaker sound system.
The watermarks distinct for each loudspeaker are watermark signatures specific for each one loudspeaker by a pattern embedded in each one channel of the multi loudspeaker sound system for identifying the loudspeaker playing back the respective channel.
According to further embodiments, a symbol pattern in a pattern individual for each loudspeaker by individual mark symbols at the same position in the symbol pattern, or a mark symbol at an individual position in the symbol pattern for each one of the loudspeakers of the multi loudspeaker sound system, or a combination of both is used for identifying the loudspeaker playing back a specific channel of the multi loudspeaker sound system.
Watermarks are pseudo randomly generated noise signals, which are e.g. generated with different seed values for each one of the loudspeakers of the multi loudspeaker sound system, or are signatures or symbol patterns designed to maximize orthogonality between signatures or symbol patterns specific for each loudspeaker of the multi loudspeaker sound system. The watermarks are synchronously embedded in the audio signals of each channel, or are embedded with a predetermined time difference to each other for calculating a relative propagation delay with regard to a specific user location being the desired listening position.
It is advantageous if all channels of the multi loudspeaker sound system reproduce simultaneously an audio signal with distinct watermarks for each one of the loudspeakers for calibrating the multi loudspeaker sound system.
It is furthermore advantageous in a first step to select one of the loudspeakers as a reference loudspeaker, which is arranged in a known or easy to determine distance to the listening position, or which according to further embodiments reproduces or is related to a standard sound pressure level at the listening position. In a second step, all channels of the multi loudspeaker sound system reproduce e.g. simultaneously an audio signal with distinct watermarks for each one of the loudspeakers for calculating a relative propagation delay of the sound originating from each one of the loudspeakers of the multi loudspeaker sound system with regard to the reference loudspeaker. That means that according to the disclosed method watermarks distinct for each loudspeaker in audio signals from the loudspeakers of the multi loudspeaker sound system are received via a microphone in a receiving device for detecting the watermarks by correlating with reference patterns individual watermarks in a correlator for identifying the loudspeakers with regard to the reproduced channel, and for measuring a relative propagation delay of the sound originating from the loudspeakers of the multi loudspeaker sound system. The measured relative propagation delay is used for adjusting the sound pressure level of the loudspeakers with respect to relative distance differences between each loudspeaker and the listening position. As the sound pressure level of the loudspeakers is determind according to an arrival of watermarks in the audio signal received with the microphone in the receiving device, the microphone of the receiving device should be arranged nearly congruent or held congruent with the listening position. Therefore, listening position and position of the microphone and position of the receiving device shall be understood in the description as the same location.
The sound system calibration based on watermarking uses the sound level of a reference loudspeaker at the listening position and the distance between reference loudspeaker and listening position. By using the distance between reference loudspeaker and listening position, the ratios of distances to all other loudspeakers are calculated if the distances to those loudspeakers are known. In turn the level difference for each one of the loudspeakers with respect to the reference loudspeaker is calculated by using a level equation as e.g. $Li = Lo - 20 \times \lg (di / do)$
wherein Li is the level difference of a loudspeaker with index i with respect to the level of the reference loudspeaker Lo, and di and do are the corresponding distances. The equation describes the situation of a spherical sound source in a free field propagation which is an approximation of the real situation in a room including reflections. Nevertheless other laws describing the sound pressure level dependence on the propagation distance as e.g. by incorporating directivity aspects of the loudspeakers are also applicable. That means, the calculation of level differences is mapped to the problem of determining differences in distances to the reference loudspeaker. This is done by using watermarking to embed an additional signal into each loudspeaker signal.
It is a further aspect of the invention to calibrate the multi loudspeaker sound system for listening positions deviating from a reference listening position. Watermarks distinct for each loudspeaker of the multi loudspeaker sound system are also used for such a calibration by measuring a relative propagation delay of the sound originating from the loudspeakers. This is performend in the same way as mentioned above as each listening position deviating from a reference listening position is a new listening position for which the relative propagation delay of the sound originating from the loudspeakers is measured. Distances to the loudspeakers in general change with a change of the listening position and in general, a multi loudspeaker sound system is calibrated only for one listening position. However, a compromise is possible by taking into account the changes in distances with respect to different listening positions for optimising the multi loudspeaker sound system e.g. with regard to two listening positions by averaging measured propagation delays for a calibration of the multi loudspeaker sound system.
It is a further aspect of the invention to provide an apparatus for calibrating a multi loudspeaker sound system with respect to a listening position, which according to an embodiment of the invention is a mobile device.
The apparatus is a receiving device receiving watermarks distinct for each loudspeaker transmitted in audio signals from the loudspeakers for identifying each one of the loudspeakers reproducing a channel of the multi loudspeaker sound system. Therefore, the receiving device is provided with a microphone receiving the watermarks distinct for each loudspeaker, a processor via an analog/digital converter connected with the microphone for at least temporarily recording the audio signals transmitted from the loudspeakers of the multi loudspeaker sound system, detecting the watermarks distinct for each one loudspeaker by correlating received watermarks with reference patterns and for determining a relative propagation delay of the audio signals originating from the loudspeakers, each reproducing a channel of the multi loudspeaker sound system, identified by the watermark in the reproduced audio signal. According to an embodiment of the invention, the receiving device is part of a remote control for controlling the multi loudspeaker sound system, wherein a controller is configured upon request to alter the sound pressure level provided by the channels and corresponding loudspeakers of the multi loudspeaker sound system at the listening position and position of the microphone respectively corresponding to relative propagation delays of the audio signals originating from the loudspeakers. The remote control is therefore provided with a calibration button for an automated calibration for one listening position or for an optimisation with regard to two listening positions.
According to a further embodiment of the invention, the receiving device is a mobile device like a smart phone or tablet personal computer with a microphone, a so-called phablet, comprising a display supplied by the processor of the mobile device for visualising a calibration of the audio signals reproduced by the loudspeakers of the multi loudspeaker sound system. The means for visualising the calibration is a software application configured to determine via a control element one of the loudspeakers of the multi loudspeaker sound system as a reference loudspeaker and is configured to determine a distance to the reference loudspeaker for calibrating the audio signals reproduced by the loudspeakers of the multi loudspeaker sound system. In one embodiment, the control element to determine one of the loudspeakers of the multi loudspeaker sound system as a reference loudspeaker and the configuration to determine a distance to the reference loudspeaker are input fields and the configuration to display the calibration of the audio signals reproduced by the loudspeakers is a bar graph visualised on the display. The bar graph visualises a sound pressure level or a propagation delay of the sound originating from loudspeakers of the multi loudspeaker sound system. In one embodiment, the receiving device provided with the display has in addition a transmitter for controling the sound pressure level received from each one of the loudspeakers at a listening position and according to a further embodiment without such transmitter the sound pressure level received from each one of the loudspeakers is manually adjusted by a user according to calibration results visualised on the display of the receiving device.
The system for efficiently calibrating a multi loudspeaker sound system comprises embedding in channels of the multi loudspeaker sound system watermarks individual for each loudspeaker which plays back the respective channel, for identifying the loudspeaker playing back the respective channel and measuring a relative propagation delay of the sound originating from the loudspeakers in a receiving device for adjusting sound pressure levels at a listening position with respect to relative distance differences between loudspeakers and listening position.
In one embodiment, the multi loudspeaker sound system comprises embedder upon request embedding in channels of the multi loudspeaker sound system watermarks individual for each loudspeaker which plays back the respective channel.
In a further embodiment, a recording medium like an optical disc or a memory stick carrying a watermarked multi-channel audio file is used for calibrating the multi loudspeaker sound system by playing back the recording medium with the multi loudspeaker sound system. Therefore, a multichannel audio file is split into separate channels, watermarks individual for each channel are embedded in each channel and a multi-channel encoder combines the watermarked audio channels into a multi-channel audio file recorded on the recording medium.
For a better understanding, the invention shall now be explained in more detail in the following description with reference to the figures. It is understood that the invention is not limited to the described embodiments and that specified features can also expediently be combined and/or modified without departing from the scope of the present invention as defined in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.
In the drawings:

Fig. 1: shows a schematic of a living room equipped with a multi loudspeaker sound system;
Fig. 2: illustrates by a schematic an example of a standard loudspeaker configuration and placement for playback;
Fig. 3: shows a schematic of a mobile device with a display for calibrating a multi loudspeaker sound system;
Fig. 4: shows a schematic of a remote control for calibrating and controlling a multi loudspeaker sound system;
Fig. 5: shows a schematic of a main device of a multi loudspeaker sound system;
Fig. 6: shows a schematic illustrating a software application configured for visualising a calibration of a multi loudspeaker sound system;
Fig. 7: shows a diagram illustrating a correlation for different loudspeakers,
Fig. 8: shows a flow diagram for embedding audio watermarks;
Fig. 9: shows a flow diagram for detecting audio watermarks;
Fig. 10: shows a schematic illustrating an analysis-synthesis framework for watermarking;
Fig. 11: shows a geometrical illustration of an embedding process based on a quantization regime;
Fig. 12: shows a geometrical illustration of an embedding process based on a constrained regime;
Fig. 13: shows a schematic illustrating the method and apparatus for pattern detection in the receiving device,
Fig. 14: shows a part of a circuit diagram of the main device of a multi loudspeaker sound system with embedder and
Fig. 15: shows a schematic illustrating the method and apparatus for manufacturing a recording medium for calibrating a multi loudspeaker sound system.

DETAILED DESCRIPTION OF PREFERED EMBODIMENTS

For a better understanding the invention shall now be explained in more detail in the following description with reference to the figures. It is understood that the invention is not limited to this exemplary embodiment and that specified features can also expediently be combined and/or modified without departing from the scope of the present invention as defined in the appended claims. Reference signs are unitarily used in drawings and formulas.
In the following the invention is explained with reference to Fig.1, which shows as an example a schematic of a typical living room equipped with a multi loudspeaker sound system like e.g. a 5.1 or 3D audio or home-theater system comprising a main device MLSS of the multi loudspeaker sound system, a center-channel loudspeaker CS, a main left-channel loudspeaker ML, a main right-channel loudspeaker MR, a left surround channel loudspeaker LS and a right surround channel loudspeaker RS. However, the disclosed calibrating system is also applicable and even more efficient for multi loudspeaker sound systems with much more channels and loudspeakers repectively as e.g. a home 22.2 multichannel sound system with UHDTV. In the exemplary embodiment illustrated in Fig. 1, the center-channel loudspeaker CS is arranged next to a television set TV and the four further loudspeakers ML, MR, LS, RS are arranged according to the side name as seen from a first or second listening position LP1 or LP2 as well as next to the walls of the room. The multi loudspeaker sound system is used to reproduce sound or speech provided by the main device MLSS of the multi loudspeaker sound system or provided by the television set TV connected to the main device MLSS of the multi loudspeaker sound system. However, having the right number of loudspeakers is not sufficient to provide a prefect listener experience if the loudspeakers are not at the right place or even not designed for the correct purpose. The International Telecommunication Union standard for multichannel recording and playback is very specific when it comes to loudspeaker configuration and placement for playback of multichannel recordings as e.g. illustrated in Fig. 2 for a 5.1 channel sound system. The 5.1 channel sound system has been specified in Recommendation ITU-R BS.775, which calls for five identical speakers placed in an arc around the centrally located listener LP. The center-channel is therefore a duplicate of the front left and right channel so they are the surrounds in this model. A comparison of the arrangements of loudspeakers shown in Fig.1 and Fig. 2 indicates immediately a discrepancy between a typical arrangement of the loudspeakers in a living room and an arrangement to guarantee that the listener hears what the recording was designed to present. In order to provide a perfect listener experience by a multi loudspeaker sound systems like 5.1, 3D audio or others, a calibration compensating misalignments and deviations with regard to the standard is therefore highly appreciated. The calibration shall be efficient and convenient for the user.
Therefore, according to the invention, a method, an apparatus and a system for efficient calibrating a multi loudspeaker sound system are disclosed, wherein a watermark distinct and therefore individual for each channel of the multi loudspeaker sound system is embedded in an audio signal reproduced with a loudspeaker LS, ML, CS, MR and RS of the multi loudspeaker sound system and is used for calibrating the multi loudspeaker sound system with a receiving device at a listening position LP1 or LP2 shown in Fig. 1. The sound system calibration based on watermarking uses a reference loudspeaker level Lo at the listening position LP1 or LP2 such as e.g. the first listening position LP1 and the distance do between the reference loudspeaker O such as e.g. the center-channel loudspeaker CS and the first listening position LP1. In the following formulas, an index i = O is assigned to reference values. Using the distance do, ratios of distances di/do to all other loudspeakers LS, ML, MR and RS, in the following indicated by an index i=1,...,M, can be calculated if differences Δ in distances Δdio = di - do, i=1,...,M are known. That means that the following equation is valid. $di / do = 1 + (Δdio / do)$
In turn the level difference Lio = Li - Lo for loudspeaker i with respect to a reference loudspeaker O can be calculated using the level equation $Li = Lo - 20 \times \lg (di / do) .$
The equation describes the situation of a spherical sound source in a free field propagation which is an approximation of the real situation in a room including reflections. Nevertheless other laws describing the sound pressure level dependence on the propagation distance can be used which e.g. incorporate directivity aspects of the sound sources and loudspeakers repectively.
Thus the calculation of the level difference Lio = Li - Lo is mapped to the problem of determining relative distance differences Δdio. This is done in a simultaneous manner by using watermarking to embed an additional signal into each loudspeaker signal.
Accordingly, a method for calibrating a multi loudspeaker sound system is disclosed, wherein different watermarks in different channels identifying the loudspeakers of the multi loudspeaker sound system are used for calibrating the multi loudspeaker sound system by simultaneously reproducing all the channels with the multi loudspeaker sound system and recording the result with a microphone MIC at the listening position LP1 or LP2 in a receiving device.
The structure of the receiving device is illustrated in Fig. 13. The user records a superposition of all audio signals, in the following denoted by z at the listening position LP1 or LP2 with a recording device using the microphone MIC in the receiving device illustrated Fig. 13. The receiving device incorporates a watermark detector which generates symbol patterns equivalent to watermarks embedded in the audio signals according to the number M of loudspeakers. After analog to digital conversion with an analog/digital converter ADC shown in Fig. 13 and whitening the signal with a whitening filter WF, the superposition of all audio signals z is correlated with each symbol pattern by a correlator CR connected to the whitening filter WF. Subsequently a decider like pattern detector PL in Fig. 13 decides whether correlation value ρ_z,_a illustrated in Fig. 7 is above a pre-determined threshold to determine the number of samples M to be moved along the buffer BUF arranged in front of whitening filter WF shown in Fig. 13. If the symbol is detected, the loudspeakers LS, ML, CS, MR and RS can be identified and the corresponding correlation lag ρ_z,_ai determined as e.g. shown in Fig. 7 for a correlation reference loudspeaker ρ_z,_ao and the one of the loudspeakers having determined as the first one ρ_z,a1 as e.g. right surround channel loudspeaker RS. Differences in the occurrence of correlation peaks between reference loudspeaker O as for example the center-channel loudspeaker CS and all other loudspeakers RS, LS, ML and MR determine the propagation difference in samples Smpl. By determining correlation lags di, i=1,...,M of the peaks in the correlation array in samples Smpl, the relative distance difference Δdio = di - do, i=1,...,M between the reference loudspeaker O and all the other ones is determined. For example, if the watermark detection device receives the sound over an acoustic path with a distance difference of Δdio^s samples with the superscript s for samples, the propagation distance with the superscript m for meter is Δdio^m = Δdio^s×c×T meters, where c×T is the distance the sound propagates in one sampling interval, with the sampling rate = 1/f, the sampling frequency f and the speed of sound in air c. With respect to the sound pressure level Lo of the reference loudspeaker O, such as e.g. the centre loudspeaker CS in Fig. 1, all the remaining loudspeaker levels Li can be adjusted according to the level equation mentioned above.
Therefore, a configuration of a receiving device like a mobile device with a microphone MIC is disclosed, which has to be placed at the listening position LP1 or LP2, and generates information about relative distance differences Δdio between the loudspeakers LS, ML, CS, MR and RS and listening posion LP1 or LP2 for adjusting the sound pressure level of the loudspeakers LS, ML, CS, MR and RS with respect to listening posion LP1 or LP2. According to the structure of the examplary embodiment of a receiving device illustrated by a block diagram in Fig. 13, the receiving device is provided with a microphone MIC via an analog/digital converter ADC connected with a buffer BUF for temporally storing the audio samples captured with the microphone MIC. The buffer BUF is via a whitening filter WF connected to a correlator CR and the whitening filter WF supplies the correlator CR for detecting a base pattern respectively a mark symbol by a feedback loop checking the number N of base pattern as e.g. a base pattern at different locations or different patterns included in the watermark signal. Therefore, the whitening filter WF applies a window to the buffered digital audio signal and transforms blocks of audio samples into a frequency domain by applying a Discrete Fourier Transformation DFT, the magnitude of Fourier coefficients is set to 1 and the whitening filter WF transforms the blocks by applying an inverse Discrete Fourier Transformation IDFT back into the time domain and applies a window to provide overlap-add blocks for correlation. The correlator CR correlates the filtered audio blocks with all mark symbols N of the different audio watermarks in a mark symbol loop MSL, and a connected pattern detector PL detects peaks and locations of symbols in patterns in the number M of used patterns and individual watermarks, respectively, for calculating corresponding time delays between arrivals of mark symbols and base patterns respectively.
That means that the pattern detector PL provides a pattern which in a further feedback loop supplies the input of the buffer BUF with a number M of pattern blocks for relating a detected pattern to further received patterns, so that individual patterns provided by corresponding sound sources as e.g. loudspeakers LS, ML, CS, MR and RS will be assigned to a corresponding channel of the multi loudspeaker sound system. That means that the pattern detector PL provides individual patterns with a mark symbol which identifies the one of the loudspeakers LS, ML, CS, MR and RS which transmitted the pattern. Furthermore, an output unit OD is connected to the pattern detector PL, wherein a calculation of relative distance differences and sound pressure level is performd as mentioned above.
According to an embodiment of the invention illustrated in Fig. 4, the receiving device is part of a remote control RC for controlling the multi loudspeaker sound system. Except the microphone MIC, the elements of the receiving device are realised in a controller of the remote control RC configured upon request to alter the sound pressure levels provided by the channels and corresponding loudspeakers LS, ML, CS, MR and RS of the multi loudspeaker sound system at the listening position LP1 or LP2.
The listening position LP1 or LP2 and position of the microphone MIC are assumed as the same position in this description.
Sound pressure levels are altered corresponding to a relative propagation delay of the audio signals originating from the loudspeakers LS, ML, CS, MR and RS.
The remote control RC is provided with a calibration botton for an automated calibration for one listening postion LP1, or for an optimisation with regard to two listening positions LP1 and L2, by taking into account the changes in distances with respect to different listening positions LP1 and LP2. Such optimisation is performed by averaging propagation delays measured with respect to the different listening positions LP1 and LP2.
In a further embodiment, calibration parameter measured and calculated for one or more listening positions LP1, LP2 are stored and assigned to a user profile for a fast calibration of the multi loudspeaker sound system, so that a calibration has not to be repeated.
According to a further embodiment of the invention, the receiving device is a mobile device MD like a smart phone or tablet personal computer with a microphone, a so-called phablet shown in Fig. 3. The mobile device MD shown in Fig.3 is provided with a display supplied by a processor of the mobile device MD for visualising the calibration of the audio signals reproduced by the loudspeakers LS, ML, CS, MR and RS of the multi loudspeaker sound system. The means for visualising the calibration is a software application APPMD illustrated in Fig. 6. The software application APPMD is configured to determine via a control element one of the loudspeakers LS, ML, CS, MR and RS of the multi loudspeaker sound system as a reference loudspeaker O and furthermore configured to determine a distance to the reference loudspeaker O for calibrating the audio signals reproduced by the loudspeakers LS, ML, CS, MR and RS of the multi loudspeaker sound system.
In one embodiment, the control element to determine one of the loudspeakers LS, ML, CS, MR and RS of the multi loudspeaker sound system as a reference loudspeaker o and the configuration to determine a distance do to the reference loudspeaker O are input fields and the configuration to display the calibration of the audio signals reproduced by the loudspeakers LS, ML, CS, MR and RS is a bar graph Level visualised on the display of the mobile device MD. The bar graph Level visualises a sound pressure level or a propagation delay of the sound originating from loudspeakers LS, ML, CS, MR and RS of the multi loudspeaker sound system after selecting one of the loudspekers LS, ML, CS, MR and RS.
In one embodiment, the receiving device provided with the display has in addition a transmitter for controling the sound pressure level Li received from each one of the loudspeakers LS, ML, CS, MR and RS at a listening position LP1 or LP2 via the main device MLSS of the multi loudspeaker sound system.
According to a further embodiment without such transmitter, the sound pressure level Li received from each one of the loudspeakers LS, ML, CS, MR and RS is manually adjusted by a user according to calibration results visualised on the display of the mobile device MD.
According to a further embodiment, the control element to determine one of the loudspeakers LS, ML, CS, MR and RS of the multi loudspeaker sound system as a reference loudspeaker O and the configuration to display a calibration of the audio signals on the display of the receiving device are realized with radio button or check boxes which indicate a selected loudspeaker and a calibration of remaining loudspeakers of the multi loudspeaker sound system. That means that the receiving device comprises a correlator for correlating the watermarks distinct for each loudspeaker of the multi loudspeaker sound system to identify the loudspeakers each reproducing a channel of the multi loudspeaker sound system and means for evaluating the corresponding sound pressure level at the listening position LP1 or LP2.
The system for efficiently calibrating a multi loudspeaker sound system comprises embedding in channels of the multi loudspeaker sound system watermarks individual for each loudspeaker which plays back the respective channel.
For embedding watermarks, the audio signal is mapped into a time-frequency representation before embedding a watermark. It is common practice in audio processing to apply a short-time Fourier transform to obtain a time-frequency representation of the signal so as to mimic the behaviour of the ear. As shown in Fig. 10, the short-time Fourier transform consists of segmenting an input signal x in blocks samples of a predetermined length using a sliding window with a hop-size of a predetermined number of samples, and applying the Discrete Fourier Transformation DFT to each block after multiplication by an analysis window WA. This analysis phase results in a collection of DFT-transformed windowed blocks X̃_n which is fed to the subsequent watermark embedding process Embedding. In the embedding process Embedding modified DFT-transformed blocks Ỹ_n , output by the audio processing application are used to reconstruct the audio signal during the so-called synthesis phase. In a nutshell, the frames are inverse-transformed and multiplied by a synthesis window w_s that suppresses audible artifacts by fading out spectral discontinuities at frame boundaries. The resulting frames are added together with the appropriate time offset as depicted in Fig. 10. Fig. 10 illustrates the analysis-synthesis framework, wherein the audio signal is mapped into a time-frequency representation before embedding a watermark, at which point the signal is mapped back to the time domain. The combination of the segmenting-windowing-DFT in the analysis phase and the IDFT-windowing overlap-add in the synthesis phase is the so-called weighted overlap add WOLA technique. We will refer to the coefficients in between the analysis and synthesis phases as WOLA coefficients.
The embedding process for embedding into the phase as illustrated in Fig. 11 and Fig. 12, essentially comprises extracting the phase ϕ_n of WOLA coefficients from incoming transformed blocks X̃_n and arranging them sequentially in a 1-D signal ϕ, applying a quantization based embedding algorithm to obtain the watermarked phases ψ, and segmenting the resulting signal in blocks of samples ψ_n to reconstruct the watermarked transformed blocks Ỹ_n , which can be subsequently inverse-transformed back to the time domain.
Assuming that the system embeds symbols taken from an Mary alphabet A, the embedding process can be written: ψ [i]= θ_(a_k) [i], a_k ∈ A, i ∈ B·N + [0, B-1], where θ_(a_k) is a sequence of angles associated with the symbol a_k and derived from a reference signal r (a k), i is the index in the 1-D signal ϕ, B is the number of samples in one block and N is the number of blocks for one symbol.
This pure replacement strategy needs to be adjusted to avoid introducing audible artifacts. Samples outside a specified frequency band are left untouched i.e. ψ[i] = ϕ[i], I ∈ B·N + {0,ζ₁ U ζ_h,B/2}.
Frequencies below frequency tap ζ_l are discarded due to their high audibility whereas frequencies above frequency tap ζ_h are ignored because of their high variability.
For the remaining angles, the embedding process is revised to guarantee that the embedding distortion, as measured by the angle difference δ[i]=|ψ[i]-ϕ[i]|, remains below some perceptual margin
[i] ∈ [0,π]. Enforcing such psychoacoustic constraints guarantees that the introduced changes remain inaudible. This is illustrated in Fig. 11 and Fig. 12 and can be formally written as: $ψ [i] = φ [i] + (δ [i] / |δ [i]|) \min \{|δ [i]|, υ [i]\}, i \in B \cdot N + ζ_{1}, ζ_{h}$

where δ[i]= θ_(a_k) [i]-ϕ[i] is the forecast embedding distortion in case of perfect quantization.
Fig. 11 and Fig. 12 are a geometrical illustration of the embedding process. The phase ϕ[i] is moved towards the intended target value θ _ak[i] while guaranteeing that the embedding distortion δ[i] never exceeds the threshold υ[i] specified by the perceptual model. In a quantization regime as shown in Fig. 11, the target angle is close enough to be reached, whereas in a limited regime, the embedding process is constrained by the perceptual margin.
At the receiver side in the receiving device, the detector is presented with audio blocks z that contain a watermark, and that may have undergone a number of modifications. To assess the presence of a symbol, the detector computes the following cross-correlation score - see correlator CR in figure 13 - for all symbols a_k for a special design of the symbol pattern in the alphabet: $ρ_{z, a_{k}} [l] = \frac{1}{B \cdot N} \sum_{i = 0}^{B \cdot N - 1} \overset{‿}{z} [i] {\overset{⁀}{r}}_{a_{k}} [i + 1], l \in 0, B \cdot N - 1$
where
(resp.
_ak) is the whitened version of z(resp.r_ak) and I is the correlation lag measured in samples. The whitening process in the whitening filter WT in Fig. 13 consists in mapping the signal to the WOLA domain, setting the magnitude of the WOLA coefficients to 1, and returning to the time domain.
In the absence of a watermark, the array ρ_{z,a_k} is expected to be close to normal distributed with zero mean. In contrast, if the symbol a_k is embedded, it should exhibit a strong peak for a given correlation lag l_k*, whose position depends on the alignment of the two signals. The detection procedure of the watermarking system therefore first isolates the position and the amplitude of the correlation peak for each symbol, $l_{k}^{*} = \arg \max_{l} |ρ_{z, a_{k}} [l]| and ρ_{k}^{*} = |ρ_{z, a_{k}} [l_{k}^{*}]|,$
and then identifies which symbol exhibits the largest peak: $\hat{a} = a_{k *}, k * = \underset{k}{arg max} ρ_{k}^{*} .$
If the peak ρ_k** is larger than some detection threshold τ_detect in ther pattern detector PL in Fig. 13, the symbol a_k* is decoded; otherwise the detector reports that there is no watermark.
In case of multiple loudspeaker i = 1,...,M setup rendering a multi-channel sound, a symbol pattern r_ (a i), i = 1,...,M will be embedded in each individual channel because the size of the alphabet A is M. This watermark identifies the loudspeaker playing back the respective channel. To ensure a reliable determination of the individual channels, the symbol pattern for each channel is designed to maximize orthogonality ρ(a_i,a_j)∼ 0 ∀ i ≠ j.
This requires a set of symbol patterns that have good code separation as e.g. patterns that are far apart from each other in marking space. In correlation based methods good code separation is equivalent to having low correlations ρ(a_i,a_j) ∼ 0 ∀ i ≠ j between different symbol patterns. In an example system, the number M of symbol patterns equivalent to the number of loudspeakers is very small compared to the dimensionality of the marking space. These random generated symbol patterns are very likely to be orthogonal to one another.
The method for embedding audio watermarks different and unique for each loudspeaker is illustrated by a flow chart in Fig. 8 which illustrates that the audio signal at first is segmented SEG in overlapping blocks of samples and a sliding window is applied - a so-called windowing WIN - to avoid interruptions and to equalise overlapping parts of the blocks of audio signals. Discrete Fourier Transformation DFT is applied to each block, which results in a collection of DFT-transformed windowed blocks which are fed to the subsequent watermark embedding process comprising a modulation MOD with symbol patterns provided by a symbol pattern generator SPG. Symbol pattern generator SPG is supplied by a noise generator NG providing different pseudo random noise signals according to a number of mark symbols N and a number M of symbol patterns and individual watermarks respectively. Mark symbols N are pseudo random noise patterns generated with a specific seed, and different symbol patterns and watermarks respectively are patterns generated with different mark symbols at the same or different position in a pattern, or different symbol patterns are patterns generated with the same mark symbol at a different location in the pattern. The audio signal is modulated with symbol patterns in the Fourier domain of the audio signal and afterwards by an inverse Discrete Fourier Transformation IDFT reconstructed during a so-called synthesis phase. The blocks are inverse-transformed and multiplied by a synthesis window WIN that suppresses audible artefacts by fading out spectral discontinuities at frame or block boundaries. The resulting blocks are added together OA with the appropriate time offset and mapped back to the time domain.
In one embodiment illustrated in Fig. 14, the main device MLSS of the multi loudspeaker sound system comprises embedders EB upon request embedding in channels of the multi loudspeaker sound system watermarks individual for each loudspeaker which plays back the respective channel. Multichannel audio files are split into separate channels 1..M in the main device MLSS of the multi loudspeaker sound system and switches SW1, ... SWM are provided for upon request embedding an individual watermark in each channel. The output of each embedder EB is connected with a volume controller VA for calibrating the sound pressure level provided by the corresponding loudspeaker LS1 ...LSM at the listening position LP1 or LP2, or for a, with respect to two listening positions LP1 and LP2, optimized reproduction of sound or speech with the multi loudspeaker sound system.
According to an embodiment, the volume controller VA is in an automated manner controlled by a receiving device transmitting results of a measured relative propagation delay of the sound originating from the loudspeakers of the multi loudspeaker sound system to the main device MLSS of the multi loudspeaker sound system.
In a further embodiment illustrated in Fig. 15, a recording medium DP like an optical disc or a memory stick carrying a watermarked multi-channel audio file is used for calibrating the multi loudspeaker sound system, by playing back the recording medium with the multi loudspeaker sound system. For preparing the recording medium DP for calibrating the multi loudspeaker sound system, a multi channel audio file is split into separate channels in an exemplary computer device and watermarks individual for each channel are embedded in each channel. The exemplary computer device comprises a multi-channel encoder which combines the watermarked audio channels WMC 1 ...WMC M into a multi-channel audio file MCE for recording the watermarked multi-channel audio file WMF on the recording medium DP.
That means that watermarks distinct for each loudspeaker are transmitted in audio signals from loudspeakers of the multi loudspeaker sound system and are used for calibrating the multi loudspeaker sound system.
Similar to the audio watermarks generated in the audio watermark embedder EB, reference patterns are generated by a reference pattern generator RPG in the audio watermark detector of the receiving device as illustrated in the flow diagram for detecting audio watermarks shown in Fig. 9. For detecting audio watermarks as shown in Fig. 9, a random phase generator RP preferably controlled by a key SK is connected to reference pattern generator RPG for a correlation of all reference patterns as generated by the audio watermark embedders EB. That means that the correlator CR of the audio watermark detector of the receiving device correlates whitened audio signals with all reference patterns of the audio signals distributed to loudspeakers L1 to Ln to perform a symbol detection SD so that a detected symbol DS is provided for further processing.
If a symbol is embedded in a received audio stream, the correlator CR exhibits a strong peak for a given correlation lag, whose position depends on the location of the symbol in the watermark and the time of arrival. The detection procedure of the watermarking system therefore first isolates the position and the amplitude of the correlation result peak for each symbol, and then identifies which symbol exhibits the largest peak. If the peak is larger than some detection threshold in the pattern detector PL shown in Fig. 13, the symbol is decoded, otherwise the audio watermark detector reports that there is no watermark and nothing will be further executed. The watermark detection method, illustrated by a flow diagram in Fig. 9, starts with an acquisition AQUI of watermarked audio signals WAS with the microphone MIC, analog/digital converting and buffering in the receiving device. The connected whitening filter WF processes audio samples stored in the buffer BUF to segment and window the samples in blocks, performs a transformation of the blocks into a frequency domain, setting a magnitude equal to 1 and a transformation back into the time domain with overlap-add blocks to provide a collection of audio blocks for correlation. That means that the whitening process performed in the whitening filter WF consists in mapping the signal to the WOLA domain, setting the magnitude of the WOLA coefficients to 1, and returning to the time domain. To assess the presence of a symbol as a mark symbol or a zero base pattern, the audio watermark detector of the receiving device computes a cross-correlation score for all symbols so that a correlation lag is determined by the number of samples between the symbols to measure and to determine a delay in arrival. The multi loudspeaker sound system is calibrated by adjusting the channels and loudspeaker levels according to a propagation delay with means of the multi loudspeaker sound system by checking a propagation delay of the audio signals originating from the loudspeakers. Adjusting sound pressure levels or a time delay with respect to differences in the audio signal received from loudspeakers of the multi loudspeaker sound system at a listening position LP1 or LP2 in comparision to a reference listening position is disclosed.
Watermarks distinct for each loudspeaker are detected by correlating sound recorded in the receiving device with known watermark sequences, and sound pressure levels and a time delay are calibrated with respect to relative distance differences between loudspeakers and the listening position LP1 or LP2.
The disclosed calibration is efficient due to a simultaneous determination of propagation delay respective distance differences between loudspeaker and microphone position. The duration needed for a calibration of a multi loudspeaker sound system is independent on the number of loudspeakers used in the setup. An automatic adaptation of playback sound pressure level with respect to the distance of the listening position LP1 or LP2 relative to the each loudspeaker is provided and according to an embodiment of the invention even the complete calibration procedure will be performed in an automated manner upon user request. Any sound can be used for calibrating and the inaudibility of the embedded watermarks is ensured by a psychoacoustic model. That means that an improvement with respect to the conventional adjustment method for a sound system consisting of multiple loudspeakers with respect to an individual listening position LP1 or LP2 is disclosed, which no longer requires a sequential measurement of the distances between loudspeakers LS, ML, CS, MR and RS and listening positions LP1 or LP2 or LP1 and LP2. The calibration is executable with a mobile device and makes the setup or so-called calibration convenient for the user.
Although the present invention has been described in terms of the presently preferred embodiment, it is to be understood that such disclosure is not to be interpreted as limiting. Various alternations and modifications will no doubt become apparent to those skilled in the art after reading the above disclosure. Accordingly, it is intended that the appended claims be interpreted as covering all alternations and modifications as fall within the true spirit and scope of the claims.

Claims

Method for calibrating a multi loudspeaker sound system to a listening position (LP1 or LP2), wherein a watermark distinct for each channel of the multi loudspeaker sound system is embedded in an audio signal reproduced with a loudspeaker (LS, ML, CS, MR and RS) of the multi loudspeaker sound system and is used for calibrating the multi loudspeaker sound system with a receiving device at the listening position (LP1 or LP2).
The method according to claim 1, wherein the watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) are watermark signatures individual for each loudspeaker (LS, ML, CS, MR and RS) by a pattern embedded in each channel for a loudspeaker (LS, ML, CS, MR and RS) of the multi loudspeaker sound system identifying the loudspeaker playing back the respective channel.
The method according to one of the claims 1 to 2, wherein the watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) are symbol patterns in a pattern individual for each loudspeaker (LS, ML, CS, MR and RS) by individual mark symbols at the same position in the symbol pattern, a mark symbol at an individual position in the symbol pattern or a combination of both for each of the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system.
The method according to one of the claims 1 to 3, wherein the watermarks distinct for each one loudspeaker (LS, ML, CS, MR and RS) of the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system are pseudo random generated noise signals.
The method according to one of the claims 1 to 4, wherein the watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) are noise signals generated with different seed values for each of the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system.
The method according to one of the claims 1 to 5, wherein the watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) are watermarks synchronously embedded in audio signals in each channel of the multi loudspeaker sound system.
The method according to one of the claims 1 to 5, wherein the watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) are watermarks at a predetermined time or with a predetermined time difference embedded in audio signals applied to loudspeakers (LS, ML, CS, MR and RS) playing back the channels of the multi loudspeaker sound system.
The method according to one of the claims 1 to 7, wherein all channels of the multi loudspeaker sound system reproduce simultaneously an audio signal with watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) for calibrating the multi loudspeaker sound system by adjusting sound pressure levels with respect to relative distance differences between the loudspeakers (LS, ML, CS, MR and RS) and the listening position (LP1 or LP2).
The method according to one of the claims 1 to 8, wherein for calibrating the multi loudspeaker sound system with respect to a listening position (LP1 or LP2),
in a first calibration step one of the loudspeakers (LS, ML, CS, MR and RS) is selected as a reference loudspeaker and a distance between listening position (LP1 or LP2) and reference loudspeaker is determined and
in a second calibration step all channels of the multi loudspeaker sound system reproduce simultaneously an audio signal with watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) to calculate a relative propagation delay of the sound originating from each loudspeaker (LS, ML, CS, MR and RS) of the multi loudspeaker sound system with regard to the reference loudspeaker.
The method according to one of the claims 1 to 9, comprising:
receiving via a microphone watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) in audio signals reproduced by the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system,

detecting the watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) by correlating the watermarks in a receiving device for identifying the loudspeakers (LS, ML, CS, MR and RS) with regard to the reproduced channel of the multi loudspeaker sound system and

evaluating a relative propagation delay of the sound originating from the loudspeakers (LS, ML, CS, MR and RS) with correlation peaks in the received audio signal by a controller of the receiving device for adjusting sound pressure levels or a time delay with respect to differences in the audio signals received from the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system at the listening position (LP1 or LP2).
The method according to one of the claims 1 to 10, wherein:
one of the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system is selected as reference loudspeaker (o) arranged in a certain distance (do) to the listening position (LP1 or LP2) and provides a certain audio level (Lo) at the listening position (LP1 or LP2) for indicating a sound pressure level (Li) or a time delay with respect to differences in the audio signals received from remaining of the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system.
The method according to one of the claims 1 to 11, wherein:
an audio level of a reference loudspeaker (o) is adjusted to a predetermined audio level (Lo) at the listening position (LP1 or LP2),

a distance (do) to the reference loudspeaker (o) is determined and

sound pressure level (Li) or delays with respect to differences in the audio signals received from the remaining of the loudspeakers (LS, ML, CS, MR and

RS) at the listening position (LP1 or LP2) are calibrated by

evaluating distances between correlation peaks of the watermarks in the audio signals received at the listening position (LP1 or LP2)
Apparatus for calibrating a multi loudspeaker sound system with respect to a listening position (LP1 or LP2) comprising:
a receiving device, receiving watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) transmitted in audio signals from the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system

for identifying each loudspeaker (LS, ML, CS, MR and RS) reproducing a channel of the multi loudspeaker sound system and

evaluating distances between correlation peaks of the watermarks in the audio signals received at the listening position (LP1 or LP2) for calibrating the multi loudspeaker sound system.
The apparatus according to claim 13, wherein the receiving device is provided with a microphone receiving the watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) to identify each loudspeaker (LS, ML, CS, MR and RS) of the multi loudspeaker sound system with regard to a channel reproduced in audio signals from each loudspeaker (LS, ML, CS, MR and RS) received at the listening position (LP1 or LP2),
a processor via an analog/digital converter connected with the microphone for recording the audio signals transmitted from the loudspeakers (LS, ML, CS, MR and RS), detecting the watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) by correlating the watermarks and determining with distances between correlation peaks a propagation delay of the audio signals originating from the loudspeakers (LS, ML, CS, MR and RS) each reproducing a channel of the multi loudspeaker sound system for calibrating the multi loudspeaker sound system.
The apparatus according to one of the claims 13 to 14, wherein the receiving device is a remote control (RC) of the multi loudspeaker sound system comprising a calibration button for performing an automated calibration of the multi loudspeaker sound system with respect to the listening position (LP1 or LP2), being the location of a microphone (MIC) of the receiving device, by initiating an embedding of a watermark distinct for each channel of the multi loudspeaker sound system in an audio signal reproduced with a loudspeaker (LS, ML, CS, MR and RS) of the multi loudspeaker sound system and for transmitting a control signal adjusting sound pressure levels (Li) of the loudspeakers with respect to relative distance differences between the loudspeakers (LS, ML, CS, MR and RS) and the listening position (LP1 or LP2).
The apparatus according to one of the claims 13 to 14, wherein the receiving device is a mobile device (MD) carrying a display for visualising a calibration of the multi loudspeaker sound system.
The apparatus according to claim 16, wherein the calibration of the multi loudspeaker sound system is visualized by a software application (APPMD) installed in and displayed on the display of the mobile device (MD).
The apparatus according to claim 17, wherein a bar graph on the display visualises a sound pressure level or a propagation delay of the sound originating from selectable loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system.
A system for calibrating a multi loudspeaker sound system with respect to a listening position (LP1 or LP2), comprising:
embedding watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) in audio signals transmitted from the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system,

receiving and recording the transmitted audio signals in a receiving device and

correlating the watermarks in the audio signals

for identifying each loudspeaker (LS, ML, CS, MR and RS) of the multi loudspeaker sound system and

for determining a propagation delay between sound originating from the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system in the receiving device,

for calibrating the multi loudspeaker sound system with respect to the listening position (LP1 or LP2).
The system according to claim 19, comprising a recording medium (DP) carrying a watermarked multi-channel audio file used for embedding watermarks distinct for each loudspeaker (LS, ML, CS, MR and RS) in audio signals transmitted from the loudspeakers (LS, ML, CS, MR and RS) of the multi loudspeaker sound system for calibrating the multi loudspeaker sound system by playing back the recording medium (DP) with the multi loudspeaker sound system.