WO2008040250A1

WO2008040250A1 - A method, a device and a system for error concealment of an audio stream

Info

Publication number: WO2008040250A1
Application number: PCT/CN2007/070772
Authority: WO
Inventors: Hualin Wan; Zhe Wang; Jun Zhang
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2006-10-01
Filing date: 2007-09-25
Publication date: 2008-04-10
Also published as: CN101155140A

Abstract

A method for concealing the error in the audio stream is provided which includes the following steps: classifying the audio frame which has been transmitted based on its contents and getting the type information of the audio frame; packaging the type information of the audio frame with the encode results of the audio frame and transmitting them; when losing frame, for the lost audio frame, determining the type information of the audio frame which has been got by classifying it based on its contents; according to the type information of the lost audio frame, reconstructing the audio frame by using the corresponding error recovery strategy. The said error concealing mode has better pertinency to the reconstruction of the lost frame and can reconstruct the lost audio frame self-adaptingly to achieve better compensating effect. A transmitting and receiving method for error concealment of an audio stream is provided. A transmitter, a receiver and a system for error concealment of an audio stream are also provided.

Description

Method, device and system for audio stream error concealment

The present invention relates to real-time audio communication technology, and more particularly to a method, apparatus and system for audio stream error concealment. Background of the invention

People have been engaged in audio classification research for a long time, but the results of audio classification and classification are different for different application scenarios. For example, 1. In the noise suppression of advanced audio equipment, the FM signal is usually classified by FM analysis or Bayesian classifier. 2. In order to better index and retrieve audio resources on the Internet, content-based audio classification and retrieval studies have been conducted. The more representative content-based audio classification work analyzes the distinctive features of audio, including loudness, pitch, harmonicity, etc., and the audio classifier is designed. 3. Audio classification Another application is a voice activation detector (VAD) that serves audio, especially a speech coder. The purpose is to detect whether voice is present during voice communication. The voice and non-voice are different respectively. Encoding method to save the channel resources without reducing the call quality.

In real-time audio transmission systems (such as VoIP), the main reasons for the degradation of sound quality caused by network transmission are delay, static interference and packet loss. Among them, packet loss is the most important cause of sound quality degradation. There is always some interference in the actual communication network, so there is a certain probability of packet loss. Although the lower layer of the protocol has error correction coding, it can only solve the error in the packet and cannot solve the packet loss problem. In addition, due to the strict delay limit of real-time audio services, packet delays beyond one limit will also be considered lost. In order to ensure a certain communication quality under a certain packet loss rate, many codec algorithms integrate error concealment technology to eliminate the impact of packet loss.

Figure 1 is a block diagram of audio stream error concealment. As shown in Figure 1, the compressed audio signal passes through the IP. After transmission of the network or other unreliable network, the received audio data packet is usually stored in a jitter buffer, which is used to solve functions such as late packet and early packet reordering, and then performs packet loss and error packet detection. If there is a packet loss or a wrong packet, the system will start error concealment for packet loss compensation, otherwise the audio packet decoding output will be correctly received.

The packet loss recovery technology in audio real-time transmission can be divided into two categories according to the processing stage: sender-based repair and receiver-based repair.

• Error-based hiding on the sender side

The packet loss recovery based on the sender is initiated by the sender and needs to be coordinated between the sender and the receiver. Common methods include increased redundancy, forward error correction, priority setting, and classification processing.

1. Increase redundancy: Increasing the redundancy of data can improve the fault tolerance of the system, but at the same time increase the bandwidth.

2. Forward Error Correction (FEC): This strategy also relies on the addition of information that repairs lost packets in the transport stream. The use of blocks or algebraic words to generate additional transport packets that aid in error correction also requires increased bandwidth.

3. Priority setting method: This technology requires network support and transmits packets according to priority. Otherwise, it cannot be implemented, and it can only improve the packet loss probability caused by network congestion.

4. Classification processing method: In the speech coding, in order to better adopt the waveform replacement technology at the receiving end, the transmitting end can classify according to the characteristics of the speech signal, for example, 3GPP2 VMR-WB and ITU-T G.729.1 further the speech frame. The description is voiced, unvoiced, voiced transition, unvoiced transition, onset, etc., and after decoding the terminal, using the voice frame type of the previous frame and the next frame, the type of the dropped frame can be inferred, and the decoder loses the frame. After the type, the information of the lost frame can be recovered better.

• Error-based hiding at the receiving end

Receiver error concealment technology that does not require the sender to participate, essentially the number received Estimating lost data through a series of methods, and optimizing according to human physiological characteristics, is basically a passive patching, which is usually easier to implement and does not increase bandwidth requirements. Error-based hiding methods based on the receiving end can be divided into three categories:

1. Insertion-based strategy: This type of technology includes methods such as splicing, muting, and noise substitution. The stitching technique will disturb the timing of the media stream, and the effect is not good. The range of mute substitution (filling the frame loss position with a mute frame) is very limited, when the packet loss frequency is 4 low

(less than 2%) and the gap width is less than 4 ms, this method is more effective; when the gap width is 40 ms, the effect is unacceptable. Compared to silent replacement, noise substitution (filling the frame loss position with a noise frame) gives a better subjective hearing experience while improving the resolution of the speech signal. When background noise is used instead of muting, the human brain can subconsciously use the correct sound to repair missing parts of the speech signal. The insertion method is independent of the speech coding and is not related to the encoding of the packet, but only the speech lost after decoding.

2. Interpolation-based strategy: Compared with the insertion technique, the interpolation technique enables the processed sound to give a relatively better subjective feeling.

3. Re-generation based strategy: Extract the decoding status from the information around the lost packet and generate a replacement packet for the lost packet. The implementation of this method is more complicated, but it will achieve better results.

In general, based on the sender error concealment will increase the network bandwidth and computational complexity, the effect is better than the receiver-based, but if the sender error concealment is independent of the receiver, that is, regardless of the media content, then it will not be based on The characteristics of the frame loss adopt the corresponding error concealment strategy (for example, the stable speech frame is very similar to the previous frame, and the frame copy strategy can achieve a good hiding effect, and the transition frame needs to consider the state of the previous and subsequent frames to determine the hidden strategy) . The technology of the receiving end can also achieve a certain hidden effect, but if the hidden strategy is independent of the audio encoding, that is, the content characteristics of the currently lost frame and surrounding audio frames are not analyzed, so that a targeted error concealment strategy is adopted. Can use error concealment strategy will be very Limited.

At present, there are more and more audio coding standards that consider error concealment requirements at both the encoding end and the receiving end. There are two main methods for representative:

(1) The encoder analyzes the audio frame characteristics before formal coding, and uses different coding methods for audio frames of different characteristics. For example, AMR-WB+ uses ACELP and TCX encoding for signal frames based on audio frame content to form 26 superframes (each superframe consists of one superframe) encoding mode. The coding mode information is used for error concealment. In the case where a frame is lost, the receiving end infers or estimates the coding mode of the super frame according to the coding type of the remaining three frames of the superframe, thereby realizing a certain error concealment function.

(2) The voice frames are classified into voiced, unvoiced, voiced transition, unvoiced transition, onset, etc. according to the characteristics of the pitch, spectrum, and the like of the voice frame. In the latest ITU-T G.729.1 protocol, the encoder divides the voice frame into voiced, unvoiced, voiced transition, unvoiced transition, onset (VMR-WB also divides the voice frame according to the frame content and its characteristics. For these 5 classes, the type is indicated by 2 bits in layer 2. To further enhance the accuracy of error concealment, G.729.1 also calculates the phase and energy of the frame, which are transmitted in layers 3 and 4 of the next frame, respectively. The decoder will attempt to recover the frame identifier of the dropped frame from the known category identifier (including the category identifier of the previous frame), thereby reconstructing the audio waveform based on the category pattern of the dropped frame, combined with its phase and energy information.

For the first type of method: Although AMR-WB+ uses four modes of ACELP, TCX256, TCX512, and TCX1024 according to audio frame characteristics, encoding mode information is represented by 2, 2, 4, and 8 bits, respectively, and encoding is used in error concealment. Mode information, inferring or estimating the coding mode of the superframe (composed of 4 frames and 1024 sample points), thus realizing certain error concealment functions, but only the coding mode of audio coding is indicated, and cannot be based on the content of the audio frame. The strategy is used for frame loss reconstruction, so efficient error concealment cannot be achieved. For the second type of method: At present, this type of error concealment technology is designed for speech frames and does not work well when dealing with other types of audio frames. For the classification detection of music and natural sounds, especially how they reconstruct packet loss information in the case of packet loss, audio communication can also tolerate a high packet loss rate. There is currently no effective method.

In summary, the current audio stream error concealment technique cannot achieve efficient error concealment of audio frames, so that the requirements for packet loss rate in the audio communication process cannot be reduced. Summary of the invention

In view of this, the embodiment of the present invention provides a method for error concealment of an audio stream, which can implement efficient error concealment of an audio stream.

Embodiments of the present invention also provide an apparatus and system for error concealment of an audio stream, and the apparatus and system can be used to implement efficient error concealment of an audio stream.

In order to achieve the above object, the following technical solutions are adopted:

A method for transmitting an audio stream error concealment, including:

The transmitted audio frame is classified according to the content, and the type information of the audio frame is obtained. The type information of the audio frame and the encoding result of the audio frame are packaged and sent out. A method for receiving an audio stream error concealment, comprising:

When a frame loss occurs, for the lost audio frame, determine the type information of the audio frame obtained when it is classified according to the content;

According to the type information of the lost audio frame, the corresponding error recovery strategy is used for audio frame reconstruction.

A method for hiding audio stream errors, including:

And sorting the content of the audio frame according to the content, and obtaining the type information of the audio frame; and packaging and transmitting the type information of the audio frame and the encoding result of the audio frame to the receiving end; When a frame loss occurs, for the lost audio frame, the receiving end determines the type information of the audio frame obtained when it is classified according to the content;

A transmitter for error concealment of an audio stream, comprising an audio encoder module, a frame encapsulation module and an audio frame classifier module;

The audio frame classifier module is configured to classify the transmitted audio frame according to content, obtain type information of the audio frame, and send the type information to the frame encapsulation module; the frame encapsulation module is configured to receive The type information of the audio frame sent by the audio frame classifier module and the encoding result of the audio frame sent by the audio encoder module are packaged and sent out by the type information of the audio frame and the encoding result of the audio frame.

A receiver for error concealment of an audio stream, comprising a frame type discriminating module and an error concealing module,

The frame type discriminating module is configured to determine type information of the audio frame obtained when the lost audio frame is classified according to the content, and send the type information to the error concealment module;

The error concealing module is configured to perform audio frame reconstruction by using a corresponding error recovery strategy according to the type information of the received lost audio frame.

An audio stream error concealment system, comprising: a transmitter and a receiver;

The transmitter is configured to classify the transmitted audio frame according to content, obtain type information of the audio frame, and package and send the type information of the audio frame and the encoded result of the audio frame to the receiver;

The receiver is configured to determine, when a frame loss occurs, type information obtained when the lost audio frame is classified according to the content, and according to the type information, perform an audio frame reconstruction by using a corresponding error recovery strategy. As shown in the foregoing technical solution, the embodiment of the present invention classifies the content of the audio frame according to the content of the audio frame, and sends the type information of the audio frame together with the encoded result of the audio frame. At the receiving end, when the packet loss occurs, according to the loss. The audio frames are classified according to the content, and the corresponding error concealment strategy is used to reconstruct the audio signal. It can be seen that the error concealment method in the embodiment of the present invention makes the reconstruction of the lost frame more targeted, and can adaptively reconstruct the audio frame to achieve better compensation effect, and bring more benefits to the receiving end user. A good subjective auditory experience, while improving the resolution of the audio frame signal, enables audio communication to tolerate higher packet loss rates. BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a block diagram of audio stream error concealment;

2 is a general flowchart of a method for error concealment of an audio stream according to an embodiment of the present invention; FIG. 3 is a schematic diagram of a system for error concealment of an audio stream according to an embodiment of the present invention; FIG. FIG. 5 is a general structural diagram of a receiver for error concealment of an audio stream according to an embodiment of the present invention; FIG. 6 is a specific flowchart of a method for transmitting error concealment of an audio stream according to an embodiment of the present invention; Schematic diagram of classification of audio frames;

8 is a specific flowchart of a method for receiving an audio stream error concealment according to an embodiment of the present invention; FIG. 9 is a specific structural diagram of a transmitter for error concealment of an audio stream according to an embodiment of the present invention; FIG. 10 is an audio stream error concealment according to an embodiment of the present invention; The specific structure of the receiver. Mode for carrying out the invention

The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

2 is a general flow chart of a method for error concealment of an audio stream according to an embodiment of the present invention. Figure 2 As shown, the method includes:

Step 201: classify the transmitted audio frame according to content, and obtain type information of the audio frame.

Step 202: Encapsulate and send out the type information of the audio frame and the encoding result of the audio frame.

Step 203: When a frame loss occurs, determine, for the lost audio frame, type information of the audio frame obtained when the content is classified according to the content;

Step 204: Perform audio frame reconstruction according to the type information of the lost audio frame by using a corresponding error recovery policy.

Steps 201 to 202 constitute an overall flow of the transmission method of the audio stream error concealment; steps 203 to 204 constitute an overall flow of the reception method of the audio stream error concealment.

FIG. 3 is a schematic structural diagram of a system for error concealment of an audio stream according to an embodiment of the present invention. As shown in FIG. 3, the system includes a transmitter 301 and a receiver 302. In the system, the transmitter 301 is configured to classify the transmitted audio frame according to content, obtain type information of the audio frame, and package and transmit the type information of the audio frame and the encoding result of the audio frame to the receiver.

The receiver 302 is configured to determine, according to the type information, the type information obtained when the lost audio frame is classified according to the content, and perform the audio frame reconstruction according to the type information. The transmitter and receiver in the system can employ the specific configurations of the transmitter 400 and the receiver 500 shown in Figs. 4 and 5, respectively.

4 is a general structural diagram of a transmitter for error concealment of an audio stream in the present invention. As shown in FIG. 4, the transmitter 400 includes an audio encoder module 410, an audio frame classifier module 420, and a frame encapsulation module 430.

In the transmitter 400, the audio encoder module 410 is configured to encode the transmitted audio frame and send the encoded result to the frame encapsulation module 430. An audio frame classifier, configured to classify the sent audio frame according to content, obtain type information of the audio frame, and The type information is sent to the frame encapsulation module 430. The frame encapsulating module 430 is configured to receive the encoding result of the audio frame sent by the audio encoder module 410 and the type information of the audio frame sent by the audio frame classifier module 420, and package the type information of the audio frame and the encoding result of the audio frame. Send it out. When the audio frame is encoded in the audio encoder 410, the audio frame may be encoded differently according to the type information of the audio frame sent by the audio frame classifier, or the same coding mode may be directly applied to all the encoded frames.

Corresponding to the transmitter, Fig. 5 is a general structural diagram of a receiver for audio stream error concealment in the present invention. The receiver includes: a frame type discrimination module 510 and an error concealment module 520.

In the receiver, the frame type discriminating module 510 is configured to use the type information of the audio frame obtained when the lost audio frame is classified according to the content, and send the type information to the error concealment module 520. The error concealing module 520 is configured to perform audio frame reconstruction according to the type information of the received lost audio frame by using a corresponding error recovery strategy.

It can be seen from the above that the present invention classifies the audio frame by the content at the transmitting end to obtain the type information of the audio frame, and sends the type information of the audio frame to the receiving end. When the frame is lost, the receiving end adopts different error recovery strategies according to the type information of the lost audio frame. Perform audio frame reconstruction to efficiently hide errors.

FIG. 6 is a specific flowchart of a method for error concealment of an audio stream according to an embodiment of the present invention. As shown in Figure 6, the method includes:

Step 601: Divide the audio signal into equally spaced audio frames.

In this step, the frame length of the audio frame is determined according to the encoding protocol.

Step 602: Analyze the content and characteristics of the audio frame to obtain type information of the audio frame.

In this step, the audio frame is divided into a voice signal frame, a noise signal frame, a mute signal frame, a tone signal frame, and the like, and then further subdivided for each type, for example, the voice signal can be further divided into Voiced, unvoiced, voiced transition, unvoiced For transition, onset, etc., the tone signal frame can be divided into stable tone frames, transitions, and the like according to the stable characteristics of the signal.

Step 603: Perform encoding compression on the transmitted audio frame.

In this step, the same encoding method may be used for the entire audio signal, or different encoding methods may be used depending on the type of the audio frame.

Step 604, package the type of the audio frame and the result of the encoding compression, and send it out.

In this step, when the package is packaged, the type information of the audio frame may be identified in the frame header of the current frame or the next frame.

So far, the transmission flow of the audio stream error concealment in this embodiment ends.

In the above process, when the audio frame classification is performed in step 602, the method shown in FIG. 7 can be used. Referring to Figure 7, the VAD is first used to detect whether the audio frame is a noise signal frame. If it is a noise signal frame, spectrum energy analysis is performed on the audio frame, and if it is a non-noise signal frame, spectrum stability analysis is performed on the audio frame.

According to the result of the spectrum energy analysis, the audio frame is divided into a mute signal frame and a noise signal frame, and then the mute signal frame or the noise signal frame can be further classified, and the type information of the audio frame is obtained.

According to the result of the spectrum stability analysis, the audio frame is divided into a speech signal frame and a tone signal frame, and then the speech signal frame or the tone signal frame can be further classified, and the port can be used for the voice signal. For voiced, unvoiced, voiced transition, etc., the tone signal frame can be refined into a stable tone frame, a transitional tone frame, and the like.

The classification of the audio frame in the above transmission method is performed at the receiving end, in the embodiment, using the method shown in FIG. As shown in FIG. 8, the receiving method includes:

Step 801: Perform frame loss detection on the audio signal. If frame loss occurs, perform step 804 and subsequent steps. Otherwise, perform step 802 and subsequent steps. In this step, it is determined whether the loss of the audio frame occurs according to the frame number carried in the audio frame.

Step 802: Detect and record the type of the audio frame.

When a frame loss occurs, the type of the audio frame recorded in this step can be used to determine the type information of the lost audio frame.

Step 803: Decode the audio frame, and output the decoding result, and end the process. In this step, according to the encoding method of the audio frame by the transmitting end, the corresponding decoding method is used for decoding.

Step 804, determining type information obtained when the lost audio frame is classified according to the content. In this step, if the type information of the audio frame is carried in the audio frame, the receiving end extracts the historical data, and infers the type of the currently lost frame according to the type information of the correctly received frame; if the type information of the audio frame is carried in other If the audio frame is correctly received, the receiving end directly extracts the type information of the currently lost frame in the corresponding correctly received audio frame.

Step 805: Reconfigure the audio frame by using the corresponding error recovery strategy adaptively according to the type of the lost audio frame, and output the reconstructed result, and end the process.

In this step, the audio frame can be reconstructed according to the type of the lost audio frame and the most suitable error recovery strategy for the type. For example, a stable speech frame is very similar to its previous frame. A frame copy strategy can achieve a good hiding effect. A transition frame needs to consider the state of the previous and succeeding frames to determine a hidden strategy.

At this point, the receiving method of the audio stream error concealment ends.

In the sending method of this embodiment, the audio frame is classified by using the method shown in FIG. 7 , and other content-based audio frame classification methods may be used, as long as the audio frame is classified according to the content. The purpose can be.

It can be seen from the above that when the transmitting end and the receiving end cooperate, the classification of the audio frame can be utilized to efficiently realize error concealment, and the tolerance of the real-time audio communication to the packet loss rate is greatly improved. The foregoing is a specific implementation manner of the method for transmitting and receiving audio stream error concealment provided in the embodiment. The two embodiments cooperate with each other to form a specific embodiment of the method for error concealment of the audio stream in the present invention. In addition, this embodiment also provides a specific implementation of the corresponding audio stream error concealed transmitter and receiver.

FIG. 9 is a specific structural diagram of a transmitter for error concealment of an audio stream according to an embodiment of the present invention. As shown

As shown in FIG. 9, the transmitter 900 includes: an audio encoder module 910, an audio frame classifier module

920. Frame encapsulation module 930 and audio frame division module 940.

In the transmitter 900, an audio frame dividing module 940 is configured to divide the audio signal into equally spaced audio frames according to different encoding protocols, and send the audio frames to the audio encoder module 910 and the audio frame classifier module 920. .

The audio encoder module 910 is configured to encode the audio frame and send the encoded result to the frame encapsulation module 930. The audio frame classifier module 920 is configured to classify the audio frames according to the content, and the specific classification manner may be the manner shown in FIG. 7, and the type information of the audio frames is sent to the frame encapsulation module 930.

The frame encapsulation module 930 is configured to receive the audio frame coding result sent by the audio encoder module 910 and the audio frame type information sent by the audio frame classifier module 920, and package and transmit the type information and the encoded result of the audio frame. When performing encapsulation and packaging, the type information of the audio frame may be encapsulated in the audio frame or the next audio frame, and may be located in a part of the frame header.

When the audio frame is encoded in the audio encoder 910, the audio frame may be encoded differently according to the type information of the audio frame sent by the audio frame classifier, or the same encoding mode may be directly applied to all the encoded frames.

FIG. 10 is a specific structural diagram of a receiver for error concealment of an audio stream according to an embodiment of the present invention. As shown in FIG. 10, the receiver 1000 includes a frame type discrimination module 1010, an error concealment module 1020, an error detection module 1030, and an audio decoder module 1040. Where frame type discrimination The module 1010 includes a discriminating sub-module 1011 and a storage sub-module 1012. The error concealing module 1020 includes a policy decision sub-module 1021 and an error concealment sub-module 1022.

In the receiver 1000, the error detection module 1030 is configured to receive an audio frame from the channel, send the received audio frame to the discriminant sub-module 1011 in the frame type discriminating module 1010, and detect whether a frame loss occurs. When the frame is dropped, the discriminating sub-module 1011 in the frame type discriminating module 1010 is notified.

In the frame type discriminating module 1010, when determining the type in which the audio frame is classified according to the content, if the type information of the audio frame is carried in the correctly received audio frame, the type information is directly extracted and stored in the storage submodule. In 1012, if the type information of the audio frame is carried in the lost audio frame, the type information obtained when the lost audio frame is classified according to the content is inferred according to the type of the preceding and succeeding frames.

In the error concealment module 1020, the policy decision submodule 1021 is configured to receive the type information of the lost frame sent by the discriminating submodule 1011, and determine the adopted error recovery strategy according to the type information, and send the result to the error concealment submodule. 1022. The error concealment sub-module 1022 is configured to reconstruct the lost audio frame according to the error recovery policy decision result sent by the policy decision sub-module 1021.

The audio frame decoder module 1040 is configured to decode the received audio frame and output the decoded result.

In the transmitter 900 of the present embodiment, the audio frame classifier module 920 uses the manner of FIG. 7 to classify audio frames. Of course, other content-based audio frame classification methods may also be used, and details are not described herein. In the receiver 1000, the frame type decision module 1010 is refined into a decision sub-module 1011 and a storage sub-module 1012, respectively performing frame type decision and storage; the error concealment module 1020 is refined into a policy decision sub-module 1021 and an error concealer. Module 1022, respectively, performs policy decisions and error concealment. The error concealment sub-module 1022 can be further divided into a plurality of different types of error concealment units, such as a noise error concealment list. Meta, voice error concealment unit, etc., used to handle error concealment of different types of audio frames. An embodiment of the audio stream error concealment system of the present invention may be: using the above FIG. 9 and

The transmitter 900 and the receiver 1000 shown in FIG. 10 are specific embodiments of the transmitter and the receiver in the audio stream error concealment system, and the audio frame outputted by the frame encapsulation module 930 in the transmitter 900 is transmitted to the receiver 1000. Error detection module 1030. Thus, an embodiment of the audio stream error concealment system of the present invention can be constructed.

It can be seen from the foregoing specific embodiments of the method, the device and the system of the present invention that the technical solution of the present invention makes the reconstruction of the lost frame more targeted, and can adaptively reconstruct the audio frame to achieve better. The compensation effect brings a better subjective hearing experience to the receiving end user, and at the same time improves the resolution of the audio frame signal, so that the audio communication can tolerate a higher packet loss rate.

The above are only the preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Claims

Claim

A method for transmitting an audio stream error concealment, the method comprising: classifying a transmitted audio frame according to content to obtain type information of the audio frame; and encoding the type information of the audio frame and the encoding of the audio frame The result is packaged and packaged for delivery.

The method according to claim 1, wherein the classifying the transmitted audio frames according to the content comprises:

Using the voice activation detector VAD to detect whether the audio frame is a noise signal frame, and if so, performing spectrum energy analysis on the noise signal frame to determine whether the audio frame is a silence signal frame or a noise signal frame; otherwise, frequency-speeching the non-noise signal frame Stability analysis determines that the audio frame is a speech signal frame or a tone signal frame.

3. Method according to claim 2, characterized in that the mute signal frame, the noise signal frame, the speech signal frame or the tone signal frame are further finely divided.

The method according to claim 1, wherein the encoding of the type information of the audio frame and the encoding result of the audio frame is packaged as: packaging the type information of the audio frame into a frame header of the audio frame.

The method according to claim 4, wherein the frame header of the audio frame is: a frame header of an audio frame represented by type information of an audio frame or an audio frame represented by type information of an audio frame The frame header of a frame.

The method according to claim 1, wherein the encoding result of the audio frame is: an encoding result obtained by using the same encoding method for all audio frames, or different encoding according to different types of audio frames. The result of the encoding obtained by the method.

A method for receiving an audio stream error concealment, the method comprising: determining, when a frame loss occurs, a type of the audio frame obtained by classifying the content of the lost audio frame;

According to the type information of the lost audio frame, use the corresponding error recovery strategy for audio Frame reconstruction.

The receiving method according to claim 7, wherein the type information of the audio frame obtained when the content is classified according to the content is:

When the type information of the lost audio frame is located in the correctly received audio frame, the type information of the lost audio frame is directly extracted from the correctly received audio frame;

When the type information of the lost audio frame is located in the lost audio frame, the type information of the lost audio frame is inferred based on the type information of the correctly received frame.

The receiving method according to claim 7 or 8, wherein before the operation of determining the type information of the audio frame obtained by classifying the lost audio frame by the content, the method further comprises : detecting whether there is a frame loss on the received audio signal, and if yes, performing the operation of determining the type information of the audio frame obtained when the content is classified according to the content; otherwise, decoding the correctly received audio frame, extracting the Type information of the audio frame carried in the frame and stored.

A method for error concealment of an audio stream, the method comprising: classifying the transmitted audio frame according to content, and obtaining type information of the audio frame; and encoding the type information of the audio frame and the encoding result of the audio frame The package is packaged and sent to the receiving end;

When a frame loss occurs, for the lost audio frame, the receiving end determines the type information of the audio frame obtained when the content is classified according to the content;

The method according to claim 10, wherein the classifying the transmitted audio frames according to content comprises:

Using the voice activation detector VAD to detect whether the audio frame is a noise signal frame, and if so, performing spectrum energy analysis on the noise signal frame to determine that the audio frame is a mute signal frame or a noise signal. No. Frame; Otherwise, the non-noise signal frame is analyzed for frequency stability, and the audio frame is determined to be a speech signal frame or a tone signal frame.

The method according to claim 10, wherein the type information of the audio frame and the encoding result of the audio frame are packaged as: packet type information of the audio frame into a frame header of the audio frame.

The method according to claim 12, wherein the frame header of the audio frame is: a frame header of an audio frame represented by type information of an audio frame or an audio frame represented by type information of an audio frame The frame header of a frame.

The method according to claim 10, wherein the determining the type information of the audio frame obtained by classifying according to the content is:

The method according to any one of claims 10 to 14, wherein after performing the operation of packaging and transmitting the type information of the audio frame and the encoding result of the audio frame to the receiving end, the receiving end is Before the operation of determining the type information of the audio frame obtained when the lost audio frame is determined according to the content, the method further comprises: detecting whether there is a frame loss on the received audio signal, and if yes, performing the determining The operation of the type information of the audio frame obtained when the content is classified according to the content, otherwise, the correctly received audio frame is decoded, and the type information of the audio frame carried in the frame is extracted and stored.

16. An audio stream error concealed transmitter, comprising an audio encoder module and a frame packing module, wherein the transmitter further comprises an audio frame classifier module;

The audio frame classifier module is configured to classify the sent audio frame according to content, obtain type information of the audio frame, and send the type information to the frame encapsulation module; The frame encapsulating module is configured to receive type information of an audio frame sent by the audio frame classifier module and an encoding result of the audio frame sent by the audio encoder module, and encode the type information of the audio frame and the audio frame. The result is packaged and packaged for delivery.

17. A receiver for error concealment of an audio stream, the receiver comprising a frame type discriminating module and an error concealing module,

The receiver according to claim 17, wherein the receiver further comprises an error detecting module and an audio decoder module,

The error detection module is configured to receive an audio frame from a channel, send the received audio frame to the frame type discriminating module, and detect whether a frame loss occurs, and if a frame loss occurs, notify the frame type discriminating module ;

The frame type discriminating module is further configured to forward the audio frame to the audio frame decoder module;

The audio frame decoder module is configured to decode an audio frame.

The receiver according to claim 18, wherein the frame type discriminating module comprises a discriminating submodule and a storage submodule,

The discriminating sub-module is configured to determine type information of the audio frame obtained when the lost audio frame is classified according to content, and send the type information to the storage sub-module, and is further configured to receive the error detection. After the frame loss notification sent by the module, sending the type information of the lost frame to the error concealment module, and forwarding the received audio frame to the audio frame decoder module; The storage submodule is configured to save type information of an audio frame.

The receiver according to claim 18 or 19, wherein the error concealment module comprises a policy decision submodule and an error concealment submodule.

The policy decision sub-module is configured to receive type information of the lost frame sent by the frame type discriminating module, and determine an adopted error recovery strategy according to the type information, and send the result to the error concealment sub-module;

The error concealment submodule is configured to reconstruct the lost audio frame according to the error recovery policy decision result sent by the policy decision submodule.

21. An audio stream error concealment system, the system comprising: a transmitter and a receiver;

The receiver is configured to determine type information obtained when the lost audio frame is classified according to the content when the frame loss occurs, and perform audio frame reconstruction according to the type information according to the type of the error recovery strategy.

The system according to claim 21, wherein the transmitter comprises an audio encoder module, a frame encapsulation module and an audio frame classifier module, and the receiver comprises a frame type discrimination module and an error concealment module;

The audio frame classifier module is configured to classify the transmitted audio frame according to content, obtain type information of the audio frame, and send the type information to the frame encapsulation module; the frame encapsulation module is configured to receive The type information of the audio frame sent by the audio frame classifier module and the encoding result of the audio frame sent by the audio encoder module, and the type information of the audio frame and the encoded result of the audio frame are packaged and sent out;

The frame type discriminating module is configured to determine that the lost audio frame is classified according to content Obtaining type information of the audio frame, and sending the type information to the error concealment module;

23. The system of claim 22, wherein the receiver further comprises an error detection module and an audio decoder module,

The audio frame decoder module is configured to decode an audio frame;

The frame encapsulating module is configured to package and send the type information of the audio frame and the encoding result of the audio frame to the error detecting module.

The receiver according to claim 22 or 23, wherein the error concealment module comprises a policy decision submodule and an error concealment submodule.