WO2011057588A1

WO2011057588A1 - Streaming media server and method for synchronization in media stream switching

Info

Publication number: WO2011057588A1
Application number: PCT/CN2010/078760
Authority: WO
Inventors: 刘启铭; 仇刚
Original assignee: 华为技术有限公司
Priority date: 2009-11-16
Filing date: 2010-11-16
Publication date: 2011-05-19
Also published as: CN102065060B; CN102065060A

Abstract

A streaming media server and a method for synchronization in media stream switching are disclosed. In the embodiments of the present invention, a streaming media data packet is modified; NTP timestamps are replaced by system time of the streaming media server in order that the Network Time Protocol (NTP) timestamps of several media streams are switched from distinct sources to the same source; or the calibration of the Real-time Transport Protocol (RTP) timestamps of several received media streams is absolutely normalized according to the system time of the streaming media server directly; thereby the synchronization in media switching is realized. The problem that media streams of different NTP timepieces can not be switched in the prior art is resolved, and the user service experience is improved.

Description

Media stream switching synchronization method and streaming server This application claims priority to Chinese patent application filed on November 16, 2009 by the Chinese Patent Office, Application No. 200910180193.2, and entitled "Media Stream Switching Synchronization Method and Streaming Media Server" The entire contents of which are incorporated herein by reference. Technical field

The present invention relates to the field of communications technologies, and in particular, to a media stream switching synchronization method and a streaming media server. Background technique

Realtime Transport Protocol (RTP): A transport protocol for multimedia data streams over the Internet. RTP is defined to operate in a one-to-one or one-to-many transmission with the purpose of providing time information and stream synchronization. RTP itself only guarantees the transmission of data when it is confirmed. It does not provide a reliable transmission mechanism for transmitting packets in sequence, nor does it provide flow control or congestion control. It relies on Real-time Control Protocol (RTCP) to provide services.

The RTCP protocol is responsible for managing the transmission quality to exchange control information between current application processes. During the RTP session, each participant periodically transmits RTCP packets, which contain statistics such as the number of transmitted packets and the number of lost packets. Therefore, the server can use this information to dynamically change the transmission rate, even Change the payload type. Used in conjunction with RTCP, RTP optimizes transmission efficiency with efficient feedback and minimal overhead, making it ideal for delivering real-time data on the network.

In the RTP packet, the initial values of the packet sequence number and timestamp are random, and the timestamp only reflects the sampling time of the first byte. RTP timestamps for different media streams may grow at different rates and have independent random offsets. Therefore, it is not possible to synchronize the time stamps of different media streams directly.

A key function of RTCP is to enable the receiver to synchronize multiple RTP streams. For example: When audio and video are transmitted together, RTP uses two streams for transmission because of different encodings, so the time stamps of the two streams are different. At the rate of operation, the receiver must synchronize the two streams to ensure that the sound is consistent with the image. In order to be able to perform stream synchronization, there is a network in the sender report that is delivered. The absolute time value represented by the time protocol NTP (Network Time Protocol) format, and then an RTP timestamp (timestamp) value is given in the sender report (Sender Report, SR), and the clock generating the value is the timestamp in the RTP packet. The clock of the field. The sender reports as an RTCP packet, which conforms to the RTCP protocol standard. Since all the streams sent by the sender and the sender report use the same absolute clock (NTP clock), the receiver can compare the absolutes of the two streams from the same data source. Time, thereby determining how to map the timestamp value in one stream to the timestamp value in another stream.

Summary of the invention

The embodiment of the invention provides a media stream switching synchronization method and a streaming media server, which can realize synchronous switching of media streams when NTP is not homologous.

A media stream switching synchronization method provided by an embodiment of the present invention includes:

The streaming server receives the sender report packet of the media stream;

The streaming media server records the system time when the sender reports the packet, and modifies the network calibration protocol network time-of-day protocol time stamp of the sender report packet according to the recorded system time;

When media stream switching is performed, synchronous switching is performed according to the network timing protocol time stamp in the sender report packet of the media stream.

The streaming media server receives the real-time transport protocol packet of the media stream;

The streaming media server replaces the real-time transport protocol timestamp in the real-time transport protocol packet according to the system timestamp when the real-time transport protocol arrives; the system timestamp is generated according to the system time, and a unique system timestamp is generated per unit time;

When the media stream is switched, the system time stamp in the real-time transport protocol packet of the media stream is synchronously switched.

A streaming media server provided by an embodiment of the present invention includes:

a data packet receiving unit, configured to receive a sender report packet of the media stream;

a timestamp modification unit, configured to record a system time when the real-time transmission control protocol packet is received, and modify a network timing protocol network timing of the sender report packet according to the recorded system time Protocol timestamp

The switching control unit is configured to perform synchronous switching according to a network calibration time stamp in the sender report packet of the media stream when the media stream is switched.

a receiving unit, configured to receive a real-time transport protocol packet of the media stream;

a timestamp replacement unit, configured to replace a real-time transport protocol timestamp in the real-time transport protocol packet according to a system timestamp when the real-time transport protocol arrives; the system timestamp is generated according to a system time, and a unique system timestamp is generated per unit time And a switching control unit, configured to perform synchronous switching according to a system timestamp in a real-time transport protocol packet of the media stream.

In the embodiment of the present invention, the streaming media data packet is modified, and the system time of the streaming media server is replaced with the NTP timestamp, and the NTP time stamp of the multiple media streams is converted from different sources to the same source, or directly according to the streaming media. The system time of the server performs absolute unified calibration on the received RTP timestamps of multiple media streams, thereby realizing the synchronization of media switching. The problem that the media stream of different NTP clocks cannot be switched in the prior art is solved, and the service experience of the user is improved. DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in view of the drawings.

1 is a flowchart of a method for media stream switching synchronization according to an embodiment of the present invention;

2 is a flowchart of a media stream synchronous switching according to an embodiment of the present invention;

3 is a flowchart of a method for media stream switching synchronization according to another embodiment of the present invention;

4 is a schematic structural diagram of a streaming media server according to an embodiment of the present invention;

5 is a schematic structural diagram of a switching control unit of a streaming media server according to an embodiment of the present invention; and FIG. 6 is a schematic structural diagram of a streaming media server according to another embodiment of the present invention. detailed description The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only embodiments of the present invention, based on the embodiments of the present invention, All other embodiments obtained by a person of ordinary skill in the art without creative efforts are within the scope of the present invention.

Because the media stream is synchronized based on the NTP clock mechanism, it is necessary to ensure that the two media streams that are switched use the same NTP clock. Otherwise, the switching between media streams cannot be completed. However, in practical applications, it is sometimes difficult to ensure all media. The NTP clocks between streams are the same, and it is often impossible to perform synchronous switching or switching errors. In response to this, the present invention proposes a solution.

The embodiment of the present invention provides a media stream switching synchronization method. The flowchart is as shown in FIG. 1 and includes:

A1, the streaming media server receives the SR packet of the media stream;

A2, the streaming media server records the system time when the SR packet is received, and modifies the network timing protocol NTP timestamp of the SR packet according to the recorded system time;

A3. When the media stream is switched, the NTP time stamp in the SR packet of the media stream is synchronously switched.

In the embodiment of the present invention, the streaming media data packet is modified, and the system time of the streaming media server is replaced with the NTP timestamp, and the NTP time stamp of the multiple media streams is converted from different sources to the same source, thereby implementing media switching. Synchronization. The problem that the media stream of different NTP clocks cannot be switched in the prior art is solved, and the service experience of the user is improved.

In the embodiment of the present invention, the method for performing the synchronous switching according to the NTP timestamp in the real-time transmission control protocol packet of the media stream may refer to the conventional implementation manner of the prior art, so that the time stamp of obtaining the media stream after the handover is more accurate, the present invention Provide a feasible switching mode, as shown in Figure 2, including:

B1, when the media stream is switched, the streaming media server is based on the RTP timestamp in the last real-time transport protocol RTP packet of the media stream before the handover, and the NTP timestamp in the SR packet of the pre-switch media stream and the SR packet in the switched media stream. The NTP timestamp recalculates the RTP timestamp in the first RTP packet of the media stream after the switch.

The specific process of recalculating the RTP timestamp in the RTCP packet of the switched media stream includes: RTP timestamp of the first RTP packet of the switched media stream = (SR packet of the switched media stream) NTP timestamp - NTP timestamp of the SR packet of the previous media stream) X Rate of sampling of the media stream before switching + RTP timestamp of the SR packet of the media stream before switching. After the timestamp of the first RTP packet of the switched media stream is obtained, the RTP timestamp in the first RTP packet may be used to regenerate the RTP timestamp for the subsequent data packet of the switched media stream. Here, the first RTP packet of the switched media stream refers to the first RTP packet sent to the user after the media switch.

B2: The RTP timestamp of the switched media stream calculated by the timestamp computing unit is added to the RTP packet of the switched media stream and sent to the user.

It can be understood that, in the embodiment of the present invention, the calculated RTP timestamp can also be adjusted according to the feedback of the user: the user periodically performs delay feedback, receives the delay feedback of the user, and adjusts and sends the RTP packet according to the delay feedback of the user. The value of the RTP timestamp, so that the delay on the user side can be eliminated by adjusting the advance time of the timestamp.

Conditional assumptions: The media streams of the two channels are consistent in media format, sample rate, and resolution. It is to be understood that the following technical details are specific examples in the implementation of the present application and do not constitute a limitation of the present invention.

In this example, the accuracy of the system time based on the commonly used streaming server is likely to fail to meet the requirements for accurate timing of each packet. Therefore, in this example, the system time + real-time clock is used.

( Real-Time Clock, RTC) The mode of the clock is timed. When the server starts, it reads the current system time and uses this as the reference time. The server uses the high-precision RTC clock that comes with the Linux system to calculate the elapsed time, adds the elapsed time to the reference time, and gets a current time. Time, the current RTC time is taken in the subsequent samples.

Under the premise of the above high-precision clock mechanism, an RTP time stamp deviation correction model based on RTC mechanism is established. The current RTC clock information is extracted from the SR packets received from each track, and the original NTP clock information is replaced as the NTP timestamp of the SR packet. After the channel is switched, the current NTP clock information may be unrelated. The time stamps of the two tracks are continuous. The specific calculation process of the model is as follows:

For each RTP packet of the track, according to the sampling rate of the track, the timestamp of the SR recently received by the track, the RTC timestamp, the timestamp of the RTP itself, the timestamp of the RTP sent by the track, etc., are calculated. The RTC time value (unit: second) is calculated as follows: N _n =N _sec +N _frac /X + (ts _r -TS)lt Equation ₍₁₎ The RTC time difference between two packets before and after the orbit (unit: second), calculated as follows: = H Equation (2)

The corrected timestamp of the current RTP packet sent by the track, calculated as follows: ts _m = (ANxt) + ts _c formula (3)

The corrected packet sequence number for the current RTP packet sent by this track is calculated as follows: seq _m =seq _c +l Equation (4)

The payload type of the current RTP packet sent by this track needs to be consistent with the initial payload type (ayload type) of the track.

= Pt formula (5)

For each SR packet of the track, since its RTC timestamp is guaranteed to be correct, then only the timestamp is corrected. Before sending, the timestamp (unit: second) of the SR packet is re-corrected according to the sampling rate of the track, the timestamp of the SR packet recently sent by the track, the RTC timestamp, and the RTC timestamp information of the currently sent SR. as follows:

TS _m = (" _sec +n _frac /X-(n _sec _ _c +n _frac _ _c /X))/t + TS _c Formula ₍₆₎ General formula (1)(2)(3)(4) ( 5) (6), the audio and video synchronization target function is as follows:

Objective function H formula (7)

Seq _m =seq _c +l formula ₍₈₎

^ _m = d -N _n )xt + ts _c formula (9)

TS _m = (n _sec +n _frac IX_{n一_+rifrac _ _c IX)) lt ₊ TS _C formula (10) Constraint "≥0," l,2, + ++,N

Ts, „≠ ts„ The parameters in the above formula are defined as follows:

TS: Timestamp of the most recent RTCP packet of the media track (timestamp).

^TS ">: The corrected timestamp of the RTCP packet sent by the media track.

X: The divisibility factor of the NTP timestamp, for example: OxFFFFFFFF.

^Ν · The integer part (in seconds) of the RTC timestamp of the SR packet recently received by the media track.

^ : The fractional part of the RTC timestamp of the SR packet recently received by the media track (unit: 1 second). t: the time scale of the orbital scale. : The original sequence number of the current RTP packet.

s ^eq >»: sequence number after correction of the current RTP package

The original timestamp of the current RTP packet.

t ^s The timestamp after the current RTP packet is corrected.

: The integer part (in seconds) of the RTC timestamp of the current SR packet.

" : The fractional part of the RTC timestamp of the current SR packet (unit: 1/sec).

P ^t : The initial payload type of the channel.

p ^{t The} current payload type of the RTP packet.

N": RTC time (in seconds) of the RTP packet whose media track number is n.

AN: The absolute difference in RTC time between two RTP packets before and after the media track (unit: second).

The sequence of the RTP packet recently sent by the s ^e track.

t ^s . The timestamp of the RTP packet recently sent by the track.

: The original timestamp of the RTP packet that was last sent by the track.

The timestamp of the SR packet recently sent by the track.

" : The integer part (in seconds) of the RTC timestamp of the SR packet sent by the track.

: The fractional part (in seconds) of the RTC timestamp of the SR packet recently sent by the track. The present invention provides another embodiment of a body stream switching synchronization method. The flowchart is as shown in FIG. 3, and includes:

C1, the streaming media server receives the RTP packet of the media stream;

C2, the streaming media server replaces the RTP timestamp in the RTP packet according to the system timestamp when the RTP arrives; the system timestamp is generated according to the system time, and a unique system timestamp is generated every unit time;

C3. When the media stream is switched, the system timestamp in the RTP packet of the media stream is synchronously switched.

The foregoing embodiment of the present invention re-establishes the reference time axis by using the local system time of the streaming media server as a reference point. In this embodiment, the system timestamp is re-generated as the RTP timestamp according to the reference time axis. The generated timestamp is based on the reference time axis and is in the same unit of the streaming server system time (the unit time is related to the time precision of the system time as follows: eg milliseconds) Received multiple packets, generated timestamp value If the media stream is switched, the RTP timestamp conversion is not required, and the media stream is synchronously switched. The streaming server provided by the present invention will be described in detail below. For the sake of explanation, only show In a part related to the embodiment of the present invention, the terminal device may be a mobile phone or a data card or the like, and the function module/unit included may be a hardware module/unit, a software module/unit or a combination of software and hardware. /unit.

The present invention provides an embodiment of a streaming media server. The schematic diagram of the structure is as shown in FIG. 4, including: a data packet receiving unit 410, configured to receive a SR packet of a media stream;

a timestamp modification unit 420, configured to record a system time when the real-time transmission control protocol packet is received, and modify a network time-of-service protocol NTP timestamp of the SR packet received by the data packet receiving unit according to the recorded system time;

The switching control unit 430 is configured to perform synchronous switching according to an NTP timestamp in the SR packet of the media stream when performing media stream switching.

As shown in FIG. 5, in this embodiment, the handover control unit 430 may include:

The timestamp generating unit 431, according to the RTP timestamp in the last real-time transport protocol RTP packet of the media stream before the handover, and the NTP timestamp in the SR packet of the media stream before the handover and the NTP timestamp in the SR packet of the switched media stream, Recalculating the RTP timestamp in the first RTP packet of the media stream after the handover is generated;

The sending unit 432 is configured to add the RTP timestamp of the switched media stream calculated by the timestamp calculating unit to the RTP packet of the switched media stream, and send the RTP packet to the user.

The timestamp generating unit 431 is further configured to: regenerate an RTP timestamp for the subsequent data packet of the switched media stream according to the RTP timestamp in the first RTP packet of the re-calculated media stream, and receive the user The delay feedback adjusts the value of the RTP timestamp of the RTP packet sent according to the user's delay feedback.

The present invention provides another embodiment of a streaming media server. The structure is shown in FIG. 6. The receiving unit 610 is configured to receive an RTP packet of a media stream.

The timestamp replacement unit 620 is configured to replace the RTP timestamp in the RTP packet according to the system timestamp when the RTP arrives; the system timestamp is generated according to the system time, and a unique system timestamp is generated every unit time;

The switching control unit 630 is configured to perform synchronous switching according to a system timestamp in the RTP packet of the media stream. The method that can be operated by the foregoing embodiment of the device of the present invention is described with reference to the description of the foregoing method embodiments, and details are not described herein again.

A person skilled in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by using hardware related to program instructions, and the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed. The foregoing storage medium includes: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like, and the program code can be stored. Medium.

The media stream switching synchronization method and the streaming media server provided by the embodiments of the present invention are described above. The description of the above embodiments is only for helping to understand the method and core ideas of the present invention. Meanwhile, for those skilled in the art, The present invention is not limited by the scope of the present invention.

Claims

Rights request

A media stream switching synchronization method, comprising:

The streaming server receives the sender report packet of the media stream;

2. The method according to claim 1, wherein the synchronizing switching according to the network calibration time stamp in the sender report packet of the media stream comprises:

When the media stream is switched, the streaming media server according to the real-time transport protocol timestamp in the last real-time transport protocol packet of the media stream before the handover and the sender of the pre-switch media stream report the network calibration protocol timestamp and the switched media stream in the packet. The sender reports the timestamp of the network calibration protocol in the packet, and recalculates the real-time transport protocol timestamp in the first real-time transport protocol packet of the switched media stream; real-time transmission of the switched media stream calculated by the timestamp computing unit The protocol timestamp is added to the user in the real-time transport protocol packet of the switched media stream.

3. The method according to claim 2, wherein the recalculating the media stream after the switching of the real-time transport protocol timestamp in the first real-time transport protocol packet comprises:

And real-time transmission protocol time stamp in the first real-time transmission protocol packet of the re-calculated mobile media stream, and real-time transmission protocol time stamp is regenerated for the subsequent data packet of the switched media stream.

The method according to any one of claims 1 to 3, characterized in that the delay feedback of the user is received, and the value of the real-time transport protocol time stamp of the real-time transport protocol packet is adjusted according to the delay feedback of the user.

5. A media stream switching synchronization method, comprising:

The streaming media server replaces the real-time transport protocol timestamp in the real-time transport protocol packet according to the system timestamp when the real-time transport protocol arrives; the system timestamp is generated according to the system time, each unit Time generates a unique system timestamp;

6. A streaming media server, comprising:

a timestamp modification unit, configured to record a system time when the real-time transmission control protocol packet is received, and modify a network time-of-service protocol network calibration time stamp of the sender report packet according to the recorded system time;

The streaming media server according to claim 6, wherein the switching control unit comprises:

a timestamp calculation unit, according to a real-time transport protocol timestamp in the real-time transport protocol packet of the last real-time transport protocol of the media stream before the handover, and a network calibration timestamp and a switched media stream in the sender report packet of the pre-switch media stream The sender reports the timestamp of the network calibration protocol in the packet, and recalculates the real-time transport protocol timestamp in the first real-time transport protocol packet of the switched media stream; the sending unit is configured to calculate the switched media after the timestamp computing unit The real-time transport protocol timestamp of the stream is added to the real-time transport protocol packet of the switched media stream and sent to the user.

The method according to claim 7, wherein the timestamp calculation unit is further configured to: according to the recalculated real-time transmission protocol timestamp in the first real-time transmission protocol packet of the switched media stream, Regenerate the real-time transport protocol timestamp for subsequent packets of the switched media stream.

The streaming media server according to any one of claims 6 to 8, wherein the timestamp calculation unit is further configured to receive delay feedback of the user, and adjust and send the real-time transmission according to the delay feedback of the user. The value of the real-time transport protocol timestamp of the protocol packet.

10. A streaming media server, comprising:

a timestamp replacement unit, configured to replace a real-time transport protocol timestamp in the real-time transport protocol packet according to a system timestamp when the real-time transport protocol arrives; the system timestamp is generated according to a system time, Generate a unique system timestamp per unit time;

And a switching control unit, configured to perform synchronous switching according to a system timestamp in a real-time transport protocol packet of the media stream.