WO2013100754A1 - A system and method for adaptive media content delivery - Google Patents

A system and method for adaptive media content delivery Download PDF

Info

Publication number
WO2013100754A1
WO2013100754A1 PCT/MY2012/000171 MY2012000171W WO2013100754A1 WO 2013100754 A1 WO2013100754 A1 WO 2013100754A1 MY 2012000171 W MY2012000171 W MY 2012000171W WO 2013100754 A1 WO2013100754 A1 WO 2013100754A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
component
bandwidth
video
frames
Prior art date
Application number
PCT/MY2012/000171
Other languages
French (fr)
Inventor
Redika REMON
Woon Hon Hock
Hafriza Bin Zakaria KHAIRIL
Original Assignee
Mimos Berhad
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mimos Berhad filed Critical Mimos Berhad
Publication of WO2013100754A1 publication Critical patent/WO2013100754A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164Feedback from the receiver or from the transmission channel
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124Quantisation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/15Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402Monitoring of the downstream path of the transmission network, e.g. bandwidth available

Definitions

  • the present invention relates to a system and method for adaptive delivering of media content over a network having varying bandwidth capacities.
  • adaptive media content delivery adapts the quality of the media content to be delivered based on the client's bandwidth connection.
  • the quality of the media content can be scaled down for delivering to the client having a low-bandwidth connection.
  • an adaptive video transmission with variable frame rate includes: (a) measuring the transmission capacity to the receiver; (b) determining the rate of change of transmission capacity; and (c) adjusting the transmitted video frame rate as a function of the said rate of change.
  • a system for adaptively transporting video over networks comprises a video/audio codec that functions to compress, code, decode and decompress video streams that are transmitted over networks having available bandwidths that vary with time and location.
  • the system adjusts the compression ratio to accommodate a plurality of bandwidths ranging from 20 Kbps for POTS to several Mbps for switched LAN and ATM environments.
  • Bandwidth adjustability is provided by offering a trade off between video resolution, frame rate and individual frame quality.
  • the system generates a video data stream comprised of Key, P and B frames from a raw source of video. Each frame type is further comprised of multiple levels of data representing varying degrees of quality.
  • several video server platforms can be utilized in tandem to transmit video/audio information with each video server platform transmitting information for a single compression/resolution level.
  • a system for adaptive media content delivery comprises of at least one media source (110), a server (130), and a client device (150).
  • the server (130) includes at least one video analytic component (132), wherein the at least one video analytic component (132) is used to analyse event in images captured by the at least one media source (110); a bandwidth rate receiver component (135), wherein the client bandwidth rate receiver component (135) is used to estimate the bandwidth of the client device (150) connection; a quality decision component (136), wherein the quality decision component (13S) is used to perform image compression for each frame and selective temporal buffer frames based on bandwidth of the client device (150) connection and event captured in the images; and a video encoder (137), wherein the video encoder (137) is used to encode the compressed and selected temporal buffer frames from the quality decision component (136) into a video.
  • a method for delivering of a media content by using the system for adaptive media content delivery (100) comprises the steps of receiving a plurality of captured images or video frames from at least one media source (110) by a video acquisition component (131); analysing each frame by at least one video analytic component (132) to determine whether an event is detected in each frame; buffering the frames in a video buffering component (134); estimating bandwidth of a client device (150) connection by the client bandwidth rate receiver component (135); performing image compression on each frame and selective temporal buffer frames by a quality decision component (136), wherein the image compression and selective temporal buffer frames are performed based on the bandwidth of the client device (150) connection and the event detected in each frame; encoding the compressed and selected temporal buffer frames into video by a video encoder (137); and transmitting the video to the client device (150).
  • the bandwidth estimation includes the steps of sending a test frame from the client device (150) to the server (130); recording arrival time of the first bit and the last bit of the test frame; and calculating the bandwidth of the client device (150) connection based on data size of the test frame, arrival time of the last bit of the test frame, and the arrival time of the first bit of the test frame.
  • the step of performing image compression on each frame and selective temporal buffer frames by a quality decision component (136) includes the steps of: (a) selecting and extracting a frame from the video buffering component (134); (b) determining whether an event is detected in the frame, wherein if an event is detected in the frame, selecting a lower image compression ratio and more frequent frame sequence for the following frame by the quality decision component (136), and wherein if there is no event detected in the frame, selecting a higher image compression ratio and less frequent frame sequence for the following by frame the quality decision component (136); (c) compressing the frame based on the selected compression ratio; (d) packetizing the compressed frame with previous selected frames; (e) comparing total size of the packetized frames with the bandwidth of the client device (150) connection, wherein if the total size of the packetized frames is lower than the bandwidth estimated by the client bandwidth rate receiver component (135), repeating steps (a) to (d) for the following frame selected from the video buffering component (134), and wherein if the total size of the packet
  • a server (130) for delivering media content to a client device (150) comprises of a video acquisition component (131), a database (133), a video buffering component (134), and a socket listener component (138).
  • the server (130) further includes at least one video analytic component (132), wherein the at least one video analytic component (132) is used to analyse event in images captured by the at least one media source (110); a bandwidth rate receiver component (135), wherein the client bandwidth rate receiver component (135) is used to estimate the bandwidth of the client device (150) connection; a quality decision component (136), wherein the quality decision component (136) is used to perform image compression for each frame and selective temporal buffer frames based on bandwidth of the client device (150) connection and event captured in the images; and a video encoder (137), wherein the video encoder (137) is used to encode the compressed and selected temporal buffer frames from the quality decision component (136) into a video.
  • at least one video analytic component 132
  • the at least one video analytic component (132) is used to analyse event in images captured by the at least one media source (110)
  • a bandwidth rate receiver component (135) wherein the client bandwidth rate receiver component (135) is used to estimate the bandwidth of the client
  • a method for delivering of a media content by using the server (130) comprises the steps of receiving a plurality of captured images or video frames by a video acquisition component (131); analysing each frame by at least one video analytic component (132) to determine whether an event is detected in each frame; buffering the frames in a video buffering component (134); estimating bandwidth of a client device (150) connection by the client bandwidth rate receiver component (135); performing image compression on each frame and selective temporal buffer frames by a quality decision component (136), wherein the image compression and selective temporal buffer frames are performed based on the bandwidth of the client device (150) connection and the event detected in each frame; encoding the compressed and selected temporal buffer frames into video by a video encoder (137); and transmitting the video to the client device (150).
  • the bandwidth estimation includes the steps of sending a test frame from the client device (150) to the server (130); recording arrival time of the first bit and the last bit of the test frame; and calculating the bandwidth of the client device (150) connection based on data size of the test frame, arrival time of the last bit of the test frame, and the arrival time of the first bit of the test frame.
  • the step of performing image compression on each frame and selective temporal buffer frames by a quality decision component (136) includes the steps of: (a) selecting and extracting a frame from the video buffering component (134); (b) determining whether an event is detected in the frame, wherein if an event is detected in the frame, selecting a lower image compression ratio and more frequent frame sequence for the following frame by the quality decision component (136), and wherein if there is no event detected in the frame, selecting a higher image compression ratio and less frequent frame sequence for the following by frame the quality decision component (136); (c) compressing the frame based on the selected compression ratio; (d) packetizing the compressed frame with previous selected frames; (e) comparing total size of the packetized frames with the bandwidth of the client device (150) connection, wherein if the total size of the packetized frames is lower than the bandwidth estimated by the client bandwidth rate receiver component (135), repeating steps (a) to (d) for the following frame selected from the video buffering component (134), and wherein if the total size of the packet
  • FIG. 1 shows a block diagram of a system for adaptive media content delivery (100) according to an embodiment of the present invention.
  • FIG. 2 shows a flowchart of a method for adaptive delivering of media content using the system (100) of FIG. 1 according to an embodiment of the present invention.
  • FIG. 3 shows a flowchart of a method for performing image compression and selective temporal buffer frames by the quality decision component (136) by using a quality decision component (136) of the system (100) of FIG. 1 according to an embodiment of the present invention.
  • FIGS. 4(a-c) show a set of video frames undergoing the method of FIG. 3.
  • FIG. 1 there is shown a block diagram of a system for adaptive media content delivery (100) according to an embodiment of the present invention.
  • the system (100) adapts the quality of media content to be delivered based on bandwidth of a client device (150) connection and event detected in the media content. This allows the system (100) to adaptively send a media content while preserving the quality of frames capturing an event.
  • the term "event” used in the description and in the appended claims refers to a behaviour of an object captured in a media content that is detected and deemed as being suspicious by a surveillance system such as, but not limited to intrusion, loitering, slip and fall, unattended object and etc.
  • the system (100) generally comprises of a media source (110), a server (130), and a client device (150).
  • the media source (110) is used for capturing a sequence of images of a location area under surveillance.
  • the media source (110) is a video camera or any other image sensor device.
  • the images captured by the media source (110) are pulled by the server ( 30) as video through either a wired or wireless connection.
  • the server (130) is connected to the client device (150) through a network such as an Internet Protocol network.
  • the server (130) comprises of a video acquisition component (131), a video analytic component (132), a database (133), a video buffering component (134), bandwidth rate receiver component ( 35), a quality decision component (136), a video encoder (137), and a socket listener component (138).
  • the video acquisition component (131) is used to acquire the video from the media source (110).
  • the video acquisition component (131) decodes the video into a plurality of frames and thereon, it sends each frame to video analytic component (132) for event analysis.
  • the video analytic component (132) performs event analysis on each frame of the video to detect an event captured in the video.
  • the event analysis is performed by analysing each pixel in each frame to identify motion blobs, filtering noise from the frames, and applying event rules such as object tracking, intrusion detection and loitering detection. Based on the event analysis performed by the video analytic component (132), each frame is tagged with event information. The frames tagged with event information are stored in a database (133) and temporarily stored in the video buffering component (134).
  • the video buffering component (134) is used to temporarily store each frame tagged with event information based on the event analysis performed by the video analytic component (132) before transmitting those images to the quality decision component (136).
  • the video buffering component (134) is connected to the quality decision component (136).
  • the quality decision component (136) is used to select image compression ratio for each frame, and to select temporal buffer frames. The selection of image compression and temporal buffer frames is based on the bandwidth of the client device (150) connection and event information tagged with each frame.
  • the selected image compression ratio is used by the quality decision component (136) to compress each frame, wherein frame tagged with an event is compressed with a higher image compression ratio than the frame without an event.
  • the image compression used to reduce the images bit rate includes any intra-codec, lossy or lossless compression.
  • the quality decision component (136) is further connected to the video encoder (137) and the client bandwidth rate receiver component (135).
  • the client bandwidth rate receiver component (135) is used to estimate the bandwidth of the client device (150) connection. More specifically, the client bandwidth rate receiver component (135) receives bit rate value based on transmission rate and packets size yield from the client device (150).
  • the video encoder (137) is used to encode the compressed and selected temporal buffer frames from the quality decision component (136) into a video.
  • the video encoder (137) is further connected to the socket listener component ( 38).
  • the socket listener component (138) is an interface component to receive a request message from the client device (150) and to deliver the video encoded by the video encoder (137) to the client device (150) over a wired or wireless network.
  • the socket listener component (138) communicates with the client device (150) either through TCP protocol, UDP protocol, HTTP protocol or any other communication protocol.
  • the client device (150) is used for downloading and playing the media content from the server (130).
  • the client device (130) includes a socket downloader component (151) and a video player (152).
  • the socket downloader component (151) is an interface component which is used to send a request message to the server (130) and to receive the video from the server (130).
  • the socket downloader component (151) is connected to the video player (152).
  • the socket downloader component (151) communicates with the server (130) either through TCP protocol, UDP protocol, HTTP protocol or any other communication protocol.
  • the video player (152) is used to decode the received video and thereon, playing and displaying the decoded video on a monitor connected to the client device (150).
  • FIG. 2 there is shown a flowchart of a method for adaptive delivering of media content by using the system (100) of FIG. 1.
  • the video acquisition component (131) receives a video from the media source (110) and thereon, decodes the video into a plurality of frames (1 - 10) as illustrated in FIG. 4a.
  • each frame of the video is analysed by the video analytic component (132) to detect any event captured in the frames as in step 202.
  • Each frame is tagged with event information based on the event analysis performed by the video analytic component (132).
  • FIG. 4b illustrates each frame of the video of FIG.
  • step 203 the frames tagged with event information are buffered in the video buffering component (134).
  • step 204 the bandwidth of the client device (150) connection is estimated by the client bandwidth rate receiver component (135).
  • the bandwidth estimation includes the steps of sending a test frame from the client device (150) to the server (130), recording arrival time of the first bit and the last bit of the test frame, and calculating the bandwidth based on the equation below:
  • T is the arrival time of the last bit of the test frame
  • the quality decision component (136) performs image compression for each frame and selective temporal buffer frames based on the bandwidth of the client device (150) connection and event information tagged with each frame (step 205).
  • the image compression used to reduce the images bit rate includes any intra-codec, lossy or lossless compression.
  • the compressed and selected temporal buffer frames are then transmitted to the video encoder (137).
  • the video encoder (137) encodes the compressed and selected temporal buffer frames into video.
  • FIG. 4c illustrates the frames of FIG. 4b encoded by the video encoder (137) based on a selected temporal buffer frames, wherein the encoded video comprises of only the first frame (1), the third frame (3), the fourth frame (4), the fifth frame (5), and the seventh frame (7).
  • the encoded video is then transmitted to the client device (150) via the socket listener component (138) as in step 206.
  • the video received by the socket downloader component (151) are then decoded and played by video player (152) as in step 207.
  • step 301 the quality decision component (136) selects and extracts a frame from the video buffering component (134) and thereon, checks whether the frame is tagged with an event. If the frame is tagged with an event, then the quality decision component
  • the quality decision component (136) selects a higher image compression ratio and less frequent frame sequence for the following frame (decision 302 and steps 305 to 307).
  • the selected image compression ratio reduces the image quality of the frame.
  • the frame is compressed according to the selected image compression and packetized with other selected frames (step 308).
  • the total size of the packetized frames is compared with the bandwidth of the client device (150) connection.
  • the quality decision component (136) repeats step 301 to decision 309, wherein the quality decision component (136) extracts the following frame from the video buffering component (134) to check whether the frame is tagged with an event. The following frame is selected and extracted based on the previous frame packetized.
  • the quality decision component (136) determines whether the packetized frames can be compressed to reduce it to a size lower than or equal to the estimated bandwidth as in decision 310. If the packetized frames can be reduced, the quality decision component (136) compresses the packetized frames and thereon, sends the packetized frames to the video encoder (137) as in step 311 and 313. Otherwise, the quality decision component (136) discards the frame from the packetized frames and sends the packetized frames to the video encoder (137) as in step 312 and 313. The frame which is discarded from the packetized frames is included for the following packetized frames to be sent to the video encoder (137).
  • the quality decision component (136) sends the packetized frames to the video encoder (137) as in step 313.
  • FIG. 4b and FIG. 4c are referred to illustrate an example of the frames stored in the video buffering component (134) and the packetized frames.
  • the size for each frame is 180kbit and the estimated bandwidth is 750kbit/s.
  • the quality decision component (136) selects and extracts the first frame (1) stored in the video buffering component (134). Since the first frame (1) is tagged with no event, the quality decision component (136) selects a higher image compression ratio. Moreover, the quality decision component (136) selects a less frequent frame sequence for the following frame, wherein the quality decision component (136) skips a frame to select the third frame (3) as the following frame. Thereon, the first frame (1) is compressed to a size of 120kbit and the compressed first frame (1) is packetized. The total size of the packetized frames is compared with the bandwidth of the client device (150) connection. Since the total size of the packetized frames of 120kbit is lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the third frame (3) stored in the video buffering component (134).
  • the third frame (3) is tagged with an event and thus, the quality decision component (136) selects a lower image compression ratio. Moreover, the quality decision component (136) selects more frequent frame sequence for the following frame, wherein the quality decision component (136) selects the fourth frame (4) as the following frame. Thereon, the third frame (3) is compressed to a size of 180kbit and then, packetized with the first frame (1). The total size of the packetized frames is compared with the bandwidth of the client device ( 50) connection. Since the total size of the packetized frames of 300kbit is lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the fourth frame (4) stored in the video buffering component (134).
  • the fourth frame (4) is tagged with an event and thus, the quality decision component (136) selects a higher image compression ratio. Moreover, the quality decision component (136) selects more frequent frame sequence for the following frame, wherein the quality decision component (136) selects the fourth frame (5) as the following frame. Thereon, the fourth frame (4) is compressed to a size of 180kbit and then, packetized with the first frame (1) and the third frame (3). The total size of the packetized frames is compared with the bandwidth of the client device (150) connection. Since the total size of the packetized frames of 480kbit is still lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the fifth frame (5) stored in the video buffering component (134).
  • the fifth frame (5) is tagged with no event and thus, the quality decision component (136) selects a higher image compression ratio. Moreover, the quality decision component (136) selects a less frequent frame sequence for the following frame, wherein the quality decision component (136) skips a frame to select the seventh frame (7) as the following frame. Thereon, the fifth frame (5) is compressed to a size of 120kbit and then, packetized with the first frame (1), the third frame (3) and the fourth frame (4). The total size of the packetized frames is compared with the bandwidth of the client device (150) connection. Since the total size of the packetized frames of 600kbit is lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the seventh frame (7) stored in the video buffering component (134).
  • the seventh frame (7) is tagged with no event and thus, the quality decision component (136) selects a higher image compression ratio. Moreover, the quality decision component (136) selects a less frequent frame sequence for the following frame, wherein the quality decision component (136) skips two frames to select the tenth frame (10) as the following frame. Thereon, the seventh frame (7) is compressed to a size of 100kbit and then, packetized with the first frame (1), the third frame (3), the fourth frame (4) and the fifth frame (5). The total size of the packetized frames is compared with the bandwidth of the client device (150) connection. Since the total size of the packetized frames of 700kbit is still lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the tenth frame (10) stored in the video buffering component (134).
  • the tenth frame (10) is tagged with an event and thus, the quality decision component (136) selects a lower image compression ratio. Moreover, the quality decision component (136) selects more frequent frame sequence for the following frame. Thereon, the tenth frame (10) is compressed to a size of 180kbit and then, packetized with the first frame (1), the third frame (3), the fourth frame (4), the fifth frame (5) and the seventh frame (7). The total size of the packetized frames is compared with the bandwidth of the client device (150) connection.
  • the quality decision component (136) discards the tenth frame (10) from the packetized frames.
  • the packetized frames comprising the first frame (1), the third frame (3), the fourth frame (4), the fifth frame (5) and the seventh frame (7) as shown in FIG. 4c are then sent to the video encoder (137).
  • the tenth frame (10) will be included for the following packetized frames to be sent to the video encoder (137).

Abstract

The present invention relates to a system for adaptive media content delivery (100). The system (100) adapts the quality of media content to be delivered based on bandwidth of a client device (150) connection and event detected in the media content. This allows the system (100) to adaptively send a media content while preserving the quality of frames capturing an event. The system (100) comprises of a media source (110), a server (130), and a client device (150). The server (130) includes a video acquisition component (131), a video analytic component (132), a database (133), a video buffering component (134), bandwidth rate receiver component (135), a quality decision component (136), a video encoder (137), and a socket listener component (138). The client device (130) includes a socket downloader component (151) and a video player (152).

Description

A SYSTEM AND METHOD FOR ADAPTIVE MEDIA CONTENT DELIVERY
FIELD OF INVENTION
The present invention relates to a system and method for adaptive delivering of media content over a network having varying bandwidth capacities.
BACKGROUND OF THE INVENTION
Delivery of media content such as video and audio over a network requires coping with bandwidth which varies with time and location. This variation in bandwidth causes data losses and latency in the media content being delivered. Thus, a client with low-bandwidth connection experiences interruption and jittering playback of the delivered media content. In regard to this, adaptive media content delivery is developed. Such adaptive media content delivery adapts the quality of the media content to be delivered based on the client's bandwidth connection. Thus, the quality of the media content can be scaled down for delivering to the client having a low-bandwidth connection.
In European Patent No. 1777969, an adaptive video transmission with variable frame rate is provided. The method includes: (a) measuring the transmission capacity to the receiver; (b) determining the rate of change of transmission capacity; and (c) adjusting the transmitted video frame rate as a function of the said rate of change.
In US Patent No. 6014694, a system for adaptively transporting video over networks is disclosed. The system comprises a video/audio codec that functions to compress, code, decode and decompress video streams that are transmitted over networks having available bandwidths that vary with time and location. Depending on the channel bandwidth, the system adjusts the compression ratio to accommodate a plurality of bandwidths ranging from 20 Kbps for POTS to several Mbps for switched LAN and ATM environments. Bandwidth adjustability is provided by offering a trade off between video resolution, frame rate and individual frame quality. The system generates a video data stream comprised of Key, P and B frames from a raw source of video. Each frame type is further comprised of multiple levels of data representing varying degrees of quality. In addition, several video server platforms can be utilized in tandem to transmit video/audio information with each video server platform transmitting information for a single compression/resolution level.
However, such adaptive media content delivery may not be suitable for a surveillance system. The clarity of the media content and the event carried by the media content are essential elements for forensic processing of the media content. Thus, by reducing the quality of the delivered media content, suspicious event that is captured in the media content cannot be seen clearly through a playback at the client.
Therefore, there is a need to provide a system for adaptive delivering of media content while considering events captured in the media content.
SUMMARY OF INVENTION
According to a first aspect of the present invention, a system for adaptive media content delivery (100) is provided. The system (100) comprises of at least one media source (110), a server (130), and a client device (150). The server (130) includes at least one video analytic component (132), wherein the at least one video analytic component (132) is used to analyse event in images captured by the at least one media source (110); a bandwidth rate receiver component (135), wherein the client bandwidth rate receiver component (135) is used to estimate the bandwidth of the client device (150) connection; a quality decision component (136), wherein the quality decision component (13S) is used to perform image compression for each frame and selective temporal buffer frames based on bandwidth of the client device (150) connection and event captured in the images; and a video encoder (137), wherein the video encoder (137) is used to encode the compressed and selected temporal buffer frames from the quality decision component (136) into a video.
According to a second aspect of the present invention, a method for delivering of a media content by using the system for adaptive media content delivery (100) is provided. The method comprises the steps of receiving a plurality of captured images or video frames from at least one media source (110) by a video acquisition component (131); analysing each frame by at least one video analytic component (132) to determine whether an event is detected in each frame; buffering the frames in a video buffering component (134); estimating bandwidth of a client device (150) connection by the client bandwidth rate receiver component (135); performing image compression on each frame and selective temporal buffer frames by a quality decision component (136), wherein the image compression and selective temporal buffer frames are performed based on the bandwidth of the client device (150) connection and the event detected in each frame; encoding the compressed and selected temporal buffer frames into video by a video encoder (137); and transmitting the video to the client device (150).
Preferably, the bandwidth estimation includes the steps of sending a test frame from the client device (150) to the server (130); recording arrival time of the first bit and the last bit of the test frame; and calculating the bandwidth of the client device (150) connection based on data size of the test frame, arrival time of the last bit of the test frame, and the arrival time of the first bit of the test frame. Preferably, the step of performing image compression on each frame and selective temporal buffer frames by a quality decision component (136) includes the steps of: (a) selecting and extracting a frame from the video buffering component (134); (b) determining whether an event is detected in the frame, wherein if an event is detected in the frame, selecting a lower image compression ratio and more frequent frame sequence for the following frame by the quality decision component (136), and wherein if there is no event detected in the frame, selecting a higher image compression ratio and less frequent frame sequence for the following by frame the quality decision component (136); (c) compressing the frame based on the selected compression ratio; (d) packetizing the compressed frame with previous selected frames; (e) comparing total size of the packetized frames with the bandwidth of the client device (150) connection, wherein if the total size of the packetized frames is lower than the bandwidth estimated by the client bandwidth rate receiver component (135), repeating steps (a) to (d) for the following frame selected from the video buffering component (134), and wherein if the total size of the packetized frames is higher than the bandwidth estimated by the client bandwidth rate receiver component (135), compressing the packetized frames if the packetized frames can be reduced to a size lower than or equal to the estimated bandwidth or discarding the frame from the packetized frames if the packetized frames cannot be reduced to a size lower than or equal to the estimated bandwidth; and (f) sending the packetized frames to the video encoder (137). According to a third aspect of the present invention, a server (130) for delivering media content to a client device (150) is provided. The server (130) comprises of a video acquisition component (131), a database (133), a video buffering component (134), and a socket listener component (138). Moreover, the server (130) further includes at least one video analytic component (132), wherein the at least one video analytic component (132) is used to analyse event in images captured by the at least one media source (110); a bandwidth rate receiver component (135), wherein the client bandwidth rate receiver component (135) is used to estimate the bandwidth of the client device (150) connection; a quality decision component (136), wherein the quality decision component (136) is used to perform image compression for each frame and selective temporal buffer frames based on bandwidth of the client device (150) connection and event captured in the images; and a video encoder (137), wherein the video encoder (137) is used to encode the compressed and selected temporal buffer frames from the quality decision component (136) into a video.
According to a fourth aspect of the present invention, a method for delivering of a media content by using the server (130) is provided. The method comprises the steps of receiving a plurality of captured images or video frames by a video acquisition component (131); analysing each frame by at least one video analytic component (132) to determine whether an event is detected in each frame; buffering the frames in a video buffering component (134); estimating bandwidth of a client device (150) connection by the client bandwidth rate receiver component (135); performing image compression on each frame and selective temporal buffer frames by a quality decision component (136), wherein the image compression and selective temporal buffer frames are performed based on the bandwidth of the client device (150) connection and the event detected in each frame; encoding the compressed and selected temporal buffer frames into video by a video encoder (137); and transmitting the video to the client device (150).
Preferably, the bandwidth estimation includes the steps of sending a test frame from the client device (150) to the server (130); recording arrival time of the first bit and the last bit of the test frame; and calculating the bandwidth of the client device (150) connection based on data size of the test frame, arrival time of the last bit of the test frame, and the arrival time of the first bit of the test frame.
Preferably, the step of performing image compression on each frame and selective temporal buffer frames by a quality decision component (136) includes the steps of: (a) selecting and extracting a frame from the video buffering component (134); (b) determining whether an event is detected in the frame, wherein if an event is detected in the frame, selecting a lower image compression ratio and more frequent frame sequence for the following frame by the quality decision component (136), and wherein if there is no event detected in the frame, selecting a higher image compression ratio and less frequent frame sequence for the following by frame the quality decision component (136); (c) compressing the frame based on the selected compression ratio; (d) packetizing the compressed frame with previous selected frames; (e) comparing total size of the packetized frames with the bandwidth of the client device (150) connection, wherein if the total size of the packetized frames is lower than the bandwidth estimated by the client bandwidth rate receiver component (135), repeating steps (a) to (d) for the following frame selected from the video buffering component (134), and wherein if the total size of the packetized frames is higher than the bandwidth estimated by the client bandwidth rate receiver component (135), compressing the packetized frames if the packetized frames can be reduced to a size lower than or equal to the estimated bandwidth or discarding the frame from the packetized frames if the packetized frames cannot be reduced to a size lower than or equal to the estimated bandwidth; and (f) sending the packetized frames to the video encoder (137).
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
FIG. 1 shows a block diagram of a system for adaptive media content delivery (100) according to an embodiment of the present invention.
FIG. 2 shows a flowchart of a method for adaptive delivering of media content using the system (100) of FIG. 1 according to an embodiment of the present invention. FIG. 3 shows a flowchart of a method for performing image compression and selective temporal buffer frames by the quality decision component (136) by using a quality decision component (136) of the system (100) of FIG. 1 according to an embodiment of the present invention.
FIGS. 4(a-c) show a set of video frames undergoing the method of FIG. 3.
DESCRIPTION OF THE PREFFERED EMBODIMENT
A preferred embodiment of the present invention will be described herein below with reference to the accompanying drawings. In the following description, well known functions or constructions are not described in detail since they would obscure the description with unnecessary detail. Referring now to FIG. 1 , there is shown a block diagram of a system for adaptive media content delivery (100) according to an embodiment of the present invention. The system (100) adapts the quality of media content to be delivered based on bandwidth of a client device (150) connection and event detected in the media content. This allows the system (100) to adaptively send a media content while preserving the quality of frames capturing an event. The term "event" used in the description and in the appended claims refers to a behaviour of an object captured in a media content that is detected and deemed as being suspicious by a surveillance system such as, but not limited to intrusion, loitering, slip and fall, unattended object and etc. The system (100) generally comprises of a media source (110), a server (130), and a client device (150).
The media source (110) is used for capturing a sequence of images of a location area under surveillance. Preferably, the media source (110) is a video camera or any other image sensor device. The images captured by the media source (110) are pulled by the server ( 30) as video through either a wired or wireless connection.
The server (130) is connected to the client device (150) through a network such as an Internet Protocol network. The server (130) comprises of a video acquisition component (131), a video analytic component (132), a database (133), a video buffering component (134), bandwidth rate receiver component ( 35), a quality decision component (136), a video encoder (137), and a socket listener component (138). The video acquisition component (131) is used to acquire the video from the media source (110). Moreover, the video acquisition component (131) decodes the video into a plurality of frames and thereon, it sends each frame to video analytic component (132) for event analysis. The video analytic component (132) performs event analysis on each frame of the video to detect an event captured in the video. The event analysis is performed by analysing each pixel in each frame to identify motion blobs, filtering noise from the frames, and applying event rules such as object tracking, intrusion detection and loitering detection. Based on the event analysis performed by the video analytic component (132), each frame is tagged with event information. The frames tagged with event information are stored in a database (133) and temporarily stored in the video buffering component (134).
The video buffering component (134) is used to temporarily store each frame tagged with event information based on the event analysis performed by the video analytic component (132) before transmitting those images to the quality decision component (136). The video buffering component (134) is connected to the quality decision component (136). The quality decision component (136) is used to select image compression ratio for each frame, and to select temporal buffer frames. The selection of image compression and temporal buffer frames is based on the bandwidth of the client device (150) connection and event information tagged with each frame. The selected image compression ratio is used by the quality decision component (136) to compress each frame, wherein frame tagged with an event is compressed with a higher image compression ratio than the frame without an event. The image compression used to reduce the images bit rate includes any intra-codec, lossy or lossless compression. The quality decision component (136) is further connected to the video encoder (137) and the client bandwidth rate receiver component (135). The client bandwidth rate receiver component (135) is used to estimate the bandwidth of the client device (150) connection. More specifically, the client bandwidth rate receiver component (135) receives bit rate value based on transmission rate and packets size yield from the client device (150).
The video encoder (137) is used to encode the compressed and selected temporal buffer frames from the quality decision component (136) into a video. The video encoder (137) is further connected to the socket listener component ( 38). The socket listener component (138) is an interface component to receive a request message from the client device (150) and to deliver the video encoded by the video encoder (137) to the client device (150) over a wired or wireless network. Moreover, the socket listener component (138) communicates with the client device (150) either through TCP protocol, UDP protocol, HTTP protocol or any other communication protocol.
The client device (150) is used for downloading and playing the media content from the server (130). The client device (130) includes a socket downloader component (151) and a video player (152).
The socket downloader component (151) is an interface component which is used to send a request message to the server (130) and to receive the video from the server (130). The socket downloader component (151) is connected to the video player (152). Moreover, the socket downloader component (151) communicates with the server (130) either through TCP protocol, UDP protocol, HTTP protocol or any other communication protocol.
The video player (152) is used to decode the received video and thereon, playing and displaying the decoded video on a monitor connected to the client device (150).
Referring now to FIG. 2, there is shown a flowchart of a method for adaptive delivering of media content by using the system (100) of FIG. 1. Initially, as in step 201 , the video acquisition component (131) receives a video from the media source (110) and thereon, decodes the video into a plurality of frames (1 - 10) as illustrated in FIG. 4a. Thereon, each frame of the video is analysed by the video analytic component (132) to detect any event captured in the frames as in step 202. Each frame is tagged with event information based on the event analysis performed by the video analytic component (132). As an example, FIG. 4b illustrates each frame of the video of FIG. 4a is tagged with event information, wherein a first frame (1), a second frame (2), a fifth frame (5), a sixth frame (6), an eighth frame (8), and a ninth frame (9) are tagged with no event while, a third frame (3), a fourth frame (4), and a tenth frame (10) are tagged with having an event. In step 203, the frames tagged with event information are buffered in the video buffering component (134). Next, in step 204, the bandwidth of the client device (150) connection is estimated by the client bandwidth rate receiver component (135). The bandwidth estimation includes the steps of sending a test frame from the client device (150) to the server (130), recording arrival time of the first bit and the last bit of the test frame, and calculating the bandwidth based on the equation below:
Figure imgf000011_0001
where R is the bandwidth of the client device (150) connection, 'Size' is the data si
size of the test frame, T is the arrival time of the last bit of the test frame, and T is
sn bO the arrival time of the first bit of the test frame.
Once the bandwidth of the client device (150) connection has been estimated, the quality decision component (136) performs image compression for each frame and selective temporal buffer frames based on the bandwidth of the client device (150) connection and event information tagged with each frame (step 205). The image compression used to reduce the images bit rate includes any intra-codec, lossy or lossless compression. The compressed and selected temporal buffer frames are then transmitted to the video encoder (137).
Thereon, the video encoder (137) encodes the compressed and selected temporal buffer frames into video. FIG. 4c illustrates the frames of FIG. 4b encoded by the video encoder (137) based on a selected temporal buffer frames, wherein the encoded video comprises of only the first frame (1), the third frame (3), the fourth frame (4), the fifth frame (5), and the seventh frame (7). The encoded video is then transmitted to the client device (150) via the socket listener component (138) as in step 206. The video received by the socket downloader component (151) are then decoded and played by video player (152) as in step 207.
Referring now to F!G. 3, there is shown a flowchart of a method for performing image compression and selective temporal buffer frames by the quality decision component (136). In step 301 , the quality decision component (136) selects and extracts a frame from the video buffering component (134) and thereon, checks whether the frame is tagged with an event. If the frame is tagged with an event, then the quality decision component
(136) selects a lower image compression ratio and more frequent frame sequence for the following frame (decision 302, and steps 303 to 304).
However, if the frame is not tagged with an event, the quality decision component (136) selects a higher image compression ratio and less frequent frame sequence for the following frame (decision 302 and steps 305 to 307). Thus, the selected image compression ratio reduces the image quality of the frame.
Based on the compression ratio selected, the frame is compressed according to the selected image compression and packetized with other selected frames (step 308).
Thereon, as in decision 309, the total size of the packetized frames is compared with the bandwidth of the client device (150) connection.
If the total size of the packetized frames is lower than the bandwidth estimated by the client bandwidth rate receiver component (135), the quality decision component (136) repeats step 301 to decision 309, wherein the quality decision component (136) extracts the following frame from the video buffering component (134) to check whether the frame is tagged with an event. The following frame is selected and extracted based on the previous frame packetized.
If the total size of the packetized frames is higher than the bandwidth estimated by the client bandwidth rate receiver component (135), the quality decision component (136) determines whether the packetized frames can be compressed to reduce it to a size lower than or equal to the estimated bandwidth as in decision 310. If the packetized frames can be reduced, the quality decision component (136) compresses the packetized frames and thereon, sends the packetized frames to the video encoder (137) as in step 311 and 313. Otherwise, the quality decision component (136) discards the frame from the packetized frames and sends the packetized frames to the video encoder (137) as in step 312 and 313. The frame which is discarded from the packetized frames is included for the following packetized frames to be sent to the video encoder (137). If the total size of the packetized frames is similar to the bandwidth estimated by the client bandwidth rate receiver component (135), the quality decision component (136) sends the packetized frames to the video encoder (137) as in step 313. To provide a better understanding of the abovementioned steps performed by the quality decision component ( 36), FIG. 4b and FIG. 4c are referred to illustrate an example of the frames stored in the video buffering component (134) and the packetized frames. The size for each frame is 180kbit and the estimated bandwidth is 750kbit/s.
Initially, the quality decision component (136) selects and extracts the first frame (1) stored in the video buffering component (134). Since the first frame (1) is tagged with no event, the quality decision component (136) selects a higher image compression ratio. Moreover, the quality decision component (136) selects a less frequent frame sequence for the following frame, wherein the quality decision component (136) skips a frame to select the third frame (3) as the following frame. Thereon, the first frame (1) is compressed to a size of 120kbit and the compressed first frame (1) is packetized. The total size of the packetized frames is compared with the bandwidth of the client device (150) connection. Since the total size of the packetized frames of 120kbit is lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the third frame (3) stored in the video buffering component (134).
The third frame (3) is tagged with an event and thus, the quality decision component (136) selects a lower image compression ratio. Moreover, the quality decision component (136) selects more frequent frame sequence for the following frame, wherein the quality decision component (136) selects the fourth frame (4) as the following frame. Thereon, the third frame (3) is compressed to a size of 180kbit and then, packetized with the first frame (1). The total size of the packetized frames is compared with the bandwidth of the client device ( 50) connection. Since the total size of the packetized frames of 300kbit is lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the fourth frame (4) stored in the video buffering component (134). The fourth frame (4) is tagged with an event and thus, the quality decision component (136) selects a higher image compression ratio. Moreover, the quality decision component (136) selects more frequent frame sequence for the following frame, wherein the quality decision component (136) selects the fourth frame (5) as the following frame. Thereon, the fourth frame (4) is compressed to a size of 180kbit and then, packetized with the first frame (1) and the third frame (3). The total size of the packetized frames is compared with the bandwidth of the client device (150) connection. Since the total size of the packetized frames of 480kbit is still lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the fifth frame (5) stored in the video buffering component (134).
The fifth frame (5) is tagged with no event and thus, the quality decision component (136) selects a higher image compression ratio. Moreover, the quality decision component (136) selects a less frequent frame sequence for the following frame, wherein the quality decision component (136) skips a frame to select the seventh frame (7) as the following frame. Thereon, the fifth frame (5) is compressed to a size of 120kbit and then, packetized with the first frame (1), the third frame (3) and the fourth frame (4). The total size of the packetized frames is compared with the bandwidth of the client device (150) connection. Since the total size of the packetized frames of 600kbit is lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the seventh frame (7) stored in the video buffering component (134).
The seventh frame (7) is tagged with no event and thus, the quality decision component (136) selects a higher image compression ratio. Moreover, the quality decision component (136) selects a less frequent frame sequence for the following frame, wherein the quality decision component (136) skips two frames to select the tenth frame (10) as the following frame. Thereon, the seventh frame (7) is compressed to a size of 100kbit and then, packetized with the first frame (1), the third frame (3), the fourth frame (4) and the fifth frame (5). The total size of the packetized frames is compared with the bandwidth of the client device (150) connection. Since the total size of the packetized frames of 700kbit is still lower than the estimated bandwidth of 750kbit/s, the quality decision component (136) selects and extracts the following frame which is the tenth frame (10) stored in the video buffering component (134).
The tenth frame (10) is tagged with an event and thus, the quality decision component (136) selects a lower image compression ratio. Moreover, the quality decision component (136) selects more frequent frame sequence for the following frame. Thereon, the tenth frame (10) is compressed to a size of 180kbit and then, packetized with the first frame (1), the third frame (3), the fourth frame (4), the fifth frame (5) and the seventh frame (7). The total size of the packetized frames is compared with the bandwidth of the client device (150) connection. Since the total size of the packetized frames of 880kbit is higher than the estimated bandwidth of 750kbit/s and it cannot be compressed to a size lower than or equal to the estimated bandwidth, the quality decision component (136) discards the tenth frame (10) from the packetized frames. The packetized frames comprising the first frame (1), the third frame (3), the fourth frame (4), the fifth frame (5) and the seventh frame (7) as shown in FIG. 4c are then sent to the video encoder (137). The tenth frame (10) will be included for the following packetized frames to be sent to the video encoder (137).
Although described in the abovementioned description that only one media source (110) is connected to the server (130), it is appreciated by a person skilled in the art that more than one media source (110) can be connected to the server (130).
While embodiments of the invention have been illustrated and described, it is not intended that these embodiments illustrated and describe all possible forms of the invention. Rather, the words used in the specifications are words of description rather than limitation and various changes may be made without departing from the scope of the invention.

Claims

1. A system for adaptive media content delivery (100) comprising at least one media source (110), a server (130), and a client device (150), wherein the system (100) is characterised in that the server (130) includes:
a) at least one video analytic component (132), wherein the at least one video analytic component (132) is used to analyse event in images captured by the at least one media source (110);
b) a bandwidth rate receiver component (135), wherein the client bandwidth rate receiver component (135) is used to estimate the bandwidth of the client device (150) connection;
c) a quality decision component (136), wherein the quality decision component (136) is used to perform image compression for each frame and selective temporal buffer frames based on bandwidth of the client device (150) connection and event captured in the images; and d) a video encoder (137), wherein the video encoder (137) is used to encode the compressed and selected temporal buffer frames from the quality decision component (136) into a video.
2. A method for delivering of a media content by using the system (100) as claimed in claim 1 , comprising the steps of:
a) receiving a plurality of captured images or video frames from at least one media source (110) by a video acquisition component (131);
b) analysing each frame by at least one video analytic component (132) to determine whether an event is detected in each frame; c) buffering the frames in a video buffering component (134); d) estimating bandwidth of a client device (150) connection by the client bandwidth rate receiver component (135);
e) performing image compression on each frame and selective temporal buffer frames by a quality decision component (136), wherein the image compression and selective temporal buffer frames are performed based on the bandwidth of the client device (150) connection and the event detected in each frame;
f) encoding the compressed and selected temporal buffer frames into video by a video encoder (137); and
g) transmitting the video to the client device (150). The method as claimed in claim 2, wherein the bandwidth estimation includes the steps of:
a) sending a test frame from the client device (150) to the server (130); b) recording arrival time of the first bit and the last bit of the test frame; and
c) calculating the bandwidth of the client device (150) connection based on data size of the test frame, arrival time of the last bit of the test frame, and the arrival time of the first bit of the test frame.
The method as claimed in claim 2, wherein the step of performing image compression on each frame and selective temporal buffer frames by a quality decision component (136) includes the steps of:
a) selecting and extracting a frame from the video buffering component (134);
b) determining whether an event is detected in the frame, wherein if an event is detected in the frame, selecting a lower image compression ratio and more frequent frame sequence for the following frame by the quality decision component (136), and wherein if there is no event detected in the frame, selecting a higher image compression ratio and less frequent frame sequence for the following by frame the quality decision component (136);
c) compressing the frame based on the selected compression ratio; d) packetizing the compressed frame with previous selected frames; e) comparing total size of the packetized frames with the bandwidth of the client device (150) connection, wherein if the total size of the packetized frames is lower than the bandwidth estimated by the client bandwidth rate receiver component (135), repeating steps (a) to (d) for the following frame selected from the video buffering component (134), and wherein if the total size of the packetized frames is higher than the bandwidth estimated by the client bandwidth rate receiver component (135), compressing the packetized frames if the packetized frames can be reduced to a size lower than or equal to the estimated bandwidth or discarding the frame from the packetized frames if the packetized frames cannot be reduced to a size lower than or equal to the estimated bandwidth; and
f) sending the packetized frames to the video encoder (137).
A server (130) for delivering media content to a client device (150) comprising a video acquisition component (131), a database (133), a video buffering component (134), and a socket listener component (138), characterised in that the server (130) further includes:
a) at least one video analytic component (132), wherein the at least one video analytic component (132) is used to analyse event in images captured by the at least one media source (110);
b) a bandwidth rate receiver component (135), wherein the client bandwidth rate receiver component (135) is used to estimate the bandwidth of the client device (150) connection;
c) a quality decision component (136), wherein the quality decision component (136) is used to perform image compression for each frame and selective temporal buffer frames based on bandwidth of the client device (150) connection and event captured in the images; and d) a video encoder (137), wherein the video encoder (137) is used to encode the compressed and selected temporal buffer frames from the quality decision component (136) into a video.
A method for delivering of a media content by using the server (130) as claimed in claim 5, comprising the steps of:
a) receiving a plurality of captured images or video frames by a video acquisition component (131);
b) analysing each frame by at least one video analytic component (132) to determine whether an event is detected in each frame;
c) buffering the frames in a video buffering component (134);
d) estimating bandwidth of a client device (150) connection by the client bandwidth rate receiver component (135);
e) performing image compression on each frame and selective temporal buffer frames by a quality decision component (136), wherein the image compression and selective temporal buffer frames are performed based on the bandwidth of the client device (150) connection and the event detected in each frame;
f) encoding the compressed and selected temporal buffer frames into video by a video encoder (137); and
g) transmitting the video to the client device (150).
The method as claimed in claim 6, wherein the bandwidth estimation includes the steps of:
a) sending a test frame from the client device (150) to the server (130); b) recording arrival time of the first bit and the last bit of the test frame; and
c) calculating the bandwidth of the client device (150) connection based on data size of the test frame, arrival time of the last bit of the test frame, and the arrival time of the first bit of the test frame.
The method as claimed in claim 6, wherein the step of performing image compression on each frame and selective temporal buffer frames by a quality decision component (136) includes the steps of:
a) selecting and extracting a frame from the video buffering component (134);
b) determining whether an event is detected in the frame, wherein if an event is detected in the frame, selecting a lower image compression ratio and more frequent frame sequence for the following frame by the quality decision component (136), and wherein if there is no event detected in the frame, selecting a higher image compression ratio and less frequent frame sequence for the following by frame the quality decision component (136);
c) compressing the frame based on the selected compression ratio; d) packetizing the compressed frame with previous selected frames; e) comparing total size of the packetized frames with the bandwidth of the client device (150) connection, wherein if the total size of the packetized frames is lower than the bandwidth estimated by the client bandwidth rate receiver component (135), repeating steps (a) to (d) for the following frame selected from the video buffering component (134), and wherein if the total size of the packetized frames is higher than the bandwidth estimated by the client bandwidth rate receiver component (135), compressing the packetized frames if the packetized frames can be reduced to a size lower than or equal to the estimated bandwidth or discarding the frame from the packetized frames if the packetized frames cannot be reduced to a size lower than or equal to the estimated bandwidth; and
sending the packetized frames to the video encoder (137).
PCT/MY2012/000171 2011-12-28 2012-06-29 A system and method for adaptive media content delivery WO2013100754A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MYPI2011700206A MY173712A (en) 2011-12-28 2011-12-28 A system and method for adaptive media content delivery
MYPI2011700206 2011-12-28

Publications (1)

Publication Number Publication Date
WO2013100754A1 true WO2013100754A1 (en) 2013-07-04

Family

ID=46754742

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/MY2012/000171 WO2013100754A1 (en) 2011-12-28 2012-06-29 A system and method for adaptive media content delivery

Country Status (2)

Country Link
MY (1) MY173712A (en)
WO (1) WO2013100754A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104735485A (en) * 2015-03-05 2015-06-24 上海小蚁科技有限公司 Method and device for playing video
US10178203B1 (en) 2014-09-23 2019-01-08 Vecima Networks Inc. Methods and systems for adaptively directing client requests to device specific resource locators

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6014694A (en) 1997-06-26 2000-01-11 Citrix Systems, Inc. System for adaptive video/audio transport over a network
WO2001027763A1 (en) * 1999-10-08 2001-04-19 Ivex Corporation Networked digital security system and methods
EP1777969A1 (en) 2005-10-10 2007-04-25 BRITISH TELECOMMUNICATIONS public limited company Adaptive video transmission with variable frame rate
WO2008119043A1 (en) * 2007-03-27 2008-10-02 Armida Technologies Wireless integrated security controller
US20110119716A1 (en) * 2009-03-12 2011-05-19 Mist Technology Holdings, Inc. System and Method for Video Distribution Management with Mobile Services

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6014694A (en) 1997-06-26 2000-01-11 Citrix Systems, Inc. System for adaptive video/audio transport over a network
WO2001027763A1 (en) * 1999-10-08 2001-04-19 Ivex Corporation Networked digital security system and methods
EP1777969A1 (en) 2005-10-10 2007-04-25 BRITISH TELECOMMUNICATIONS public limited company Adaptive video transmission with variable frame rate
WO2008119043A1 (en) * 2007-03-27 2008-10-02 Armida Technologies Wireless integrated security controller
US20110119716A1 (en) * 2009-03-12 2011-05-19 Mist Technology Holdings, Inc. System and Method for Video Distribution Management with Mobile Services

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10178203B1 (en) 2014-09-23 2019-01-08 Vecima Networks Inc. Methods and systems for adaptively directing client requests to device specific resource locators
CN104735485A (en) * 2015-03-05 2015-06-24 上海小蚁科技有限公司 Method and device for playing video

Also Published As

Publication number Publication date
MY173712A (en) 2020-02-17

Similar Documents

Publication Publication Date Title
US8238441B2 (en) Method and system for scalable representation, storage, transmission and reconstruction of media streams
KR101734835B1 (en) A method and apparatus for retransmission decision making
US20080259796A1 (en) Method and apparatus for network-adaptive video coding
US20090190652A1 (en) System and method for controlling transmission of moving image data over network
EP2919453A1 (en) Video stream switching
JP4661373B2 (en) Transmission device and transmission program for controlling discard of specific media data
WO2008094092A1 (en) Method and arrangement for video telephony quality assessment
RU2009116472A (en) DYNAMIC MODIFICATION OF VIDEO PROPERTIES
KR102059222B1 (en) Content-dependent video quality model for video streaming services
WO2016018543A1 (en) Automatic and adaptive selection of profiles for adaptive bit rate streaming
KR101080237B1 (en) System and method for modifying a video stream based on a client or network environment, system and method for facilitating a video stream and a computer usable medium
CN106162199B (en) Method and system for video processing with back channel message management
EP1679895A1 (en) Medium signal transmission method, reception method, transmission/reception method, and device
EP1187460A2 (en) Image transmitting method and apparatus and image receiving method and apparatus
CN111093083A (en) Data transmission method and device
US20020184645A1 (en) Measurement of quality of service
CN110099250B (en) Monitoring video quality judgment method and play control device
US6412013B1 (en) System for controlling data output to a network
CN107580781B (en) Video encoder
US20070110168A1 (en) Method for generating high quality, low delay video streaming
WO2013100754A1 (en) A system and method for adaptive media content delivery
TWI566550B (en) Image information transmission method and packet communication system
US8565083B2 (en) Thinning of packet-switched video data
Seeling et al. Video quality evaluation for wireless transmission with robust header compression
EP3348071A1 (en) Method and system for panoramic multimedia streaming

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12751379

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12751379

Country of ref document: EP

Kind code of ref document: A1