US20010016008A1 - Method and apparatus for use in transmitting video information over a communication network - Google Patents
Method and apparatus for use in transmitting video information over a communication network Download PDFInfo
- Publication number
- US20010016008A1 US20010016008A1 US09/169,724 US16972498A US2001016008A1 US 20010016008 A1 US20010016008 A1 US 20010016008A1 US 16972498 A US16972498 A US 16972498A US 2001016008 A1 US2001016008 A1 US 2001016008A1
- Authority
- US
- United States
- Prior art keywords
- recited
- discrete
- video frame
- components
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/649—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding the transform being applied to non rectangular image segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/65—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using error resilience
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2365—Multiplexing of several video streams
- H04N21/23655—Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
Definitions
- This invention relates generally to communication networks and, more particularly, relates to a video encoder for generating information representative of a video frame, a method for transmitting the generated information over a communication channel, and a method for reconstructing the video image from the generated information.
- Video communications over a communication network are currently being carried out using source encoders and channel encoders that were designed independently of each other.
- the tradition of separating the source and channel encoding tasks arises from the theoretical foundation of Shannon's separation principle of source and channel encoding that espouses the notion that this separation is optimal.
- the task of the source encoder is to do the best job it can in compressing the input signal so that the compressed bit rate will match the available bit rate of the communication channel.
- the subject invention may be generally described as a combined application and network level solution for achieving robust video transmission over bandwidth-limited and error-prone communication channels. More specifically, the desired video transmission is achieved through the use of a method for generating information indicative of a video frame, a method for transmitting the generated information over the communication channel, and a method for reconstructing the video information from the generated information.
- the video frame is segmented into a plurality of discrete spatial components and each of the plurality of discrete spatial components is transformed into a plurality of discrete frequency components.
- the video components are evaluated to determine the peak number of bits required for their transmission, bandwidth is requested as a function of this determination, the most important video components are first transmitted in the requested bandwidth and any remaining bandwidth is used to transmit any remaining video components.
- the video components received are checked for transmission errors and, if any such errors are present, video components from a previous transmission are substituted therefor.
- the invention overcomes the noted deficiencies in the prior art by, among other things, reducing the reliance on error correction and error recovery schemes that are traditionally used in transmission over error-prone channels and preventing bandwidth wastage though intelligent bandwidth reservation and utilization while guaranteeing the promised spatial and temporal video resolution at the receiver.
- FIG. 1 illustrates a block diagram of an exemplary communication network in which the subject invention may reside
- FIG. 2 illustrates the exemplary network of FIG. 1 in the form of a cellular communication network
- FIG. 3 illustrates a further block diagram of a communication source, network router, and communication receiver that reside in the communication network of FIGS. 1 and 2;
- FIG. 4 illustrates a flow chart diagram depicting a method for generating information indicative of a video frame performed in accordance with an embodiment of the subject invention
- FIG. 5 illustrates a block diagram of an embodiment of a video encoder that functions to perform the method illustrated in FIG. 4;
- FIG. 6 illustrates two pictures of a video frame that has been spatially segmented in accordance with the method illustrated in FIG. 4;
- FIG. 7 illustrates two pictures of a video frame for the purpose of demonstrating the before and after effects of performing a threshold-based spatial segmentation on the difference image obtained by subtracting two successive video frames as done in the motion estimation and compensation component of the subject invention
- FIG. 8 illustrates the steps and corresponding results when segmenting a video frame in the frequency domain in accordance with the method illustrated in FIG. 4;
- FIG. 9 illustrates a block diagram of a package for use in transmitting the information indicative of the video frame generated in accordance with the method illustrated in FIG. 4;
- FIG. 10 is a flow chart diagram illustrating a method for reserving bandwidth and subsequently for transmitting the information indicative of the video frame over the communication channel;
- FIG. 11 illustrates two graphs for the purpose of demonstrating bandwidth demand versus bandwidth usage before and after using the method illustrated in FIG. 10;
- FIG. 12 illustrates the comparative effects of error propagation on a video frame with no segmentation, with spatial segmentation, and with both spatial and frequency segmentation performed according to the method illustrated in FIG. 4;
- FIG. 13 illustrates a flow chart diagram depicting a method for reconstructing a video frame at the receiver
- FIG. 14 illustrates the movement of video frame segments when performing the frame reconstruction method illustrated in FIG. 13.
- the network system includes a communication source 20 illustrated in the exemplary form of a personal computer.
- the communication source 20 includes a processing unit 21 , a system memory 22 , and a system bus 23 .
- the system bus 23 functions to couple the various system components including the system memory 22 to the processing unit 21 .
- the system bus 23 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
- the system memory 22 includes read only memory (ROM) 24 and random access memory (RAM) 25 .
- a basic input/output system (BIOS) 26 containing the basic routines that help to transfer information between elements within the communication source 20 , such as during start-up, is stored in ROM 24 .
- the communication source 20 may also include a hard disk drive 27 , a magnetic disk drive 28 , or an optical disk drive 30 . It will be appreciated that these devices respectively allow for reading from and writing to a hard disk, reading from or writing to a removable magnetic disk 29 and for reading from or writing to a removable optical disk 31 , such as a CD ROM or other optical media.
- the hard disk drive 27 , magnetic disk drive 28 , and optical disk drive 30 are connected to the system bus 23 by a hard disk drive interface 32 , a magnetic disk drive interface 33 , and an optical disk drive interface 34 , respectively.
- the drives and their associated computer-readable media provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for the communication source 20 . It will be appreciated by those skilled in the art that other types of computer readable media that can store data may also be used. Examples of such media devices include, but are not limited to, magnetic cassettes, flash memory cards, digital videodisks, Bernoulli cartridges, random access memories, and read only memories.
- a number of program modules may be stored in one or more of the memory devices and typically include an operating system 35 , one or more applications programs 36 , other program modules 37 , and program data 38 .
- a user may enter commands and information into the communication source 20 through input devices such as a keyboard 40 and pointing device 42 .
- Other input devices may include a microphone, joystick, game pad, satellite dish, scanner, or the like.
- These and other input devices are often connected to the processing unit 21 through a serial port interface 46 that is coupled to the system bus, but may be connected by other interfaces, such as a parallel port, game port or a universal serial bus (USB).
- a monitor 47 or other type of display device is also connected to the system bus 23 via an interface, such as a video adapter 48 .
- the communication source 20 may also include other peripheral output devices, not shown, such as speakers and printers.
- the communication source 20 operates in a networked environment using logical connections to one or more remote communication receivers 49 , also illustrated in the exemplary form of a personal computer.
- the connection is typically made through a further processing device 100 that is responsible for network routing.
- the remote communication receiver 49 will include many or all of the elements described above relative to the communication source 20 including the memory storage devices and monitor 47 .
- program modules depicted relative to the communication source 20 may be stored in the memory storage devices of the remote communication receiver 49 .
- the network environment comprises a wireless, cellular communication network such as depicted in FIG. 2.
- a wireless, cellular communication network such as depicted in FIG. 2.
- LAN local area network
- WAN wide area network
- FIG. 2 a cellular communication network having architecture that is similar to the architecture that is found in a Global System for Mobility (GSM) network is illustrated.
- GSM Global System for Mobility
- mobile terminals MT communicate over a radio channel with a network router in the form of a base transceiver station BTS that is located within the same cell.
- the base transceiver stations BTSs of neighboring cells communicate over high speed wired or wireless channels, with a base station controller BSC that serves as their manager.
- the base station controllers BSCs in a geographic region communicate over high speed wired or wireless channels with a mobile switching center MSC that serves as their manager.
- the mobile switching centers MSCs are connected to a public network that may include public switched telephone networks PSTNs, integrated services device networks ISDNs, or asynchronous transport mode ATM networks.
- the mobile terminals MT within the network are envisioned to be communication sources 20 of the following types: 1) communication sources 20 VO performing voice packet transmissions; 2) communication sources 20 DA performing data packet transmissions; and 3) communication sources 20 VI performing video packet transmission.
- remote communication receivers 49 linked to the public network as a destination for the transmissions originating from the mobile terminals MT are remote communication receivers 49 of the following type: 1) remote communication receivers 49 VO receiving voice transmissions; 2) remote communication receivers 49 DA receiving data transmissions; and 3) remote communication receivers 49 VI receiving video transmissions. It will be appreciated by those skilled in the art that the remote communication receivers 49 may themselves be mobile.
- the communication network provides for the efficient transmission of video communications from the communication sources 20 VI to one or more associated remote communication receivers 49 VI for subsequent display to remote users.
- the communication sources 20 VI are each provided with a video camera 152 that functions to capture video images as a series of digital video frames.
- the video camera 152 is connected to a video encoder 154 via the system bus 23 .
- the video encoder 154 functions to generate information that is representative of the video frame which is subsequently supplied to a transmitter 155 for broadcasting over the communication channel to the network router 100 , e.g., base station BS.
- the remote communication receivers 49 VI each include a video decoder 156 that functions to recreate the video frame from the information received over the communication network.
- the decoded video frame may then be forwarded to an associated monitor 47 for viewing through the use of an appropriate video adapter 157 .
- an appropriate video adapter 157 For a description of the communications protocol that is preferably used in broadcasting the video information over the communication network the reader is referred to commonly owned, co-pending U.S. patent application Ser. No. ______ (Attorney Docket No. 84428) entitled “________” that is incorporated herein by reference in its entirety.
- the video encoder 154 functions to generate information representative of a video frame for use in providing visual communications over a communication network.
- the video encoding algorithm utilized within the encoder 154 generates the information by decomposing individual frames of a video sequence into a plurality of discrete, generally homogeneous video regions by performing segmentation in both the spatial and frequency domains.
- the encoder includes a spatial segmenter 160 and a frequency segmenter 162 .
- the video encoder 154 includes a frame classifier 158 and a compressor 164 comprised of a quantizer 166 and run length encoder 168 .
- the components of the encoder 154 are similar to those found within an encoder that currently utilizes the MPEG-1 coding standard.
- the frame classifier 158 is provided to classify the image frames, or segments thereof, as either an inter-frame (when it is to be coded with respect to other frames or segments) or an intra-frame (when it is to be coded independently). This classification is typically based on the order and motion activity of the video frame. For example, the first video frame or anchor frame of the video sequence or frames containing drastic scene changes are intra-coded while the remaining frame are inter-coded. Additionally, every nth video frame may be encoded as an intra-frame, where n is some predetermined number, in order to minimize inter-frame error propagation that may arise from time to time.
- the spatial segmenter 160 is provided to create a spatial segmentation map for the video frame.
- a segmentation map is derived only when there is significant motion activity within the video frame sequence.
- a predetermined threshold a design variable
- the previous most recently derived segmentation map is reused.
- a new segmentation map is derived and forwarded to the remote computer 40 , typically with the encoded frame information in a transmission packet.
- the communication receiver 49 VI requires the segmentation map in order to recreate the video images for display.
- An advantage to biasing the segmentation decision on motion activity is found in the significant reduction in overhead owing to the fact that a new segmentation map does not have to be forwarded to the communication receiver 49 VI with every frame. Accordingly, the encoder and the decoder are designed to continue to use the same segmentation map until some significant scene change appears in the incoming frame at which time a new segmentation map is generated from the statistics of that frame.
- the spatial segmenter 160 To dynamically generate the segmentation map for those image frames that are independently coded, i.e., intra-frames, the spatial segmenter 160 first preferably partitions the digital image frame into blocks of size 16 ⁇ 16 pixels. For each block the variance of pixel intensity is then calculated. Adjacent blocks of similar variances that are within a predetermined threshold are then merged. The merged blocks form the spatial segmentation map examples of which are illustrated in FIG. 6.
- T is a threshold value for controlling the decomposition of the image frame.
- T log ⁇ ( ⁇ max 2 / ⁇ min 2 ) / N + ⁇ ( 3 )
- ⁇ is the mean value of the pixel intensity of all of the blocks in the video frame.
- the video encoder 154 further includes a motion estimator/compensator 170 that functions to determine motion differences between two sequential image frames.
- the estimator/compensator 170 segments the image frame into 16 ⁇ 16 blocks and utilizes a matching algorithm to obtain motion vectors.
- the matching algorithm is employed to see if any of the blocks within the current image frame is sufficiently similar to a block within the previous image frame.
- the absolute error for each pixel as well as for each block be calculated to determine if the blocks are similar within a predetermined threshold. Exemplary results achieved as a result of performing these steps may be seen in FIG. 7.
- the affected blocks are preferably classified as intra-frame blocks and a new segmentation map for these blocks is generated using the spatial segmenter 160 and the related methodology described previously.
- the motion difference between the images is typically deemed to be sufficient if: 1) any blocks have moved greater than a predetermined distance within the images; or 2) any blocks are determined to have no general match with any block in the previous image frame. The latter condition is utilized for the reason that it signifies that a new action object has likely entered the video sequence.
- the spatial segmenter 160 uses a predefined segmentation map to merge the blocks without regard to the intra-frame or inter-frame considerations set forth above.
- This predefined segmentation map is preferably arranged such that segments overlaying regions of less visual importance within the video frame sequence, such as background, contain a large number of merged blocks while segments overlaying areas of more visual importance, such as action objects, contain fewer merged blocks. Accordingly, it will be appreciated that the type of video to be transmitted should dictate the form of the predetermined segmentation map that is utilized. As will be appreciated, this method for segmenting the video frame has the advantage of requiring a lesser amount of computational energy when compared to the dynamic segmentation method described previously.
- the use of a static segmentation map has the advantage of providing a means for conserving power within the communication source 20 VI.
- this method also has the advantage of providing a means to conserve bandwidth as the segmentation map needs to be transmitted to the communication receivers 49 VI only one time owing to its static nature.
- the spatial segments be provided with a transmission priority.
- This transmission priority is preferably established as a function of the information contained within the various segments. For example, larger segments that would typically contain background information would be given a lower transmission priority while smaller segments that would typically contain action information would be given a higher transmission priority.
- each of the spatial segments is further decomposed into a plurality of discrete frequency components as illustrated in FIG. 8.
- the frequency segmenter 162 preferably utilizes a known wavelet-based technique that functions to represent an image signal at different levels of resolution.
- wavelets when used in sub-band coding, emphasize both spatial localization and frequency localization during the band splitting procedure. In this manner, wavelet transformation provides for multi-resolution decomposition of the image frame.
- the image frame is thus represented by compact, non-redundant and exact representations at different scales.
- the frequency segmenter 162 is preferably embodied in a two-tap Harr filter that functions to decompose the luminance (Y) components of each segment into four bands, namely, one DC-subband (low-low band) and three AC-subbands (low-high, high-low, and high-high bands).
- the Haar filter was choosen because of its low computational cost and capability to decompose the arbitrarily shaped regions generated by the spatial segmenter 160 without causing any undesirable boundary effects.
- the frequency segmenter 162 also preferably applies an 8 ⁇ 8 Discrete Cosine Transform (DCT) in a moving window to each of the subbands.
- the DCT functions to further decorrelate the spectral components and causes energy compaction in coefficients.
- the DCT transformation step can be skipped if a more elaborate wavelet transformation is used such as a Daubechies 6-tap filter.
- the compressor applying the Harr transformation followed by the DCT to the image frame segments de-correlates the signals of the original image and concentrates the image information into a relatively small number of coefficients. Since most images consist primarily of smooth or low frequency components, it is often true that the DC-subband contains most of the energy of the image. As the DC-subband provides a smoothed and downsampled version of the original image while the AC-subbands provide information of edges and textures, the DC-subbands of each of the spatial segments should be given a still higher priority for transmission than should be given to the AC-subbands.
- a compressor 164 in the form of a quantizer 166 and standard run length encoder 168 is provided.
- the quantizer 166 maximizes this compression by eliminating all but a few coefficients which still allows a good quality image to be reconstructed. This is possible owing to the above-noted observation that most of the image may be found within the DC-subband. More specifically, the quantizer 166 attempts to maximize compression by eliminating those coefficients below a user specified threshold and by applying a different quantizer to each segment as a function of its priority.
- the coefficients of the DC-subband are further quantized with a lower step size than are the AC-subbands. In this manner, the more important information of the image is preserved.
- all coefficients of the lowest frequency sub-bands are preferred to be protected for all video regions. For inter-frame, it is preferred that all motion information be protected.
- the package that is used to transmit the information indicative of the video image is illustrated.
- the package includes a picture header, a segment map, and a segment data layer.
- the picture header is similar to the picture header utilized in the H.263 video coding standard and contains all information needed to decode one video frame such as a 20-bit start code (PSC), a 5-bit temporal reference (TR) or frame number, and type information (PTYPE).
- the header also contains a segment identifier (SI) and a bit-field representing the total number of segments (NS) within the picture.
- SI segment identifier
- NS bit-field representing the total number of segments
- the spatial segmentation methods described above will have to be adapted to ensure that the number of discrete spatial segments that the video frame is divided into does not exceed sixteen.
- the first bit is preferably utilized to inform the communication receiver 49 VI that the segment map contained within the package is new.
- the DC subbands and the AC subbands for each video frame are transmitted in different slices.
- the slices are arranged such that the DC subbands first appear in their spatial segmentation transmission priority order followed by the corresponding AC subbands similarly ordered.
- This structure allows the system to apply unequal error protection (UEP) and transmit the video based on the transmission priority of the segments and the subbands.
- UEP unequal error protection
- the segmentation approach described above facilitates guaranteed quality of service during the transmission of the video information by generally reserving bandwidth as a function of the transmission priority of the subbands of the video segments. Specifically, the number of bits required for transmitting the DC subbands of all the video segments within a video frame is determined. This procedure is repeated for all the video frames within the video sequence that is to be transmitted. Then the maximum (peak) of all these numbers is multiplied by the desired target video frame display rate and the result determines the amount of bandwidth that is to be statically reserved at the connection establishment time. If the reservation is successful, it ensures that there will always be bandwidth available for transmitting at least the DC subbands of all the video frames in the video sequence.
- FIG. 12 illustrates error propagation with and without spatial segmentation as a result of a video bit stream that has been corrupted with errors not caught by an error detector.
- a dynamic reduction in non-reserved bandwidth whereby some of the segments fail to reach the decoder in a timely manner will result in the receiver not being able to reconstruct the complete image.
- the receiver employs a frame reconstructing algorithm, illustrated in FIG. 12, that utilizes a combination of the current and previously transmitted video segments that were correctly received.
- the video segments stored in the receiver's region store may not belong to the same video frame. Accordingly, when used to reconstruct the video frame in this manner, the temporal differences between the current video segment and the previously correctly received corresponding video segment dictates how good or bad the final picture looks. When this difference is large, visual quality is impaired by the tearing effect. However, when the previous good video segment is from a preceding sequential frame, the quality tends to be visually acceptable. Therefore, to reduce the tearing effect, the communication receiver 49 VI may demand from the communication source 20 VI the immediate transmission of the video segment(s) that it had to substitute in order to reconstruct the current frame.
- the communication source 20 VI may then switch the transmission priority of the requested segment(s) to ensure that the requested video segment(s) definitely reach the communication receiver 20 VI.
- the difference between the current video segment and the ones stored in the receiver's region store is never too much and the tearing effect is mitigated.
- the invention described herein provides a solution to the problem of guaranteeing a temporal and spatial video frame resolution at the receiver for time-bounded video transmitted over a communication channel.
- this is achieved by using a channel sensitive video encoder, determining the appropriate amount of channel bandwidth to reserve, utilizing the reserved bandwidth without wastage, and developing a error-concealment mechanism for reconstructing the video frame from incomplete data or data received in error, without the need for re-transmissions.
Abstract
Description
- This invention relates generally to communication networks and, more particularly, relates to a video encoder for generating information representative of a video frame, a method for transmitting the generated information over a communication channel, and a method for reconstructing the video image from the generated information.
- Video communications over a communication network are currently being carried out using source encoders and channel encoders that were designed independently of each other. The tradition of separating the source and channel encoding tasks arises from the theoretical foundation of Shannon's separation principle of source and channel encoding that espouses the notion that this separation is optimal. According to this principle, the task of the source encoder is to do the best job it can in compressing the input signal so that the compressed bit rate will match the available bit rate of the communication channel.
- Shannon's separation principal does, however, fail to address the error characteristics of the communication channel on which the video information will traverse. Importantly, developing an encoder without regard to these error characteristics has lead to the result that the image signal is rendered effectively useless if even a few bits are corrupted during transmission. This problem becomes especially pronounced in a wireless communication network where large variations and unpredictability in the error characteristics abound. Unfortunately, currently utilized video coding standards including H.261, MPEG-1, MPEG-2 and H.263, which were developed without regard to the error characteristics of the communication channel, fail to provide the error resiliency needed to minimize this problem.
- Furthermore, currently utilized error protection and error recovery schemes that are employed to alleviate the noted error-induction problem have the undesirable side effect of wasting bandwidth within the communication channel. For example, a commensurate amount of bandwidth must be allocated to video transmissions at the expense of other transmissions when additional error checking bits are added to the video bit-stream. Furthermore, using an automatic repeat request (ARQ) procedure that functions to initiate the re-transmission of those transmissions deemed to be in error, while providing error recovery, adds both delay and bandwidth overhead to the transmissions. Delay for the case of interactive real-time video applications causes unacceptable application performance, while bandwidth overhead reduces the efficiency of the overall system.
- From the foregoing, it is evident that a need exists for software and hardware mechanisms that ensure timely and robust delivery of video data over bandwidth-limited, error-prone communication channels.
- In accordance with this need, the subject invention may be generally described as a combined application and network level solution for achieving robust video transmission over bandwidth-limited and error-prone communication channels. More specifically, the desired video transmission is achieved through the use of a method for generating information indicative of a video frame, a method for transmitting the generated information over the communication channel, and a method for reconstructing the video information from the generated information.
- For generating the information indicative of a video frame, the video frame is segmented into a plurality of discrete spatial components and each of the plurality of discrete spatial components is transformed into a plurality of discrete frequency components. For transmitting the generated information over the communication channel, the video components are evaluated to determine the peak number of bits required for their transmission, bandwidth is requested as a function of this determination, the most important video components are first transmitted in the requested bandwidth and any remaining bandwidth is used to transmit any remaining video components. For reconstructing the video information from the generated information the video components received are checked for transmission errors and, if any such errors are present, video components from a previous transmission are substituted therefor. In this manner, the invention overcomes the noted deficiencies in the prior art by, among other things, reducing the reliance on error correction and error recovery schemes that are traditionally used in transmission over error-prone channels and preventing bandwidth wastage though intelligent bandwidth reservation and utilization while guaranteeing the promised spatial and temporal video resolution at the receiver.
- A better understanding of the objects, advantages, features, properties and relationships of the invention will be obtained from the following detailed description and accompanying drawings which set forth an illustrative embodiment which is indicative of the various ways in which the principles of the invention may be employed.
- For a better understanding of the invention, reference may be had to a preferred embodiment shown in the accompanying drawings in which:
- FIG. 1 illustrates a block diagram of an exemplary communication network in which the subject invention may reside;
- FIG. 2 illustrates the exemplary network of FIG. 1 in the form of a cellular communication network;
- FIG. 3 illustrates a further block diagram of a communication source, network router, and communication receiver that reside in the communication network of FIGS. 1 and 2;
- FIG. 4 illustrates a flow chart diagram depicting a method for generating information indicative of a video frame performed in accordance with an embodiment of the subject invention;
- FIG. 5 illustrates a block diagram of an embodiment of a video encoder that functions to perform the method illustrated in FIG. 4;
- FIG. 6 illustrates two pictures of a video frame that has been spatially segmented in accordance with the method illustrated in FIG. 4;
- FIG. 7 illustrates two pictures of a video frame for the purpose of demonstrating the before and after effects of performing a threshold-based spatial segmentation on the difference image obtained by subtracting two successive video frames as done in the motion estimation and compensation component of the subject invention;
- FIG. 8 illustrates the steps and corresponding results when segmenting a video frame in the frequency domain in accordance with the method illustrated in FIG. 4;
- FIG. 9 illustrates a block diagram of a package for use in transmitting the information indicative of the video frame generated in accordance with the method illustrated in FIG. 4;
- FIG. 10 is a flow chart diagram illustrating a method for reserving bandwidth and subsequently for transmitting the information indicative of the video frame over the communication channel;
- FIG. 11 illustrates two graphs for the purpose of demonstrating bandwidth demand versus bandwidth usage before and after using the method illustrated in FIG. 10;
- FIG. 12 illustrates the comparative effects of error propagation on a video frame with no segmentation, with spatial segmentation, and with both spatial and frequency segmentation performed according to the method illustrated in FIG. 4;
- FIG. 13 illustrates a flow chart diagram depicting a method for reconstructing a video frame at the receiver; and
- FIG. 14 illustrates the movement of video frame segments when performing the frame reconstruction method illustrated in FIG. 13.
- Turning to the drawings, wherein like reference numerals refer to like elements, the invention is illustrated as being implemented in a suitable network environment. Although not required, the invention will be described in the general context of computer-executable instructions, such as program modules, being executed by a processing device such as a personal computer, mainframe computer, or the like. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the invention may be practiced with other processing devices such as consumer electronic devices having one or more processors such as, for example, mobile telephones. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communication network and where program modules are located in both local and remote memory storage devices.
- With reference to FIG. 1, an exemplary network system in which the invention may reside is illustrated. The network system includes a
communication source 20 illustrated in the exemplary form of a personal computer. Thecommunication source 20 includes aprocessing unit 21, asystem memory 22, and asystem bus 23. Thesystem bus 23 functions to couple the various system components including thesystem memory 22 to theprocessing unit 21. Thesystem bus 23 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. Thesystem memory 22 includes read only memory (ROM) 24 and random access memory (RAM) 25. A basic input/output system (BIOS) 26, containing the basic routines that help to transfer information between elements within thecommunication source 20, such as during start-up, is stored inROM 24. Thecommunication source 20 may also include ahard disk drive 27, amagnetic disk drive 28, or anoptical disk drive 30. It will be appreciated that these devices respectively allow for reading from and writing to a hard disk, reading from or writing to a removablemagnetic disk 29 and for reading from or writing to a removableoptical disk 31, such as a CD ROM or other optical media. - When incorporated into the
communication source 20, thehard disk drive 27,magnetic disk drive 28, andoptical disk drive 30 are connected to thesystem bus 23 by a harddisk drive interface 32, a magneticdisk drive interface 33, and an optical disk drive interface 34, respectively. The drives and their associated computer-readable media provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for thecommunication source 20. It will be appreciated by those skilled in the art that other types of computer readable media that can store data may also be used. Examples of such media devices include, but are not limited to, magnetic cassettes, flash memory cards, digital videodisks, Bernoulli cartridges, random access memories, and read only memories. - A number of program modules may be stored in one or more of the memory devices and typically include an
operating system 35, one ormore applications programs 36,other program modules 37, andprogram data 38. A user may enter commands and information into thecommunication source 20 through input devices such as akeyboard 40 and pointing device 42. Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to theprocessing unit 21 through aserial port interface 46 that is coupled to the system bus, but may be connected by other interfaces, such as a parallel port, game port or a universal serial bus (USB). Amonitor 47 or other type of display device is also connected to thesystem bus 23 via an interface, such as avideo adapter 48. In addition to themonitor 47, thecommunication source 20 may also include other peripheral output devices, not shown, such as speakers and printers. - The
communication source 20 operates in a networked environment using logical connections to one or moreremote communication receivers 49, also illustrated in the exemplary form of a personal computer. The connection is typically made through afurther processing device 100 that is responsible for network routing. In the illustrated embodiment, theremote communication receiver 49 will include many or all of the elements described above relative to thecommunication source 20 including the memory storage devices and monitor 47. Furthermore, within such a networked environment, it will be appreciated that program modules depicted relative to thecommunication source 20, or portions thereof, may be stored in the memory storage devices of theremote communication receiver 49. - The description that follows will refer to acts and symbolic representations of operations that are performed by the
processing devices processing devices - In a preferred embodiment, the network environment comprises a wireless, cellular communication network such as depicted in FIG. 2. Nevertheless, while the invention will be described hereinafter in that context, those of skill in the art will appreciate how to adapt the features of the subject invention for us in other communication networks such as a local area network (LAN) or a wide area network (WAN). Accordingly, the description that follows is not intended to be limiting.
- Turning to FIG. 2, a cellular communication network having architecture that is similar to the architecture that is found in a Global System for Mobility (GSM) network is illustrated. As will be well understood by those of skill in the art, mobile terminals MT communicate over a radio channel with a network router in the form of a base transceiver station BTS that is located within the same cell. The base transceiver stations BTSs of neighboring cells communicate over high speed wired or wireless channels, with a base station controller BSC that serves as their manager. In turn, the base station controllers BSCs in a geographic region communicate over high speed wired or wireless channels with a mobile switching center MSC that serves as their manager. The mobile switching centers MSCs are connected to a public network that may include public switched telephone networks PSTNs, integrated services device networks ISDNs, or asynchronous transport mode ATM networks.
- More specifically, the mobile terminals MT within the network are envisioned to be
communication sources 20 of the following types: 1) communication sources 20VO performing voice packet transmissions; 2) communication sources 20DA performing data packet transmissions; and 3) communication sources 20VI performing video packet transmission. Meanwhile, linked to the public network as a destination for the transmissions originating from the mobile terminals MT areremote communication receivers 49 of the following type: 1) remote communication receivers 49VO receiving voice transmissions; 2) remote communication receivers 49DA receiving data transmissions; and 3) remote communication receivers 49VI receiving video transmissions. It will be appreciated by those skilled in the art that theremote communication receivers 49 may themselves be mobile. - In accordance with one important aspect of the invention, the communication network provides for the efficient transmission of video communications from the communication sources20VI to one or more associated remote communication receivers 49VI for subsequent display to remote users. For this purpose, as illustrated in FIG. 3, the communication sources 20VI are each provided with a
video camera 152 that functions to capture video images as a series of digital video frames. Within the communication sources 20VI, thevideo camera 152 is connected to avideo encoder 154 via thesystem bus 23. Thevideo encoder 154 functions to generate information that is representative of the video frame which is subsequently supplied to atransmitter 155 for broadcasting over the communication channel to thenetwork router 100, e.g., base station BS. Meanwhile, the remote communication receivers 49VI each include avideo decoder 156 that functions to recreate the video frame from the information received over the communication network. The decoded video frame may then be forwarded to an associatedmonitor 47 for viewing through the use of anappropriate video adapter 157. For a description of the communications protocol that is preferably used in broadcasting the video information over the communication network the reader is referred to commonly owned, co-pending U.S. patent application Ser. No. ______ (Attorney Docket No. 84428) entitled “______” that is incorporated herein by reference in its entirety. - Turning to FIGS. 4 and 5, the
video encoder 154 functions to generate information representative of a video frame for use in providing visual communications over a communication network. Generally, the video encoding algorithm utilized within theencoder 154 generates the information by decomposing individual frames of a video sequence into a plurality of discrete, generally homogeneous video regions by performing segmentation in both the spatial and frequency domains. For this purpose, the encoder includes aspatial segmenter 160 and afrequency segmenter 162. Additionally, thevideo encoder 154 includes aframe classifier 158 and a compressor 164 comprised of aquantizer 166 and runlength encoder 168. With the exception of thespatial segmenter 160 and thefrequency segmenter 162, the components of theencoder 154 are similar to those found within an encoder that currently utilizes the MPEG-1 coding standard. - The
frame classifier 158 is provided to classify the image frames, or segments thereof, as either an inter-frame (when it is to be coded with respect to other frames or segments) or an intra-frame (when it is to be coded independently). This classification is typically based on the order and motion activity of the video frame. For example, the first video frame or anchor frame of the video sequence or frames containing drastic scene changes are intra-coded while the remaining frame are inter-coded. Additionally, every nth video frame may be encoded as an intra-frame, where n is some predetermined number, in order to minimize inter-frame error propagation that may arise from time to time. - The
spatial segmenter 160 is provided to create a spatial segmentation map for the video frame. Generally, for intra-frames, the entire image is segmented and a segmentation map is derived, for inter-frames, segmenting is employed and a segmentation map is derived only when there is significant motion activity within the video frame sequence. When the motion activity is below a predetermined threshold (a design variable), the previous most recently derived segmentation map is reused. When the motion activity is above the threshold, a new segmentation map is derived and forwarded to theremote computer 40, typically with the encoded frame information in a transmission packet. As will be better understood from the description that follows, the communication receiver 49VI requires the segmentation map in order to recreate the video images for display. An advantage to biasing the segmentation decision on motion activity is found in the significant reduction in overhead owing to the fact that a new segmentation map does not have to be forwarded to the communication receiver 49VI with every frame. Accordingly, the encoder and the decoder are designed to continue to use the same segmentation map until some significant scene change appears in the incoming frame at which time a new segmentation map is generated from the statistics of that frame. - To dynamically generate the segmentation map for those image frames that are independently coded, i.e., intra-frames, the
spatial segmenter 160 first preferably partitions the digital image frame into blocks ofsize 16×16 pixels. For each block the variance of pixel intensity is then calculated. Adjacent blocks of similar variances that are within a predetermined threshold are then merged. The merged blocks form the spatial segmentation map examples of which are illustrated in FIG. 6. -
-
-
- wherein η is the mean value of the pixel intensity of all of the blocks in the video frame.
- At this time, it is noted that there exists a trade-off between complexity and coding performance when choosing the block size. While using smaller sized blocks achieves relatively better spatial decomposition, the overhead incurred for sending the segmentation map to the decoder is relatively higher. For this reason, the block size of 16×16 was selected as it was seen to yield relatively good adaptability with acceptable levels of overhead for each intra-frame image.
- As described, for inter-frames, segmentation is employed only when there is significant motion activity within the video frame sequence. For this purpose, the
video encoder 154 further includes a motion estimator/compensator 170 that functions to determine motion differences between two sequential image frames. To perform this task, the estimator/compensator 170 segments the image frame into 16×16 blocks and utilizes a matching algorithm to obtain motion vectors. The matching algorithm is employed to see if any of the blocks within the current image frame is sufficiently similar to a block within the previous image frame. During this search, it is preferred that the absolute error for each pixel as well as for each block be calculated to determine if the blocks are similar within a predetermined threshold. Exemplary results achieved as a result of performing these steps may be seen in FIG. 7. - If the motion difference between the two images is sufficient, the affected blocks are preferably classified as intra-frame blocks and a new segmentation map for these blocks is generated using the
spatial segmenter 160 and the related methodology described previously. The motion difference between the images is typically deemed to be sufficient if: 1) any blocks have moved greater than a predetermined distance within the images; or 2) any blocks are determined to have no general match with any block in the previous image frame. The latter condition is utilized for the reason that it signifies that a new action object has likely entered the video sequence. In a further embodiment, it is preferred to reclassify a video frame as an intra-frame when the motion difference, i.e., the number of affected blocks, is above some maximum value. In this case, a completely new segmentation map for the entire frame would be generated using thespatial segmenter 160. - In yet another embodiment of the invention, the
spatial segmenter 160 uses a predefined segmentation map to merge the blocks without regard to the intra-frame or inter-frame considerations set forth above. This predefined segmentation map is preferably arranged such that segments overlaying regions of less visual importance within the video frame sequence, such as background, contain a large number of merged blocks while segments overlaying areas of more visual importance, such as action objects, contain fewer merged blocks. Accordingly, it will be appreciated that the type of video to be transmitted should dictate the form of the predetermined segmentation map that is utilized. As will be appreciated, this method for segmenting the video frame has the advantage of requiring a lesser amount of computational energy when compared to the dynamic segmentation method described previously. As a result, the use of a static segmentation map has the advantage of providing a means for conserving power within the communication source 20VI. In addition, this method also has the advantage of providing a means to conserve bandwidth as the segmentation map needs to be transmitted to the communication receivers 49VI only one time owing to its static nature. - When establishing a segmentation map, it is also preferred that the spatial segments be provided with a transmission priority. This transmission priority is preferably established as a function of the information contained within the various segments. For example, larger segments that would typically contain background information would be given a lower transmission priority while smaller segments that would typically contain action information would be given a higher transmission priority.
- Once the image frame has been spatially segmented, each of the spatial segments is further decomposed into a plurality of discrete frequency components as illustrated in FIG. 8. For this purpose, the
frequency segmenter 162 preferably utilizes a known wavelet-based technique that functions to represent an image signal at different levels of resolution. Generally, wavelets, when used in sub-band coding, emphasize both spatial localization and frequency localization during the band splitting procedure. In this manner, wavelet transformation provides for multi-resolution decomposition of the image frame. The image frame is thus represented by compact, non-redundant and exact representations at different scales. - More specifically, the
frequency segmenter 162 is preferably embodied in a two-tap Harr filter that functions to decompose the luminance (Y) components of each segment into four bands, namely, one DC-subband (low-low band) and three AC-subbands (low-high, high-low, and high-high bands). The Haar filter was choosen because of its low computational cost and capability to decompose the arbitrarily shaped regions generated by thespatial segmenter 160 without causing any undesirable boundary effects. Following the wavelet decomposition, thefrequency segmenter 162 also preferably applies an 8×8 Discrete Cosine Transform (DCT) in a moving window to each of the subbands. The DCT functions to further decorrelate the spectral components and causes energy compaction in coefficients. However, the DCT transformation step can be skipped if a more elaborate wavelet transformation is used such as a Daubechies 6-tap filter. - As described, the compressor applying the Harr transformation followed by the DCT to the image frame segments de-correlates the signals of the original image and concentrates the image information into a relatively small number of coefficients. Since most images consist primarily of smooth or low frequency components, it is often true that the DC-subband contains most of the energy of the image. As the DC-subband provides a smoothed and downsampled version of the original image while the AC-subbands provide information of edges and textures, the DC-subbands of each of the spatial segments should be given a still higher priority for transmission than should be given to the AC-subbands.
- For the purpose of conserving communication channel bandwidth, information that comprises each of the subbands for each of the segments is preferably further compressed before it is transmitted over the communications network. Accordingly, a compressor164 in the form of a
quantizer 166 and standardrun length encoder 168 is provided. Preferably, thequantizer 166 maximizes this compression by eliminating all but a few coefficients which still allows a good quality image to be reconstructed. This is possible owing to the above-noted observation that most of the image may be found within the DC-subband. More specifically, thequantizer 166 attempts to maximize compression by eliminating those coefficients below a user specified threshold and by applying a different quantizer to each segment as a function of its priority. Within these regions, the coefficients of the DC-subband are further quantized with a lower step size than are the AC-subbands. In this manner, the more important information of the image is preserved. In particular, in order to preserve the visual quality of an intra-frame, all coefficients of the lowest frequency sub-bands are preferred to be protected for all video regions. For inter-frame, it is preferred that all motion information be protected. - Turning to FIG. 9, the package that is used to transmit the information indicative of the video image is illustrated. Generally, the package includes a picture header, a segment map, and a segment data layer. The picture header is similar to the picture header utilized in the H.263 video coding standard and contains all information needed to decode one video frame such as a 20-bit start code (PSC), a 5-bit temporal reference (TR) or frame number, and type information (PTYPE). In accordance with this invention, the header also contains a segment identifier (SI) and a bit-field representing the total number of segments (NS) within the picture. As a means to further conserve bandwidth, the segment identifier (SI) was selected to be a four-bit field. Accordingly, as each segment within the segmentation map would be represented by a unique 4 bit segment index, the spatial segmentation methods described above will have to be adapted to ensure that the number of discrete spatial segments that the video frame is divided into does not exceed sixteen. Within the segment map, the first bit is preferably utilized to inform the communication receiver49VI that the segment map contained within the package is new.
- Within the segment data layer, the DC subbands and the AC subbands for each video frame are transmitted in different slices. Preferably the slices are arranged such that the DC subbands first appear in their spatial segmentation transmission priority order followed by the corresponding AC subbands similarly ordered. This structure allows the system to apply unequal error protection (UEP) and transmit the video based on the transmission priority of the segments and the subbands.
- Specifically, the segmentation approach described above facilitates guaranteed quality of service during the transmission of the video information by generally reserving bandwidth as a function of the transmission priority of the subbands of the video segments. Specifically, the number of bits required for transmitting the DC subbands of all the video segments within a video frame is determined. This procedure is repeated for all the video frames within the video sequence that is to be transmitted. Then the maximum (peak) of all these numbers is multiplied by the desired target video frame display rate and the result determines the amount of bandwidth that is to be statically reserved at the connection establishment time. If the reservation is successful, it ensures that there will always be bandwidth available for transmitting at least the DC subbands of all the video frames in the video sequence. This guarantee when combined with the video frame reconstruction procedure, to be described next, guarantees that a minimum video frame temporal resolution can be supported at the receiver. This procedure is illustrated in FIG. 10. As it is likely that most of the time the compressor164 will produce bits far below this peak number, as illustrated in FIG. 11, the notion of statistical multiplexing within a frame is introduced to avoid under-utilizing the allocated bandwidth. With statistical multiplexing, any bandwidth leftover after the DC subbands have been transmitted is used for transmitting the remaining AC subbands. Also, packets whose retransmission has been requested by a communication receiver 49VI can be sent using the left over bandwidth that was reserved. Furthermore, additional bandwidth can be requested dynamically for the purpose of transmitting any packets that remain after all of the statically allocated bandwidth has been used. In this manner, optimum bandwidth utilization may be achieved.
- At the receiver, transmission errors due to channel imperfections can cause corruption in some of the transmitted video segments rendering them un-decodable. For example, FIG. 12 illustrates error propagation with and without spatial segmentation as a result of a video bit stream that has been corrupted with errors not caught by an error detector. A dynamic reduction in non-reserved bandwidth whereby some of the segments fail to reach the decoder in a timely manner will result in the receiver not being able to reconstruct the complete image. Accordingly, to minimize any errors and be able to create a complete image from incomplete data that may arise if either of these events occur, the receiver employs a frame reconstructing algorithm, illustrated in FIG. 12, that utilizes a combination of the current and previously transmitted video segments that were correctly received.
-
- In the case when some of the Sij are incorrectly received, Si is formed using the last corresponding jth video segment that was received correctly. This methodology of image reconstruction at the receiver is generally illustrated in FIG. 14 for a simple horizontal partitioning scheme.
- As will be understood from the foregoing discussion, the video segments stored in the receiver's region store may not belong to the same video frame. Accordingly, when used to reconstruct the video frame in this manner, the temporal differences between the current video segment and the previously correctly received corresponding video segment dictates how good or bad the final picture looks. When this difference is large, visual quality is impaired by the tearing effect. However, when the previous good video segment is from a preceding sequential frame, the quality tends to be visually acceptable. Therefore, to reduce the tearing effect, the communication receiver49VI may demand from the communication source 20VI the immediate transmission of the video segment(s) that it had to substitute in order to reconstruct the current frame. The communication source 20VI may then switch the transmission priority of the requested segment(s) to ensure that the requested video segment(s) definitely reach the communication receiver 20VI. With such a scheme, the difference between the current video segment and the ones stored in the receiver's region store is never too much and the tearing effect is mitigated.
- From the foregoing descriptions, it will be appreciated that the invention described herein provides a solution to the problem of guaranteeing a temporal and spatial video frame resolution at the receiver for time-bounded video transmitted over a communication channel. In particular, this is achieved by using a channel sensitive video encoder, determining the appropriate amount of channel bandwidth to reserve, utilizing the reserved bandwidth without wastage, and developing a error-concealment mechanism for reconstructing the video frame from incomplete data or data received in error, without the need for re-transmissions.
- In view of the many possible embodiments to which the principles of this invention may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures is meant to be illustrative only and should not be taken as limiting the scope of invention. For example, those skilled in the art will recognize that the elements of the illustrated embodiments described in software and algorithmic form may be implemented in hardware and vice versa or that the illustrated embodiments can be modified in arrangement and detail without departing from the spirit of the invention. Therefore, the invention as described herein contemplates all such embodiments as may come within the scope of the following claims and equivalents thereof.
Claims (57)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/169,724 US6754266B2 (en) | 1998-10-09 | 1998-10-09 | Method and apparatus for use in transmitting video information over a communication network |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/169,724 US6754266B2 (en) | 1998-10-09 | 1998-10-09 | Method and apparatus for use in transmitting video information over a communication network |
Publications (2)
Publication Number | Publication Date |
---|---|
US20010016008A1 true US20010016008A1 (en) | 2001-08-23 |
US6754266B2 US6754266B2 (en) | 2004-06-22 |
Family
ID=22616928
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/169,724 Expired - Lifetime US6754266B2 (en) | 1998-10-09 | 1998-10-09 | Method and apparatus for use in transmitting video information over a communication network |
Country Status (1)
Country | Link |
---|---|
US (1) | US6754266B2 (en) |
Cited By (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2003075579A2 (en) * | 2002-03-05 | 2003-09-12 | Koninklijke Philips Electronics N.V. | Method and system for layered video encoding |
US20050111836A1 (en) * | 1998-12-04 | 2005-05-26 | Sony Corporation. | Data recording and/or reporducing apparatus, method, and system and recording medium |
US20070182728A1 (en) * | 2006-02-06 | 2007-08-09 | Seiko Epson Corporation | Image display system, image display method, image display program, recording medium, data processing apparatus, and image display apparatus |
US20070201754A1 (en) * | 2006-02-24 | 2007-08-30 | Sony Corporation And Sony Electronics Inc. | System and method for performing wavelet-based texture feature extraction and classification |
US7375752B2 (en) | 2000-11-16 | 2008-05-20 | Stmicroelectronics, Ltd. | Solid state imaging device with dedicated single pixel readout channels and associated methods |
EP2101497A1 (en) * | 2008-03-12 | 2009-09-16 | Thomson Licensing | Method and apparatus for transmitting an image in a wireless network |
US20100150113A1 (en) * | 2008-12-17 | 2010-06-17 | Hwang Hyo Sun | Communication system using multi-band scheduling |
US20100162345A1 (en) * | 2008-12-23 | 2010-06-24 | At&T Intellectual Property I, L.P. | Distributed content analysis network |
EP2211553A1 (en) * | 2007-11-13 | 2010-07-28 | Fujitsu Limited | Encoder and decoder |
US20110170607A1 (en) * | 2010-01-11 | 2011-07-14 | Ubiquity Holdings | WEAV Video Compression System |
US20110292287A1 (en) * | 2003-03-20 | 2011-12-01 | Utc Fire & Security Americas Corporation, Inc. | Systems and methods for multi-stream image processing |
US20120327243A1 (en) * | 2010-12-22 | 2012-12-27 | Seyyer, Inc. | Video transmission and sharing over ultra-low bitrate wireless communication channel |
US20140215017A1 (en) * | 2013-01-25 | 2014-07-31 | Mobitv, Inc. | Prioritized side channel delivery for download and store media |
WO2016048186A1 (en) * | 2014-09-24 | 2016-03-31 | Закрытое акционерное общество "Элекард наноДевайсез" | Rapid selection of spatial prediction mode in hevc coding system |
JP2017120979A (en) * | 2015-12-28 | 2017-07-06 | Kddi株式会社 | Moving picture decoding device, moving picture decoding method, moving picture encoding device, moving picture encoding method and computer-readable recording medium |
US20200021823A1 (en) * | 2018-07-16 | 2020-01-16 | The Regents Of The University Of California | Sampled image compression methods and image processing pipeline |
US11102488B2 (en) * | 2019-05-31 | 2021-08-24 | Ati Technologies Ulc | Multi-scale metric-based encoding |
Families Citing this family (69)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8266657B2 (en) | 2001-03-15 | 2012-09-11 | Sling Media Inc. | Method for effectively implementing a multi-room television system |
US6263503B1 (en) * | 1999-05-26 | 2001-07-17 | Neal Margulis | Method for effectively implementing a wireless television system |
JP4409749B2 (en) * | 2000-11-20 | 2010-02-03 | パイオニア株式会社 | Map display system |
US20030041162A1 (en) * | 2001-08-27 | 2003-02-27 | Hochmuth Roland M. | System and method for communicating graphics images over a computer network |
US7526565B2 (en) * | 2003-04-03 | 2009-04-28 | International Business Machines Corporation | Multiple description hinting and switching for adaptive media services |
US8346605B2 (en) | 2004-06-07 | 2013-01-01 | Sling Media, Inc. | Management of shared media content |
EP1769399B1 (en) | 2004-06-07 | 2020-03-18 | Sling Media L.L.C. | Personal media broadcasting system |
US7769756B2 (en) | 2004-06-07 | 2010-08-03 | Sling Media, Inc. | Selection and presentation of context-relevant supplemental content and advertising |
US9998802B2 (en) | 2004-06-07 | 2018-06-12 | Sling Media LLC | Systems and methods for creating variable length clips from a media stream |
US7917932B2 (en) | 2005-06-07 | 2011-03-29 | Sling Media, Inc. | Personal video recorder functionality for placeshifting systems |
US8099755B2 (en) * | 2004-06-07 | 2012-01-17 | Sling Media Pvt. Ltd. | Systems and methods for controlling the encoding of a media stream |
US7975062B2 (en) | 2004-06-07 | 2011-07-05 | Sling Media, Inc. | Capturing and sharing media content |
USRE48433E1 (en) | 2005-01-27 | 2021-02-09 | The Chamberlain Group, Inc. | Method and apparatus to facilitate transmission of an encrypted rolling code |
US8422667B2 (en) | 2005-01-27 | 2013-04-16 | The Chamberlain Group, Inc. | Method and apparatus to facilitate transmission of an encrypted rolling code |
US9148409B2 (en) * | 2005-06-30 | 2015-09-29 | The Chamberlain Group, Inc. | Method and apparatus to facilitate message transmission and reception using different transmission characteristics |
US20060282855A1 (en) * | 2005-05-05 | 2006-12-14 | Digital Display Innovations, Llc | Multiple remote display system |
US7667707B1 (en) | 2005-05-05 | 2010-02-23 | Digital Display Innovations, Llc | Computer system for supporting multiple remote displays |
US8200796B1 (en) | 2005-05-05 | 2012-06-12 | Digital Display Innovations, Llc | Graphics display system for multiple remote terminals |
US11733958B2 (en) | 2005-05-05 | 2023-08-22 | Iii Holdings 1, Llc | Wireless mesh-enabled system, host device, and method for use therewith |
US8019883B1 (en) | 2005-05-05 | 2011-09-13 | Digital Display Innovations, Llc | WiFi peripheral mode display system |
US7702952B2 (en) | 2005-06-30 | 2010-04-20 | Sling Media, Inc. | Firmware update for consumer electronic device |
US7899864B2 (en) * | 2005-11-01 | 2011-03-01 | Microsoft Corporation | Multi-user terminal services accelerator |
US8112513B2 (en) * | 2005-11-30 | 2012-02-07 | Microsoft Corporation | Multi-user display proxy server |
US7493912B2 (en) * | 2006-06-09 | 2009-02-24 | Hartman Brian T | Fixed cone sleeve valve having cone supported by means downstream of the gate in its closed position |
US7460725B2 (en) * | 2006-11-09 | 2008-12-02 | Calista Technologies, Inc. | System and method for effectively encoding and decoding electronic information |
US7720300B1 (en) | 2006-12-05 | 2010-05-18 | Calister Technologies | System and method for effectively performing an adaptive quantization procedure |
US9250084B2 (en) * | 2007-08-10 | 2016-02-02 | Cisco Technology, Inc. | System and method for navigating using multiple modalities |
US8477793B2 (en) * | 2007-09-26 | 2013-07-02 | Sling Media, Inc. | Media streaming device with gateway functionality |
US8350971B2 (en) | 2007-10-23 | 2013-01-08 | Sling Media, Inc. | Systems and methods for controlling media devices |
US8060609B2 (en) | 2008-01-04 | 2011-11-15 | Sling Media Inc. | Systems and methods for determining attributes of media items accessed via a personal media broadcaster |
US8667279B2 (en) | 2008-07-01 | 2014-03-04 | Sling Media, Inc. | Systems and methods for securely place shifting media content |
US8381310B2 (en) | 2009-08-13 | 2013-02-19 | Sling Media Pvt. Ltd. | Systems, methods, and program applications for selectively restricting the placeshifting of copy protected digital media content |
US8667163B2 (en) * | 2008-09-08 | 2014-03-04 | Sling Media Inc. | Systems and methods for projecting images from a computer system |
US20100070925A1 (en) * | 2008-09-08 | 2010-03-18 | Sling Media Inc. | Systems and methods for selecting media content obtained from multple sources |
US8325796B2 (en) | 2008-09-11 | 2012-12-04 | Google Inc. | System and method for video coding using adaptive segmentation |
US9191610B2 (en) * | 2008-11-26 | 2015-11-17 | Sling Media Pvt Ltd. | Systems and methods for creating logical media streams for media storage and playback |
US8438602B2 (en) | 2009-01-26 | 2013-05-07 | Sling Media Inc. | Systems and methods for linking media content |
US8171148B2 (en) * | 2009-04-17 | 2012-05-01 | Sling Media, Inc. | Systems and methods for establishing connections between devices communicating over a network |
US8406431B2 (en) | 2009-07-23 | 2013-03-26 | Sling Media Pvt. Ltd. | Adaptive gain control for digital audio samples in a media stream |
US9479737B2 (en) * | 2009-08-06 | 2016-10-25 | Echostar Technologies L.L.C. | Systems and methods for event programming via a remote media player |
US20110032986A1 (en) * | 2009-08-07 | 2011-02-10 | Sling Media Pvt Ltd | Systems and methods for automatically controlling the resolution of streaming video content |
US8799408B2 (en) | 2009-08-10 | 2014-08-05 | Sling Media Pvt Ltd | Localization systems and methods |
US8532472B2 (en) * | 2009-08-10 | 2013-09-10 | Sling Media Pvt Ltd | Methods and apparatus for fast seeking within a media stream buffer |
US20110035466A1 (en) * | 2009-08-10 | 2011-02-10 | Sling Media Pvt Ltd | Home media aggregator system and method |
US8966101B2 (en) | 2009-08-10 | 2015-02-24 | Sling Media Pvt Ltd | Systems and methods for updating firmware over a network |
US9525838B2 (en) * | 2009-08-10 | 2016-12-20 | Sling Media Pvt. Ltd. | Systems and methods for virtual remote control of streamed media |
US20110035765A1 (en) * | 2009-08-10 | 2011-02-10 | Sling Media Pvt Ltd | Systems and methods for providing programming content |
US9565479B2 (en) | 2009-08-10 | 2017-02-07 | Sling Media Pvt Ltd. | Methods and apparatus for seeking within a media stream using scene detection |
US9160974B2 (en) | 2009-08-26 | 2015-10-13 | Sling Media, Inc. | Systems and methods for transcoding and place shifting media content |
US8314893B2 (en) | 2009-08-28 | 2012-11-20 | Sling Media Pvt. Ltd. | Remote control and method for automatically adjusting the volume output of an audio device |
US20110113354A1 (en) * | 2009-11-12 | 2011-05-12 | Sling Media Pvt Ltd | Always-on-top media player launched from a web browser |
US9015225B2 (en) | 2009-11-16 | 2015-04-21 | Echostar Technologies L.L.C. | Systems and methods for delivering messages over a network |
US8799485B2 (en) * | 2009-12-18 | 2014-08-05 | Sling Media, Inc. | Methods and apparatus for establishing network connections using an inter-mediating device |
US8626879B2 (en) | 2009-12-22 | 2014-01-07 | Sling Media, Inc. | Systems and methods for establishing network connections using local mediation services |
US9178923B2 (en) * | 2009-12-23 | 2015-11-03 | Echostar Technologies L.L.C. | Systems and methods for remotely controlling a media server via a network |
US9275054B2 (en) | 2009-12-28 | 2016-03-01 | Sling Media, Inc. | Systems and methods for searching media content |
US8856349B2 (en) | 2010-02-05 | 2014-10-07 | Sling Media Inc. | Connection priority services for data communication between two devices |
US9154799B2 (en) | 2011-04-07 | 2015-10-06 | Google Inc. | Encoding and decoding motion via image segmentation |
US9554132B2 (en) | 2011-05-31 | 2017-01-24 | Dolby Laboratories Licensing Corporation | Video compression implementing resolution tradeoffs and optimization |
US9262670B2 (en) | 2012-02-10 | 2016-02-16 | Google Inc. | Adaptive region of interest |
CN103065326B (en) * | 2012-12-26 | 2015-06-24 | 西安理工大学 | Target detection method based on time-space multiscale motion attention analysis |
GB2516837B (en) | 2013-07-31 | 2015-12-09 | Ip Access Ltd | Network elements, wireless communication system and methods therefor |
US9392272B1 (en) | 2014-06-02 | 2016-07-12 | Google Inc. | Video coding using adaptive source variance based partitioning |
US9578324B1 (en) | 2014-06-27 | 2017-02-21 | Google Inc. | Video coding using statistical-based spatially differentiated partitioning |
US10652743B2 (en) | 2017-12-21 | 2020-05-12 | The Chamberlain Group, Inc. | Security system for a moveable barrier operator |
US11074773B1 (en) | 2018-06-27 | 2021-07-27 | The Chamberlain Group, Inc. | Network-based control of movable barrier operators for autonomous vehicles |
WO2020028502A1 (en) | 2018-08-01 | 2020-02-06 | The Chamberlain Group, Inc. | Movable barrier operator and transmitter pairing over a network |
US10997810B2 (en) | 2019-05-16 | 2021-05-04 | The Chamberlain Group, Inc. | In-vehicle transmitter training |
US11711862B1 (en) | 2021-07-15 | 2023-07-25 | T-Mobile Usa, Inc. | Dual connectivity and carrier aggregation band selection |
Family Cites Families (39)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4751742A (en) | 1985-05-07 | 1988-06-14 | Avelex | Priority coding of transform coefficients |
JPS62230281A (en) * | 1986-03-31 | 1987-10-08 | Toshiba Corp | Picture transmission system |
US4821119A (en) * | 1988-05-04 | 1989-04-11 | Bell Communications Research, Inc. | Method and apparatus for low bit-rate interframe video coding |
SE8802229D0 (en) | 1988-06-14 | 1988-06-14 | Ericsson Telefon Ab L M | MOBILE RADIO STATION PROCEDURE |
US5214789A (en) | 1989-11-17 | 1993-05-25 | Uniden America Corporation | Radio channel allocation based on location of mobile users |
US5001561A (en) * | 1990-05-02 | 1991-03-19 | At&T Bell Laboratories | Embedded coding system for video signals |
US5115309A (en) | 1990-09-10 | 1992-05-19 | At&T Bell Laboratories | Method and apparatus for dynamic channel bandwidth allocation among multiple parallel video coders |
US5455629A (en) | 1991-02-27 | 1995-10-03 | Rca Thomson Licensing Corporation | Apparatus for concealing errors in a digital video processing system |
EP0502545B1 (en) * | 1991-03-07 | 1996-09-11 | Mitsubishi Denki Kabushiki Kaisha | Encoding apparatus for encoding a digital image signal |
US5625713A (en) * | 1991-08-09 | 1997-04-29 | Ricoh Corporation | Apparatus and method for increasing the throughput of an acoustic or image compression system |
US5315670A (en) | 1991-11-12 | 1994-05-24 | General Electric Company | Digital data compression system including zerotree coefficient coding |
US5216503A (en) | 1991-12-24 | 1993-06-01 | General Instrument Corporation | Statistical multiplexer for a multichannel image compression system |
US5235633A (en) | 1991-12-26 | 1993-08-10 | Everett Dennison | Cellular telephone system that uses position of a mobile unit to make call management decisions |
US5436665A (en) * | 1992-03-03 | 1995-07-25 | Kabushiki Kaisha Toshiba | Motion picture coding apparatus |
GB2270811B (en) | 1992-09-17 | 1996-02-07 | Sony Broadcast & Communication | Video data compression |
US5933567A (en) * | 1993-01-13 | 1999-08-03 | Hitachi America, Ltd. | Method and apparatus for controlling the position of the heads of a digital video tape recorder during trick play operation and for recording digital data on a tape |
US5412741A (en) | 1993-01-22 | 1995-05-02 | David Sarnoff Research Center, Inc. | Apparatus and method for compressing information |
KR940020832A (en) * | 1993-02-25 | 1994-09-16 | 김주용 | Adaptive Quantization Method of High-definition Television and System Coder Using the Same |
US5513379A (en) | 1994-05-04 | 1996-04-30 | At&T Corp. | Apparatus and method for dynamic resource allocation in wireless communication networks utilizing ordered borrowing |
US5572221A (en) | 1994-10-26 | 1996-11-05 | Telefonaktiebolaget Lm Ericsson | Method and apparatus for detecting and predicting motion of mobile terminals |
US5825759A (en) | 1994-10-26 | 1998-10-20 | Telefonaktiebolaget Lm Ericsson | Distributing network services and resources in a mobile communications network |
US5629780A (en) * | 1994-12-19 | 1997-05-13 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Image data compression having minimum perceptual error |
US5621467A (en) | 1995-02-16 | 1997-04-15 | Thomson Multimedia S.A. | Temporal-spatial error concealment apparatus and method for video signal processors |
US5675384A (en) | 1995-10-03 | 1997-10-07 | Nec Usa, Inc. | VBR MPEG video encoding for ATM networks with dynamic bandwidth renegotiation |
KR100461210B1 (en) * | 1995-10-25 | 2005-06-17 | 코닌클리케 필립스 일렉트로닉스 엔.브이. | Recorded and readable media containing segmented image coding methods and systems and corresponding decoding methods and systems, and coded signals and computer readable storage media storing encoded signals |
KR100308627B1 (en) * | 1995-10-25 | 2001-11-02 | 마찌다 가쯔히꼬 | Low bit rate encoder using overlapping block motion compensation and zerotree wavelet coding |
US5896176A (en) | 1995-10-27 | 1999-04-20 | Texas Instruments Incorporated | Content-based video compression |
US6002802A (en) * | 1995-10-27 | 1999-12-14 | Kabushiki Kaisha Toshiba | Video encoding and decoding apparatus |
US5877812A (en) | 1995-11-21 | 1999-03-02 | Imedia Corporation | Method and apparatus for increasing channel utilization for digital video transmission |
KR100196872B1 (en) | 1995-12-23 | 1999-06-15 | 전주범 | Apparatus for restoring error of image data in image decoder |
US6011587A (en) * | 1996-03-07 | 2000-01-04 | Kokusai Denshin Denwa Kabushiki Kaisha | Packet video bitrate conversion system |
DE19609859C1 (en) * | 1996-03-13 | 1997-07-24 | Siemens Ag | Method for forming an image transformation matrix for an arbitrarily shaped image segment of a digital image, by a computer |
US5943347A (en) | 1996-06-07 | 1999-08-24 | Silicon Graphics, Inc. | Apparatus and method for error concealment in an audio stream |
US5926476A (en) | 1996-07-09 | 1999-07-20 | Ericsson, Inc. | Network architecture for broadband data communication over a shared medium |
EP0861001B1 (en) | 1997-02-07 | 2012-05-23 | Texas Instruments Incorporated | Error resilient video encoding |
US6078616A (en) | 1997-03-13 | 2000-06-20 | Sony Corporation | Methods and apparatus for error concealment utilizing temporal domain motion vector estimation |
US6052598A (en) | 1997-09-30 | 2000-04-18 | At&T Corp | Method for predicting the location of a mobile station in a mobile communications network |
US6141453A (en) | 1998-02-11 | 2000-10-31 | Motorola, Inc. | Method, device and digital camera for error control and region of interest localization of a wavelet based image compression system |
US6125278A (en) | 1998-07-27 | 2000-09-26 | Wieczorek; Alfred A. | Method for optimizing resource allocation based on subscriber transmission history |
-
1998
- 1998-10-09 US US09/169,724 patent/US6754266B2/en not_active Expired - Lifetime
Cited By (39)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7750935B2 (en) * | 1998-12-04 | 2010-07-06 | Sony Corporation | Apparatus for setting a data stream band width and associated methodology |
US20050111836A1 (en) * | 1998-12-04 | 2005-05-26 | Sony Corporation. | Data recording and/or reporducing apparatus, method, and system and recording medium |
US7375752B2 (en) | 2000-11-16 | 2008-05-20 | Stmicroelectronics, Ltd. | Solid state imaging device with dedicated single pixel readout channels and associated methods |
WO2003075579A3 (en) * | 2002-03-05 | 2003-12-31 | Koninkl Philips Electronics Nv | Method and system for layered video encoding |
WO2003075579A2 (en) * | 2002-03-05 | 2003-09-12 | Koninklijke Philips Electronics N.V. | Method and system for layered video encoding |
US8681859B2 (en) * | 2003-03-20 | 2014-03-25 | Utc Fire & Security Americas Corporation, Inc. | Systems and methods for multi-stream image processing |
US20110292287A1 (en) * | 2003-03-20 | 2011-12-01 | Utc Fire & Security Americas Corporation, Inc. | Systems and methods for multi-stream image processing |
US20070182728A1 (en) * | 2006-02-06 | 2007-08-09 | Seiko Epson Corporation | Image display system, image display method, image display program, recording medium, data processing apparatus, and image display apparatus |
CN101390108B (en) * | 2006-02-24 | 2012-07-04 | 索尼株式会社 | System and method for performing wavelet-based texture feature extraction and classification |
US7734107B2 (en) | 2006-02-24 | 2010-06-08 | Sony Corporation | System and method for performing wavelet-based texture feature extraction and classification |
US20070201754A1 (en) * | 2006-02-24 | 2007-08-30 | Sony Corporation And Sony Electronics Inc. | System and method for performing wavelet-based texture feature extraction and classification |
WO2007100701A2 (en) * | 2006-02-24 | 2007-09-07 | Sony Corporation | System and method for performing wavelet-based texture feature extraction and classification |
WO2007100701A3 (en) * | 2006-02-24 | 2008-04-24 | Sony Corp | System and method for performing wavelet-based texture feature extraction and classification |
EP2211553A4 (en) * | 2007-11-13 | 2011-02-02 | Fujitsu Ltd | Encoder and decoder |
EP2211553A1 (en) * | 2007-11-13 | 2010-07-28 | Fujitsu Limited | Encoder and decoder |
US8824268B2 (en) | 2008-03-12 | 2014-09-02 | Thomson Licensing | Method and apparatus for transmitting an image in a wireless network |
US20100322152A1 (en) * | 2008-03-12 | 2010-12-23 | Thomson Licensing | Method and apparatus for transmitting an image in a wireless network |
EP2101497A1 (en) * | 2008-03-12 | 2009-09-16 | Thomson Licensing | Method and apparatus for transmitting an image in a wireless network |
KR101560013B1 (en) | 2008-03-12 | 2015-10-13 | 톰슨 라이센싱 | Method and apparatus for transmitting an image in a wireless network |
US8571568B2 (en) * | 2008-12-17 | 2013-10-29 | Samsung Electronics Co., Ltd. | Communication system using multi-band scheduling |
US20100150113A1 (en) * | 2008-12-17 | 2010-06-17 | Hwang Hyo Sun | Communication system using multi-band scheduling |
US9078019B2 (en) | 2008-12-23 | 2015-07-07 | At&T Intellectual Property I, L.P. | Distributed content analysis network |
US20100162345A1 (en) * | 2008-12-23 | 2010-06-24 | At&T Intellectual Property I, L.P. | Distributed content analysis network |
US8495699B2 (en) * | 2008-12-23 | 2013-07-23 | At&T Intellectual Property I, L.P. | Distributed content analysis network |
US9843843B2 (en) | 2008-12-23 | 2017-12-12 | At&T Intellectual Property I, L.P. | Distributed content analysis network |
US20110170607A1 (en) * | 2010-01-11 | 2011-07-14 | Ubiquity Holdings | WEAV Video Compression System |
US9106925B2 (en) * | 2010-01-11 | 2015-08-11 | Ubiquity Holdings, Inc. | WEAV video compression system |
US20120327243A1 (en) * | 2010-12-22 | 2012-12-27 | Seyyer, Inc. | Video transmission and sharing over ultra-low bitrate wireless communication channel |
US10375534B2 (en) * | 2010-12-22 | 2019-08-06 | Seyyer, Inc. | Video transmission and sharing over ultra-low bitrate wireless communication channel |
US20140215017A1 (en) * | 2013-01-25 | 2014-07-31 | Mobitv, Inc. | Prioritized side channel delivery for download and store media |
US9338204B2 (en) * | 2013-01-25 | 2016-05-10 | Mobitv, Inc. | Prioritized side channel delivery for download and store media |
WO2016048186A1 (en) * | 2014-09-24 | 2016-03-31 | Закрытое акционерное общество "Элекард наноДевайсез" | Rapid selection of spatial prediction mode in hevc coding system |
JP2017120979A (en) * | 2015-12-28 | 2017-07-06 | Kddi株式会社 | Moving picture decoding device, moving picture decoding method, moving picture encoding device, moving picture encoding method and computer-readable recording medium |
CN108464003A (en) * | 2015-12-28 | 2018-08-28 | 凯迪迪爱通信技术有限公司 | Animated image decoding device, animated image coding/decoding method, moving picture encoding equipment, moving picture encoding method and computer readable storage medium |
US20180302650A1 (en) * | 2015-12-28 | 2018-10-18 | Kddi Corporation | Moving image decoding apparatus, moving image decoding method, moving image encoding apparatus, moving image encoding method and computer-readable storage medium |
EP3399750A4 (en) * | 2015-12-28 | 2019-01-02 | KDDI Corporation | Moving image decoding device, moving image decoding method, moving image encoding device, moving image encoding method, and computer-readable recording medium |
US20200021823A1 (en) * | 2018-07-16 | 2020-01-16 | The Regents Of The University Of California | Sampled image compression methods and image processing pipeline |
US11582467B2 (en) * | 2018-07-16 | 2023-02-14 | The Regents Of The University Of California | Sampled image compression methods and image processing pipeline |
US11102488B2 (en) * | 2019-05-31 | 2021-08-24 | Ati Technologies Ulc | Multi-scale metric-based encoding |
Also Published As
Publication number | Publication date |
---|---|
US6754266B2 (en) | 2004-06-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6754266B2 (en) | Method and apparatus for use in transmitting video information over a communication network | |
US6519004B1 (en) | Method for transmitting video information over a communication channel | |
US6289297B1 (en) | Method for reconstructing a video frame received from a video source over a communication channel | |
US9918085B2 (en) | Media coding for loss recovery with remotely predicted data units | |
US7668382B2 (en) | Block-based fast image compression | |
US6275614B1 (en) | Method and apparatus for block classification and adaptive bit allocation | |
US5896176A (en) | Content-based video compression | |
US7792372B2 (en) | Method and apparatus for encoding/decoding image based on region of interest | |
US20020009143A1 (en) | Bandwidth scaling of a compressed video stream | |
US20060188014A1 (en) | Video coding and adaptation by semantics-driven resolution control for transport and storage | |
US8116581B2 (en) | Efficient image representation by edges and low-resolution signal | |
US20020009233A1 (en) | Color encoding and decoding method | |
US20070053445A1 (en) | System and method for providing video content and concealment dependent error protection and scheduling algorithms | |
US8411743B2 (en) | Encoding/decoding system using feedback | |
US6882686B2 (en) | System and method for object-oriented video processing | |
CA2188840C (en) | Content-based video compression | |
EP1227684A2 (en) | Encoding of video signals | |
Barua et al. | Saliency guided wavelet compression for low-bitrate image and video coding | |
US7218842B1 (en) | Efficient methods of performing motion compensation based decoding and recoding of compressed video bitstreams | |
JP2001238220A (en) | Moving picture coding apparatus and moving picture coding method | |
KR100494275B1 (en) | Content-based video compression | |
EP1830574A2 (en) | Video compression using resynchronization words between sequences of symbols | |
Nystrom et al. | Multiple description image coding using regions of interest | |
Leontaris et al. | Region-of-interest video compression with a composite and a long-term frame | |
Bahl et al. | Adaptive region-based multiscaled motion-compensated video coding for error-prone communication channels |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MICROSOFT CORPORATION, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BAHL, PARAMVIR;REEL/FRAME:009944/0564 Effective date: 19990304 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
AS | Assignment |
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034541/0001 Effective date: 20141014 |
|
FPAY | Fee payment |
Year of fee payment: 12 |