|Publication number||US4731664 A|
|Application number||US 06/794,667|
|Publication date||15 Mar 1988|
|Filing date||4 Nov 1985|
|Priority date||8 Nov 1984|
|Also published as||CA1243110A, CA1243110A1|
|Publication number||06794667, 794667, US 4731664 A, US 4731664A, US-A-4731664, US4731664 A, US4731664A|
|Inventors||Mitsuo Nishiwaki, Shuzo Tsugane, Naoki Mukawa, Hideo Kuroda|
|Original Assignee||Nec Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (5), Referenced by (23), Classifications (28), Legal Events (5)|
|External Links: USPTO, USPTO Assignment, Espacenet|
The present invention relates to a method and an arrangement for refreshing a frame memory in an interframe encoding system.
In a television system, thirty frames of video signals are transmitted in one second, with a 33 msec. time interval between two successive frames. There is little variation from one frame to the other. In the particular case of video signals televising a conference (hereinafter referred to as "TV conference signals"), video signals of two successive frames differ very little since participants very rarely exhibit motions. With such video signals, considerable compression of frequency bandwidth is effected by taking a difference between two frames and transmitting only the difference from the areas of motions. This is known as an "interframe encoding system."
In the prior art interframe encoding system, a frame of video signals is segmented into blocks, each of which has a size of m lines x n picture cells (pixels). When all the pixels in one block are, for example, static, the block is then considered as being ineffective and an ineffective block signal is transmitted. On the other hand, when a block contains areas of motion, the block is considered as being effective and an effective block signal is transmitted. At the same time, the difference between the pixels contained in the effective block is quantized and encoded to be transmitted to the receiving side. Details of such a block encoding system are available in the conference record of the International Conference on Digital Satellite Communications, 1975, pp. 309-314, under the title of "H-1 NETEC SYSTEM: INTERFRAME ENCODER FOR NTSC TELEVISION SIGNAL" (reference 1) by T. Ishiguro et al.
In the prior art interframe encoding system, as described above, the interframe difference alone is transmitted. At the receiving side, the interframe difference is added to the content from a frame memory to reproduce the original video signal. If a code error occurs in a transmission line, the original video signal cannot be reproduced precisely at the receiving side because the interframe difference which is transmitted differs from the interframe difference received at the receiving side. The effect of such a code error in the transmission line continues until the contents of the frame memories at the transmitting and receiving sides become identical. It is, therefore, necessary to refresh the contents of the frame memories with updated data. Such a technique is generally termed as the refreshing of a frame memory.
The conventional refreshing methods can roughly be classified into two classes:
(i) A "demand" refreshing method is one in which the contents of the frame memories of both transmitting and receiving sides are refreshed (1) by transmitting, via an additional transmission line, a refreshing demand signal, and (2) by synchronizing the refreshing of the two frame memories when a code error on the transmission line is detected at the receiving side.
(ii) A "periodic" refreshing method in which a command signal, instructing a refreshing of the frame memory at the receiving side, is supplied periodically from the transmitting side at given intervals irrespective of code error occurrences on the transmission line.
However, since the demand refreshing method requires an additional transmission line for sending the refreshing demand signal from the receiving side to the transmitting side, the demand method cannot be employed in a one-way communication system which is not provided with such an additional line. A periodic refreshing method is described in U.S. Pat. No. 4,051,530 (reference 2). The difference between the pixels is taken for each pixel, thereby requiring an enormous amount of refreshing data to finish the refreshing of all the frame memory contents. It has, therefore, been a common practice to limit the number of pixels per frame to be refreshed to, for example, those pixels required for five lines. However, this common practice is still defective since a large amount of the refreshing data for pixels in five lines will be produced at one time. Thus, a necessary video signal which is indicative of the movements of an object cannot be adequately transmitted. For example, assume that the refreshing data consists of an 8 bit PCM data per pixel, the number of pixels per line is 455, and the number of lines to be refreshed per frame is five lines. Then, the refreshing data amounting to as much as 5×455×8=18,200 bits will occur all at once, in a burst. That amount corresponds to approximately 36% of 50 Kbits, which is the approximate transmission capacity per frame period, when a video signal is transmitted at 1.5 Mb (thirty frames)/s, making it difficult to transmit sufficient motion of the object, as mentioned above.
An object of the present invention is to provide a method for refreshing a frame memory which prevents refreshing data from being produced all at once.
According to one aspect of the present invention, there is provided a method for refreshing a frame memory which is used in a system for efficiently encoding video signals. The video signal stored in the frame memory is divided into n number of blocks, which in turn, comprise m number of lines each. Each of the blocks is further segmented into l number of sub-blocks. Each of the sub-blocks is refreshed once for every one frame, repeating l number of times to complete refreshing one block. By repeating such a block refreshing for n number of times, the frame memory can be refreshed.
The above and other objects, features and advantages of the present invention will become apparent from the following detailed description when taken with the accompanying drawings.
FIGS. 1A and 1B are block diagrams showing one embodiment of the present invention;
FIG. 2 is an illustration showing the construction of a video frame;
FIG. 3 is an illustration explaining assignment of lines for periodic refreshing; and
FIGS. 4 and 5 are circuit diagrams of the refreshing controller at the transmitting and receiving sides, respectively.
Referring now to FIG. 1A, a video signal supplied to an input terminal 1 is band limited with a low-pass filter 2, and then is supplied to an analog-to-digital "A/D" converter 3 and a synchronization ("synch") signal separator 4. The sync separator 4 separates from the video signal a horizontal sync signal ("H sync") and a frame sync signal ("F sync").
Responsive to the H sync signal and F sync signal, a clock generator 5 generates various clock pulses (C) which are necessary for encoding. Generator 5 supplies the clock pulses to circuits which require them. The clock generator 5 also generates a multiplexer control signal (C1) which is necessary for controlling a multiplexer 14. A refreshing controller 6 is responsive to the H sync and F sync signals and generates a refreshing control signal R, a line indication signal CRM for designating lines to be refreshed, and a switch control signal SWCONT. Both the signals R and SWCONT are described later.
The A/D converter 3 samples an analog video signal and converts the sample into an 8 bit PCM signal, per pixel. A prediction signal from a switch 12 is subtracted from the PCM signal in a subtractor 7 to produce the difference which is then quantized in a quantizer 8 in accordance with a predetermined quantization characteristic. The quantized output is added to the prediction signal from the switch 12 in an adder 9, to obtain a locally decoded signal. The locally decoded signal is stored in a frame memory 10 and in a one-sample memory 11 for generating the prediction signal from the subsequent encoding. The frame memory 10 delays the locally decoded signal by one frame period while the one sample memory 11 delays the same decoded signal by one sample period. The switch 12 receives the prediction signals from the frame memory 10 and from the one sample memory 11. Depending on the switch control signal SWCONT from the refreshing controller 6, switch 12 selects either the output from the one sample memory 11 when refreshing is required or the output from the frame memory 10 when refreshing is not necesarry. The prediction signal from the switch 12 is supplied to the subtractor 7 and the adder 9.
The quantizer 8 supplies a signal which is indicative of the quantization level to a code converter 13. When there is no difference between (1) each pixel in one block comprising m lines x n pixels, and (2) each corresponding pixel of the previous period, the code converter 13 deems the block as being ineffective. Responsive thereto, code converter 13 gives an ineffective block signal alone and deems all the other blocks effective. For the effective block, the code converter 13 converts, in succession to an effective block signal. The converter quantizes the levels of all the pixels contained in the effective block into codes V, in accordance with the predetermined code assignment. A shorter code word is assigned to a quantization level which occurs at higher probability while a longer code word is assigned to a less probable quantization level.
A mode controller 16 detects the amount of data stored in a buffer memory 15 and controls the amount of code generation in accordance with an encoding mode signal. The mode signal relates to the detected memory amount, which prevents overflow of the buffer memory 15.
More specifically, when the buffer occupancy increases, the mode controller 16 controls the quantizer 8 and causes it to roughly quantize signals. Alternatively, the mode controller 16 selects the coding mode, such as a sub-sampling coding mode, in which every other pixel is coded, instead of coding all of the pixels. A field repeating mode is one in which, instead of coding all of the fields, every other field is coded.
In a time division, the multiplexer 14 multiplexes the F sync signal, the encoded codes V from the code converter 13, the refreshing signal R, the line indication signal CRM from the refreshing controller 6, and the encoding mode signal Q from the mode controller 16. The multiplexed signal is fed to a buffer memory 15. The buffer memory 15 temporarily stores the output from the multiplexer 14, reads out the stored data at a given transmission rate, constructs a transmission frame, and then transmits the constructed frame to a digital transmission line.
Referring to FIG. 1B, at the receiving side, the transmitted video signal is supplied to a terminal 101 and is stored in a buffer memory 102. A clock generator 113 extracts clock data from the transmitted signal and generates, on the basis of that extracted clock data, various clock pulses (C, C1) which are required for decoding and supplies them to the necessary circuits.
A demultiplexer 103 responds to the clock pulse C1 by reading out data from the buffer memory 102 and by supplying: (1) the encoding mode signal Q to a mode controller 114, (2) the refreshing control signal R and the line indication signal CRM to a refreshing controller 104, and (3) the ineffective or effective block signal and the encoded codes V to a decoder 105, respectively. The mode controller 114 controls the decoding mode according to the encoding mode signal Q so that the decoding mode corresponds with the encoding mode used at the transmitting side, such as the sub-sampling mode or field repeating mode. In response to the refreshing signal R and the line indication signal CRM, a refreshing controller 104 produces the switch control signal SWCONT to a switch 109 synchronously with the switching operation of the switch 12 at the transmitting side. The decoder 105 decodes the ineffective or effective block signal and the code V to a decoded signal.
An adder 106 obtains the original digital video signal by adding the decoded signal from the decoder 105 and the prediction signal from the switch 109 and supplies the same to a digital-to-analog ("D/A") converter 110, a frame memory 107, and a one sample memory 108, respectively. The frame memory 107 has one frame delay, while the one sample memory 108 has one sample delay.
In response to the switch control signal SWCONT from the refreshing controller 104, switch 109 selects the prediction signal from the one sample memory 108 when the refreshing is required while selecting the prediction signal from the frame memory 10 when the refreshing is not required. The selected prediction signal is supplied to the adder 106. As a result, during the period of the refreshing, the input of the adder 106 and the input of the one sample memory 108 are added to produce the refreshing data, which is used to refresh the contents of the frame memory 107. In this manner, the influence of the code error in the transmission line can be eliminated. The D/A converter 110 receives the original digital video signal supplied from the adder 106 and converts it into the original analog video signal. This analog video signal is band limited by a low-pass filter 111 and transmitted to a video signal output terminal 112.
The method for designating lines to be refreshed will now be explained in relationship to the method for constructing a video frame in the multiplexer 14.
Referring now to FIG. 2, a video frame comprises sixty-four blocks, each of which consists of eight lines. Of these sixty-four blocks, the first block comprises a frame sync signal, a frame mode signal, a line mode signal, and video data. Each of the second through sixty-fourth blocks comprise a line mode signal and video data.
The frame sync signal is assigned with a pattern which never appears with other signals. Thus, it is possible to synchronize the video frames by detecting the pattern from the data sequence. The frame mode contains a signal showing what coding mode is applied to the whole frame, e.g a sub-sampling mode, as mentioned above. The frame mode also contains the line indication signal CRM which designates the lines to be refreshed as mentioned above. The line indication signal CRM is a signal indicative of the serial number of lines refreshed in the block (m lines) in question. If one block consists of eight lines and refreshing is continuously conducted for two lines at one time, then the line indication signal CRM comprises 2 bits. The line mode has a coding mode which is applied to each block (eight lines). The line mode also includes the signal Q showing quantization characteristics supplied from the mode controller 16 and the refreshing control signals R. A "0" at the signal position R indicates that lines requiring refreshing exist, whereas a "1" at the signal position R means that none of the m lines (block) requires refreshing.
FIG. 3 is a chart to explain how lines are designated for refreshing. For example, when the bits CRM 1 and CRM 2 of the line indication signal CRM are "1" and "0", respectively, then the fifth and sixth lines of the eight lines are refreshed while the rest of the eight lines are encoded in the coding mode designated by the line mode. As mentioned above, if two lines per each block are refreshed once for one frame, 256 frames will be required to refresh one frame. It should be noted that the structural components of the systems shown in FIGS. 1A and 1B are disclosed in the reference 1 and 2, except for the refreshing controllers 6 and 104. The refreshing controllers 6 and 104 will, therefore, be described in more detail hereinafter.
FIG. 4 shows the refreshing controller 6 at the transmitting side.
Referring to FIG. 4, a counter 20 counts the number of lines in one frame by counting the number of H sync signals supplied from the sync separator 4. A counter 21, on the other hand, counts the number of F sync signals. A first comparator 22 compares the output of the fourth through ninth bits (B4-B9) of the counter 20 with the output of the third through eighth bits (b3-b8) of the counter 21 and produces the refreshing signal R. For example, when the first block is refreshed, the counter output b3-b8 keeps "000000" during the first through fourth frames, while the counter output B4-B9 holds "000000" during the first through eighth lines in each frame from the first through fourth frames. The comparator 22, therefore, produces the refreshing signal R for four times for one block. It is noted that since the first and the second bits (b1-b2) of the counter 21 change from frame to frame, the output b1-b2 is used as the line indication signal having the refreshing bits CRM 1 and CRM 2.
A second comparator 23 and an AND gate 24 form the switch control signal SWCONT for controlling the switch 12 to transmit the output from the one sample memory 11 as refreshing data during the refreshing period. The comparator 23 compares the counter output B2-B3 which does not vary in the first and the second lines of each block as compared with the counter output b1-b2 indicative of the first through fourth frames. Comparator 23 produces a signal corresponding to the lines to be refreshed in the first through fourth frames. For example, when "00" indicating the first frame appears at the output b1-b2, "00" showing the first and second lines appears at the output B2-B3. The comparator 23 compares the outputs B2-B3 and b1-b2, and produces a signal for refreshing the first and second lines.
The AND gate 23 responds to the output of the comparator 23 and the refreshing signal R from the comprator 22 produces the switch control signal SWCONT. Since the signal R is outputted in correspondence with the block position which is to be refreshed once per frame, the switch control signal is outputted to refresh two lines once per frame. As described above, the refreshing control signal R and the line indication bits CRM 1 and CRM 2 are multiplexed by the multiplexer 14 (FIG. 1A).
Referring now to FIG. 5 which shows details of the refresh controller 104 at the receiving side. A counter 30 generates an H-synch signal by counting the clock pulses from the clock generator 113 and supplies the H-sync signal to a counter 31 which outputs the number of lines by counting the H-sync signal. Since the output (bs-b3) of the counter 31 changes in every two lines, a line indication signal is obtained at the receiving side by comparing the output b2-b3 in a comparator 32 with the separate line indication bits CRM 1 and CRM 2. An AND gate 33 performs an AND operation in response to the refresh line indication signal and the refreshing signal R to form the switch control signal SWCONT at the receiving side.
Although the method for refreshing a frame memory of a simple interframe encoding system has been described in the foregoing specification, the method can be applied for other encoding systems which incorporate motion compensation, predictive processing, and the like. In the foregoing description, each block which contains a plurality of sub-blocks to be refreshed exists only once in one frame; however, it is again possible that a plurality of blocks containing sub-blocks exist in one frame.
As has been described above, according to the method of the present invention for an interframe encoding system, the block of encoding is further segmented into smaller sub-blocks for periodic refreshing. Thus, refreshing data can be transmitted without producing a burst of data at one time. As a result, the amount of data assigned for transmitting the movements of an object can be increased, for enabling high quality encoding.
Those whose are skilled in the art will readily perceive how to modify the invention. Therefore, the appended claims are to be construed to cover all equivalent structures which fall within the true scope and spirit of the invention.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US4051530 *||20 Oct 1975||27 Sep 1977||Nippon Telegraph And Telephone Public Corporation||Frame-to-frame coding system|
|US4125873 *||29 Jun 1977||14 Nov 1978||International Business Machines Corporation||Display compressed image refresh system|
|US4183058 *||31 Jan 1977||8 Jan 1980||Micro Consultants, Limited||Video store|
|US4183096 *||25 May 1978||8 Jan 1980||Bell Telephone Laboratories, Incorporated||Self checking dynamic memory system|
|US4357686 *||24 Sep 1980||2 Nov 1982||Sperry Corporation||Hidden memory refresh|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US4868653 *||5 Oct 1987||19 Sep 1989||Intel Corporation||Adaptive digital video compression system|
|US4875095 *||15 Jun 1988||17 Oct 1989||Kokusai Denshin Denwa Kabushiki Kaisha||Noise-shaping predictive coding system|
|US4951138 *||2 Mar 1989||21 Aug 1990||L'etat Francais (Cnet)||Method to reduce the throughput rate of a sequence of data for assistance in the reconstruction of an electronic picture using a sub-sampled signal|
|US4969040 *||26 Oct 1989||6 Nov 1990||Bell Communications Research, Inc.||Apparatus and method for differential sub-band coding of video signals|
|US4972261 *||23 Aug 1988||20 Nov 1990||The General Electric Company, P.L.C.||Motion compensation image signal encoding system|
|US4982284 *||28 Jun 1989||1 Jan 1991||Nec Corporation||Video signal decoding system having a frame synchronizer function|
|US5021879 *||24 Sep 1990||4 Jun 1991||U.S. Philips Corporation||System for transmitting video pictures|
|US5036391 *||30 Dec 1988||30 Jul 1991||Thomson Consumer Electronics||Synchronization method for the transmission, of an asynchronous channel, or a series of pictures encoded by means of a variable length code, and device for the implementation of this method|
|US5138447 *||11 Feb 1991||11 Aug 1992||General Instrument Corporation||Method and apparatus for communicating compressed digital video signals using multiple processors|
|US5361096 *||4 Oct 1993||1 Nov 1994||Nec Corporation||Method and apparatus for multiplex transmission of video signals in a plurality of channels with refresh control utilizing intraframe coding|
|US5542008 *||28 Jan 1994||30 Jul 1996||Victor Company Of Japan, Ltd.||Method of and apparatus for compressing image representing signals|
|US5644658 *||6 Jun 1995||1 Jul 1997||Victor Company Of Japan, Ltd.||Method of and apparatus for compressing image representing signals|
|US5686962 *||30 Dec 1994||11 Nov 1997||Samsung Electronics Co., Ltd.||Motion image coder using pre-filter to reduce quantization error|
|US5719962 *||29 Jan 1997||17 Feb 1998||Victor Company Of Japan, Ltd.||Method of and apparatus for compressing image representing signals|
|US6025888 *||3 Nov 1997||15 Feb 2000||Lucent Technologies Inc.||Method and apparatus for improved error recovery in video transmission over wireless channels|
|US6590441 *||1 Jun 2001||8 Jul 2003||Qualcomm Incorporated||System and method for tuning a VLSI circuit|
|US6661844 *||8 Feb 2002||9 Dec 2003||Thomson Licensing S.A.||Adaptive method of encoding and decoding a series of pictures by transformation, and devices for implementing this method|
|US7800690||15 Dec 2004||21 Sep 2010||Thomson Licensing||Low noise encoding and decoding apparatus and method|
|US20050190292 *||15 Dec 2004||1 Sep 2005||Haoping Yu||Low noise encoding and decoding appratus and method|
|EP0429678A1 *||19 Jun 1990||5 Jun 1991||Fujitsu Limited||Dynamic image encoder and dynamic image decoder|
|EP0434427A2 *||20 Dec 1990||26 Jun 1991||Matsushita Electric Industrial Co., Ltd.||Moving-image signal encoding apparatus|
|EP0444839A2 *||25 Feb 1991||4 Sep 1991||Sony Corporation||Video signal transmission system and method and apparatus for coding a video signal|
|WO1991000671A1 *||22 Jun 1990||10 Jan 1991||Thomson Brandt Gmbh||Signal processing system|
|U.S. Classification||375/240.24, 375/E07.255, 365/222, 375/E07.148, 375/E07.244, 375/E07.211, 375/E07.277, 375/E07.224|
|International Classification||H04N7/32, H04N7/60, H04N7/26, H04N7/36, G06F12/00, H04N7/50|
|Cooperative Classification||H04N19/61, H04N19/152, H04N19/107, H04N19/50, H04N19/503, H04N21/236, H04N21/434|
|European Classification||H04N21/434, H04N21/236, H04N7/36, H04N7/32B, H04N7/26A4C2, H04N7/50R, H04N7/50|
|4 Nov 1985||AS||Assignment|
Owner name: NEC CORPORATION, 33-1, SHIBA 5-CHOME, MINATO-KU, T
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST.;ASSIGNORS:NISHIWAKI, MITSUO;TSUGANE, SHUZO;MUKAWA, NAOKI;AND OTHERS;REEL/FRAME:004480/0829
Effective date: 19851101
Owner name: NIPPON TELEGRAPH AND TELEPHONE CORPORATION, 1-6, U
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST.;ASSIGNORS:NISHIWAKI, MITSUO;TSUGANE, SHUZO;MUKAWA, NAOKI;AND OTHERS;REEL/FRAME:004480/0829
Effective date: 19851101
|11 Oct 1988||CC||Certificate of correction|
|15 Aug 1991||FPAY||Fee payment|
Year of fee payment: 4
|3 Apr 1995||FPAY||Fee payment|
Year of fee payment: 8
|7 Sep 1999||FPAY||Fee payment|
Year of fee payment: 12