CA1131766A

CA1131766A - Digital video effects system employing a chroma-key tracking technique

Info

Publication number: CA1131766A
Application number: CA307,185A
Authority: CA
Inventors: Masao Inaba; Kazuo Kashigi
Original assignee: Nippon Electric Co Ltd
Current assignee: NEC Corp
Priority date: 1977-07-11
Filing date: 1978-07-11
Publication date: 1982-09-14
Also published as: US4409618A; AU514964B2; GB2001824A; US4200890A; GB2001824B; FR2397762B1; DE2830420C3; AU3794278A; DE2830420B2; FR2397762A1; DE2830420A1

Abstract

ABSTRACT OF THE DISCLOSURE
In chroma-key tracking systems it is known to compress and insert a second television picture into an area corresponding to a chroma-key frame located at a specified position in a first television picture. When the frame is shifted, there is a time lag before the compressed picture shifts correspondingly. In that interval, the second picture is compressed to in-clude the unused and noisy component of the shifted frame. A poor picture results. The present invention describes a chroma-key tracking system which prevents the generation of this unnecessary and noisy component of the shifted frame. A key signal is generated which represents the chroma-key frame and which is positioned in the first picture. A position signal, pro-duced in response to the key signal, determines the position of a frame cir-cumscribed about the key frame. An imaginary frame is produced in response to an imaginary-frame position signal to be greater in size than the key frame. A compressed second picture is produced from a compressed second video signal which in turn is a function of a second video signal and the imaginary-frame position signal. The second picture is compressed to be identical in size to the imaginary frame. The first video and second com-pressed video signals are then gated in response to the key signal with the result that the compressed picture is inserted into the chroma-key frame in the first picture.

Description

1~317~

This invention relates to digital video effects systems, and more particularly to chroma-key tracking systems for electronically compressing and moving a television picture into a position specified by a chroma-key screen in another picture and for inserting the compressed picture into an area corresponding to the chroma-key screen.
The so-called keyed insertion technique by which a part of one tel-evision picture is inserted into another picture to produce a special effect is frequently used in television broadcasting. One example of the technique is the chroma-key insertion by which a part of a first picture is designated by the chroma-key signal produced from that picture and the designated part is inserted into the second picture. However, since a chroma-key signal un-dergoes a change in position and dimension with any movement of the televi-sion camera employed for the pickup of the second picture, the shooting angle of the camera must be changed accordingly. This involves serious difficulty.
To eliminate the difficulty, a method has been proposed in which the picture to be inserted is compressed in accordance with the chroma-key signal (Japanese Patent Publication No. 53-9896 published on April 10, 1978 to Chubu Nippon Broadcasting Corporation~. Accordingly, the position and dimensions of the first picture are determined by comparing the chroma-key signal frame with a standard television frame.
The following description of the keyed insertion technique will be better understood when read in conjunction with the following drawings, in which;
Figures lA to lE and 2A to 2C show television pictures relating to video signals produced by a conventional system.
Figure 3 is a block diagram of an embodiment of this invention;

. ~
~`

.:
:
.

1'1 3~ ~6Ç;

Figure 4 is a block diagram of the frame-position signal detecting circuit used in the embodiment shown in Figure 3;
Figure S illustrates the relationships on the full picture among various frames obtained by one embodiment of this invention;
Figure 6 shows the dime~sional relationship between a chroma-key frame and an imaginary-frame;
Figure ~ is a block diagram of the arithmetic circuit employed in the embodiment shown in Figure 3; and Figure 8 is a block diagram of the picture compressing circuit employed in the embodiment shown in Figure 3.
Referring to Figure 1, it is assumed that picture B is compressed into the size of a chroma-key signal C to produce a picture D and that the picture D is inserted into a picture A to produce a picture E. In this in-stance, the chroma-key signal C serves as the standard signal to designate into what position and what dimensions picture D should be compressed.
Generally, the chroma-key signal C is produced by mixing, in appropriate proportions, the blue component which is the main constituent of the various chromatic components (the red, green and blue components) constituting the picture A, ~ith the two other components. It is usual therefore, when pro-ducing a chroma-key signal, to compose picture A with the hatched portion left in blue.
In the temporal relationship between a chroma-key signal and the compressed picture corresponding to it, the latter tends to delay as will be explained below. Accordingly, when a chroma-key signal moves, an inconvenience may arise from the inability of the compressed picture to move correspondingly.
Thus, Figure 2A shows a normal ou~put picture Figure 2B shows another :

.
. . :
.
.

76~;

output picture immediately after the movement of the chroma-key signal towards the right side of the frame. Since the compressed picture inserted into the chroma-key frame moves with a time lag, it momentarily remains in its original position as depicted in 2A.
The result is the unnecessary part of the chroma-key signal represented by hatching in Figure 2B.
Moreover, since brighter or white parts of the object involve much of the blue component, noises occur in addition to the desired key signal in many actual instances, as illustrated in Figure 2C. If the picture is compressed to correspond with such a chroma-key signal, it will be compressed to a size includ-ing the noise component (the size represented by dotted lines in Figure 2C) and look like Figure 2D. The insertion of picture D
into the picture C would result in the picture shown in Figure 2E.
This result is undesirable because of the failure of the inserted picture to be fully contained within the designated chroma-key frame. If the noise disappears, the picture will return to the state illustrated in Figure lE. However, noise components usually appear intermittently, and when they do appear, the results are unpleasant to look at.
An object of the present invention is therefore to pro-vide a chroma-key tracking system which, even when the chroma-key frame is shifted in position, prevents any unnecessary part of the frame from being generated.
Another object of this invention is to provide a chroma-key tracking system which is capable of correctly extracting the chroma-key frame even when the video signal on which the chroma-. key signal is based has a component which cannot be readily dis-. ~ :
::

~, : .

113~7f~6 , tinguished from the chroma-key signal.
According to the present invention, there is provided a tracking system for producing special effects on a television picture~_~ictur~ comprised of a first picture and a second picture inserted into said first picture, said first and second pictures being represented by first and second video signals respectively, said system comprising: means for producing a key signal repre-senting a key frame to be positioned on said first picture; means responsive to said key signal for producing a position signal ; 10 representins a position of a circumscribed frame of said key frame; means responsive to said position signal for producing an imaginary-frame position signal representing an imaginary frame greater than said key frame; means responsive to said second video signal and said imaginary-frame position signal for producing a compressed second video signal representing a compressed second picture, said compressed second picture being identical in size to said imaginary frame; and means responsive to said key signal for selectively combining said first video signal and said compressed second video signal, whereby ~ said special effects are produced.
The invention will now be described in detail with refer-ence to Figures 3 to 8.
With reference to Figure 3 showing an embodiment of this invention, a first video signal A is supplied to a first input terminal 1, a second video signal B to a second input terminal 2 and a control signal to a third input terminal 3. A chroma-key signal generator 11 generates, in response to the first video signal from the first input terminal l, a chroma-key signal. The chroma-key signal is supplied to a gate circuit 12 via a switch 19, . .

, , .

f ~f~

and gated by the output of a circumscribed frame signal generator 15 to be deprived of its noise component. The output of the gate circuit 12 is supplied to a frame-position signal detecting cir-cuit 13 for detecting four values, i.e., those of the leftmost and rightmost points in the horizontal direction and the upper-most and lowermost points in the vertical direction of -4a-,~' , , ........... . .
, ~;317~6 the chroma-key signal.
One example of the frame position signal detecting circuit 13 is illustrated in Figure 4. A chroma-key signal 31, whose noise component has been removed by the gate circuit 12, is shaped by a wave shaper 33, and fed to horizontal and vertical position detectors 30H and 30V.
In the horizontal position detector 30H, a counter 35, so composed as to be driven by clock pulses 34 to advance by one per clock pulse, is cleared to zero by a pulse 32 for each horizontal scanning period of a television signa] The counter 35 feeds the counted values to a minimum-lo va]ue detector 36 and a maximum-value detector 37. The minimum value detector 36 detects the counted value of the counter 35 as the minimum horizontal value at the leading edge of the chroma-key signal once every horizontal scanning per~od. The minimum horizontal value represents the leftmost posi_ tion of the chroma-key signal in the horizontal direction. The maximum-value detector 37 holds the counted value of the counter 35 at the trailing edge of the chroma-key signal to detect the last-held value in each horizontal scanning period. The detected value is the maximum horizontal value re-presenting the rightmost position of the chroma-key signal in the horizontal direction. The detectors 36 and 37 are reset by the clear pulse 32.
The detected minimum and maximum horizontal values are supplied to hysterçsis circuits 38 and 39, respectively, which are for removing the jitters present at the rising and trailing edges of the chroma-key signal.
A chroma-key signal, even if the original picture from which it is made is motionless, usually is susceptible to some j~tters at its leading and trailing edges. Accordingly, the outputs of the maximum-value detector 36 and the minimum-value detector 37 are constantly fluctuating in minute degree. The _ 5 _ 113176~

hysteresis circuits 38 and 39, so composed that their outputs may not vary even if their inputs minutely fluctuate, greatly contribute to stabilization of the functioning of the system. The stabilized outputs 40 and 41 are fed to an arithmetic circuit 14 shown in Figure ~.
In the vertical position detector 30V, a counter 42, so composed as to be driven by the horizontal synchronizing pulse 49 to advance by one per horizontal period, is reset to zero by a clear pulse 32' for each television field. The vertical position detector 30V functions in substantially the same manner as the horizontal position detector 30H except for the period of opera-lo tion. A minimum-value detector 43 detects a minimum vertical value represent-ing the uppermost position of the chroma-key signal. A maximum-value detector 44 detects a maximum vertical value representing the lowermost position of the chroma-key signal. The detected minimum and maximum vertical values are supplied to hysteresis circuits 45 and 46, respectively to remove jitters.
The stabilized outputs 47 and 48 are supplied to the arithmetic circuit 14 of Figure 3.
The four values obtained from the circumscribed frame signal detector 13 represent the dimensions and position of the quadrilateral circumscribing around the chroma-key signal. Thus in Figure 5, a reference numeral 21 in-dicates the dimensions of the standard picture, 25 shows the chroma-key frame and the four detected values correspond to the points of the four corners of the circumscribed frame 24. These four values are fed to the arithmetic cir-cuit 14 and so corrected as to enlarge the dimensions of the frame. In Figure 5, reference numerals 22 and 23 show two frames i.e., imaginary frames corres-ponding to the corrected four values.
One example of the dimensional relationship between the circumscribed : --: - : : :

1~3~766 frame 24 and the imaginary frames 22 or 23 is illustrated in Figure 6. In Figure 6, the abscissa represents the size of the circumscribed frame 24 (hereinafter called the chroma key size) based on the input chroma-key frame 25; and the ordinate represents the size of imaginary frames 22 or 23. The increment in frame size and the chroma-key size are based on the following facts: Even if the chroma-key size is minimal, the increment is not zero but has a certain value. While this value is preferably small to minimize the part to be cut off by a key signal when a compressed picture is inserted into an-other by means of a chroma-key signal, it must be greater than a certain level in view of the possibility of the key signal shifting a bit. The value of the increment should therefore be made externally controllable. Next, it is pro-vided that the increment should decrease as the chroma-key size approaches its maximum size such that the sum of the chroma-key size and increment will not exceed the maximum size of the imaginary frame. This is so because the rate of compression is never gr~ater than one. It is further provided that the increment is greatest when the chroma-key size is at its median because, in the usual state of use, this size of signal is used most frequently and since the greatest number of frame movements are likely to occur at this point.
Whereas the relationship between the chroma-key size and the increment may be determined on the basis of the aforementioned, the system will function if the increment remains constant, although proportioned to the chroma-key size of the set in a relationship combining both.
One example of the arithmetic circuit 14 will be described below with reference to Figure 70 Since the underlying operation is common to both the horizontal and vertical directions, an example of the horizontal direction is given. The minimum value 40 and the maximum value 41 in the horizontal 113~766 - direction from the position detector 13 are supplied to the arithmetic circuit 14. In a subtraction circuit 71 is subtracted the minimum value from the max-imum. The output of the subtraction circuit 71 thus represents the length of ; the circumscribed frame 24 in the horizontal direction. This value then be-comes the input to the coefficient unit 72, and is multiplied by a certain coefficient to let the increment of the frame size include the proportional component of the ~hroma-key s~,ze. Next, a certain constant is added by the adder 73. Then, this output is input to a limiter 74 so as to be restricted from exce~ding a certain value, and so that the frame increment can be prevent-lo ed from becoming too great when the chroma-key size is large. The output of the limiter 74 is led to a sub~ractor 75, and subtracted from the minimum value 40. The remainder of subtraction is input to a minimum value limiter 76.
The minimum value limiter 76 is so composed as to replace a negative input with zero and leave positive inputs unaltered.
Meanwhile, the output of the limiter 74 is also input to an adder 77 to be added to the maximum value 41. The sum is input to a maximum value limiter 78, which is so composed as to replace an input value exceeding the maximum conceivable value for a key signal with such maximum value. The remaining signals not exceeding the maximum value are allowed to pass unaltered.
Since the minimum value 40 is operated upon in the manner described to become smaller and the maximum value 41 is operated on to become greater, the chroma-key signal frame size is corrected to become correspondingly greater.
Correction in the vertical direction is achieved in the same manner as the horizontal correction described above.
The four corrected values are fed to the frame signal generator 15 (Figure 3) to generate a frame signal corresponding to the frame 22 (Figure 5).

i - 8 ' ~: :, ::, This frame signal is supplied to the gate circuit 12 to gate the chroma-key signal and thereby removing the noise component. The noise component outside the frame 22 is thus eliminated.
The arithmetic circuit 14, mean~hile, supplies the picture compress-ing circuit 16 with various values needed for compressing a second video signal to a si~e corresponding to the frame 23. The output of the compressing circuit 16 is supplied to the mixer-keyer 17 as one of its inputs. The mixer-keyer 17 are also supplied with the first video signal A and the chroma-key signal, so that the compressed second video signal B be inserted into and keyed with the first video signal A to give the required output signal ~. The size of the picture compres~ed by the compressing circuit 16 corresponds to the frame 23, and is greater than the circumscribed frame 2~, because the second video signal B is deliberately compressed to a greater si~e than the actual key signal to prevent the emergence, as described above, of the unnecessary part of the chroma-key frame, represented by hatching in Figure 2B, in the output signal which occurs as a result of the time lag between the movement of the chroma-key signal and that of the compressed picture which in turn is due to a delay in the p~ocessing of the signal when the picture is compressed.
Figure 8 is a schematic diagram of the picture compressing dircuit 16, in which the second video signal B from the input terminal 3 is supplied to the analogue/digital converter 81aand conver~ed into a PCM (pulse code modulation) signal. At the same time, the signal B is also supplied to the write-in clock generator 82 to generate a continuous wave phase-locked to the color burst signal. This continuous wa*e is multiplied and then output as a clock pulse to be input into the analogue/digital converter 81. The output PCM
signal from the analogue/digital converter 81 is supplied to the interpolating _ g _ , : ~

1131-7~ti circuit 83.
! The function of the interpolating circuit 83 is to alter the number of picture elements in the horizontal direction and the number of scanning lines in the vertical direction. When, for instance, a picture is to be compressed in a ratio of 1/1.5 in ~he horizontal direction, the circuit 83 allows the first of the series of input picture elements to pass therethrough unaltered. It then interpolates between the second and third picture element to create a picture element corresponding exactly to the middle of the two elements. This new element is then output as the second output picture ele-ment. The fourth input picture(of the two to deliver it as the second of the output picture elements,) element is allowed to pass through the interpolator i and is output as the third output picture~element. By repeating this pro-cedure, the number of output picture elements can be reduced to 1/1.5 of the number of input picture elements~ This is equivalent to a 1.5-fold expansion of the sample gap in the analogue/digital converterO The output of the inter-polating circuit 83 is recorded in memory 84. Accordingly, the compressed picture is already recorded in memory 84. These controls are effected by the signals of the outputs 93 and 94 from the arithmetic circuit 14. The signal 93 controls the gap between the picture elements newly created in the inter-polating circuit 83, and the signal 94 controls the write-in address generator 85 which generates the address when a signal is to be recorded in memory 84 in such a manner that the address value is increased by one increment every time a picture element arrives.
The readout address generator 86 generates the read-out ~ddress to be used when a signal is to be read from memory 84. The switch 87 is to be used for the selection of the write-in address when the signal is to be re-:, : :

~131~

corded, or of the read-out address when it is to be read out. The read-out clock generator 90, using the reference sync signal 92 as input, generates the read-out clock signal which actuates the read-out address generator 86 and the digital/analogue converter 88. The digital/analogue converter 88 converts the read out PCM
signal from the memory 84 into an analogue signal. This analogue signal is fed to the process amplifier 89 and is amplified therein and output as the signal 91. This output signal, in the form of a picture signal compressed into the prescribed position and size, is supplied to the mixer-keyer 17 as one of its inputs.
As explained above, this system, with which it is possible to automatically compress an input picture into the size of a chroma-key signal supplied from outside, is very effective in the production of television programs. Although the chroma-key signal has been referred to in the above description as an example of the key signal, the principle of the present invention is of course directly applicable to the wipe key or the like from the wave generator 18. Certainly, an unprecedented feature of this system is its stability of function in those instances when the key signal contains a noise component.
Reference is made to copending Canadian application Serial No. 309,421 filed on August 15, 1978 by Nippon Electric Co., Ltd. which claims subject matter disclosed but not claimed in the present application.

B

Claims

THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:

1. A tracking system for producing special effects on a television picture comprised of a first picture and a second picture inserted into said first picture, said first and second pictures being represented by first and second video signals respectively, said system comprising: means for producing a key signal representing a key frame to be positioned on said first picture; means responsive to said key signal for producing a position signal representing a position of a circumscribed frame of said key frame; means responsive to said position signal for producing an imaginary-frame position signal representing an imaginary frame greater than said key frame; means responsive to said second video signal and said imaginary-frame position signal for producing a compressed second video signal represent-ing a compressed second picture, said compressed second picture being identical in size to said imaginary frame; and means respon-sive to said key signal for selectively combining said first video signal and said compressed second video signal, whereby said special effects are produced.

2. A tracking system in accordance with claim 1 wherein said position signal producing means includes means for removing electrical noise signals from said key signal.

3. A tracking system in accordance with claim 2 wherein said frame position signal includes two horizontal position signal values and two vertical position signal values, the tracking system further including means for detecting minimum and maximum horizontal position signal values and minimum and maximum vertical position signal values.

4. A tracking system in accordance with claim 3 wherein there is further included means for subtracting said minimum horizontal position signal value from said maximum horizontal position value and said minimum vertical position signal value from said maximum vertical position signal value, said horizontal difference being representative of said circumscribed frame length and said vertical difference being representative of said circumscribed frame height.

5. A tracking system in accordance with claim 4 further including means responsive to said horizontal and vertical differences for generating said imaginary frame position signal.