EP0592196A2

EP0592196A2 - Noise eliminating circuits

Info

Publication number: EP0592196A2
Application number: EP19930307916
Authority: EP
Inventors: Tetsujiro Int. Prop. Div. Legal And Int. Kondo; Yasuhiro Int. Prop. Div. Legal And Int Fujimori; Kunio Int. Prop. Div. Legal And Int. Kawaguchi
Original assignee: Sony Corp
Current assignee: Sony Corp
Priority date: 1992-10-08
Filing date: 1993-10-06
Publication date: 1994-04-13
Anticipated expiration: 2013-10-06
Also published as: KR940010805A; KR100295562B1; EP0592196A3; DE69328602D1; EP0592196B1; JPH06121192A; US5404178A; DE69328602T2

Abstract

A noise eliminating circuit comprising a noise eliminating filter for moving image part to be used when an input image is of the moving image part, a noise eliminating filter for stationary image part to be used when the input image is of the station image part, these filters being adaptively selected between according to a motion of the input image, a detecting circuit for detecting the motion of the input image, and a selecting circuit for selecting between outputs of these filters based on an output of the detecting circuit, wherein the noise eliminating circuit for stationary image part outputs an average input image of number of stationary frames n to be entered. The constitution effectively suppresses a noise introduced in a moving image or a stationary image, generating an output signal with a good picture quality. Further, if outputs of the filters are adaptively mixed according to the motion of the input signal, a picture quality at a transition point between the moving image and the stationary image is substantially improved.

Description

This invention relates to noise eliminating circuits such as may be used for preprocessing immediately before encoding an input signal.
It is known that an apparatus for compressing and decompressing an input image signal often has noise eliminating means immediately in front of an encoder provided in a data compression system of the apparatus. Of noise eliminating circuits used for the noise eliminating means, a noise eliminating circuit based on three-dimensional processing is known.
Fig. 10 shows a block diagram of an example of the above-mentioned noise eliminating circuit. In the figure, a terminal 11 is supplied with a digitally converted input signal (image signal) for example. The input signal is supplied to a noise eliminating filter for stationary image part 12 if an input image based on pixels constituting the input signal is the stationary image part and a noise eliminating filter for moving image part 13 if the input image is the moving image part. The filters 12 and 13 perform noise elimination processing on the input signal according to contents of the input image. Output from each of the filters is sent to a switch 20.
The input image is also supplied to a motion detector 30 which detects a motion of the input image on a pixel basis. The switch 20 is adaptively operated by motion control output according to the detected motion of the input image. Thus, a pixel in the stationary image part is noise-eliminated by a time filter which performs filter processing relative to time and a dynamic, moving pixel is noise-eliminated through a spatial filter.
Thus, performing noise elimination by adaptively selecting between the filters according to a movement of an input image achieves noise elimination without degrading a picture quality of the input image.
The noise eliminating filter for stationary image 12 is a simple mean-value filter having a frame memory 15. In this filter, a memory output value from a preceding frame is multiplied by k (0 < k < 1) in a coefficient multiplier 16 and a resulting value is added in an adder 18 to a value obtained by multiplying an input image of a current frame supplied to the input terminal 11 by (1 - k) in a coefficient multiplier 17. A resultant value provides a filter output.
Fig. 11 describes an operation of the noise eliminating filter 12. If a random noise indicated by a dot is introduced in a particular pixel of each frame and k = 0.5, then noise levels a, b, c and d get smaller sequentially by the second power due to a filter effect in time of the noise eliminating filter 12.
The noise eliminating filter for moving image part 13 is an intermediate-value filter such as a median filter, a type of a spatial filter. The intermediate-value filter uses a level of a pixel which is intermediate between levels of two pixels before and after a pixel in attention as a level of the pixel in attention. Fig. 12 describes an operation of this filter.
For an input image variation like a curve P, if noise n is introduced in pixel b in attention (an intrinsic level of the pixel is also b) for example, an intermediate value among a, n and c is selected; that is, c is selected. Since, in a next pixel, n is greater than d, the noise n is selected for the first time. Since this noise level is lower than d, even if the noise n is selected, it does not extremely stand out from pixels around it and therefore is not prominent so much.
In this connection, a normal output without filtering results in a pixel level of Fig. 12 and the noise n is selected when the pixel level is low, making the noise level stand out and the selected noise n prominent.
Fig. 13 describes detection of a motion of of an input image. In the figure, a difference between each of 3 x 3 pixels around pixel e in attention in a current frame (nth frame) and each of the same pixel locations in a preceding frame (n - 1 frame) is obtained (a - a', b - b', ... i - i'). Then sum S of the differences is obtained.
A value of the sum S is compared with a predetermined threshold value. If the sum S is greater than the threshold value, a pixel in attention at the time is determined to be a moving pixel (moving image part); if the sum S is smaller than the threshold value, the pixel in attention is determined to be a stationary pixel (stationary image part).
However, since a noise eliminating circuit having the above-mentioned constitution uses a time filter (mean-value filter), an averaged output remains to an end.
For example, if a noise (of noise level a) gets in a pixel 1 of the nth frame as shown in Fig. 11, the noise component will not disappear several frames later. Effects of the noise will remain for long, accompanying an after-image accordingly.
Besides, since an influence to be given by a noise to an immediately succeeding frame is as large as 1/2, a picture quality in transition from a moving image part to a stationary image part is not improved effectively.
According to one aspect of the invention there is provided a noise eliminating circuit wherein a noise eliminating filter for moving image part to be used when an input image is the moving image part and a noise eliminating filter for stationary image part to be used when the input image is the stationary image part are adaptively switched between according to a motion of the input image; based on an output from a detector for detecting the motion of the input image, one of outputs of the above-mentioned noise eliminating filters is selected; and the above-mentioned noise eliminating filter for stationary image part outputs an input image obtained by averaging the number of input stationary continuous frames n.
According to another aspect of the invention there is provided a noise eliminating circuit wherein a noise eliminating filter for moving image part to be used when an input image is the moving image part and a noise eliminating filter for stationary image part to be used when the input image is the stationary image part are adaptively selected between according to a motion of the input image; based on an output of a detector for detecting the motion of the input image, outputs of the above-mentioned noise eliminating filters are adaptively mixed to be output; and the above-mentioned noise eliminating filter for stationary image part outputs average input image information determined by the number of continuous stationary frames n in an input stationary pixel.
Respective further aspects of the invention are set forth in claims 1, 2, 5 and 6.
In a preferred embodiment of the invention shown in Fig. 1, based on motion information given by a motion detector 30, a stationary counter 50 calculates the number of continuous stationary frames n in a stationary pixel (refer to Fig. 2) and, weighting factors l/n and (n-l)/n from the number of continuous stationary frames n. These weighting factors are used when the input image is the stationary image part.
That is, a frame memory 41 and multiplier 42 provided in the noise eliminating filter for stationary image part 40 calculate a final output signal of a previous frame on which noise averaging processing in terms of time, the processing to be determined by the number of continuous stationary frames n at a stationary pixel has been performed. A multiplier 43 weights an input signal of a current frame by n. Output signals of the previous and current frames are added together to be used for an output of the noise eliminating filter 40.
Thus, since n = 1 at the transition point from the moving image part to the stationary image part, only a signal of the current frame is output from an adder 44, resulting in a filter output having no after-image. Likewise, since n = 1 also at a transition point from the stationary image part to the moving image part, only the signal of the current frame is output from the adder 44. Consequently, a picture quality at a moving part of an image is not degraded.
In other cases than mentioned above, the output signal of the previous frame may be weighted by n to be output, so that an output signal with a noise suppressed is obtained.
Embodiments of the present invention provide a noise eliminating circuit for effectively improving picture quality, especially the picture quality at a transition point between images.
Embodiments of the invention will now be described, by way of example, with reference to the accompanying drawings in which:

Fig. 1 is a block diagram illustrating a noise eliminating circuit practiced as a preferred embodiment of the invention;
Fig. 2 shows an example of processing of a median filter used as a spatial filter;
Fig. 3 is a diagram describing the number of continuous stationary frames at particular pixels;
Fig. 4 is a block diagram illustrating a motion detector;
Fig. 5 is a diagram describing an example of a Laplacian filter for edge detection;
Fig. 6 is a diagram illustrating a variation to the embodiment of Fig. 1;
Fig. 7 is a diagram illustrating another variation to the embodiment of Fig. 1;
Fig. 8 is a diagram illustrating a variation to the embodiment of Fig. 7;
Fig. 9 is a diagram illustrating an embodiment of the invention wherein filter outputs are mixed rather than selected between;
Fig. 10 is a diagram illustrating a previously proposed noise eliminating circuit;
Fig. 11 is a diagram illustrating an operation of a filter for stationary image part;
Fig. 12 is a diagram illustrating an operation of a filter for moving image part ; and
Fig. 13 is a diagram describing motion detection.

A noise eliminating circuit practiced as a preferred embodiment of this invention and as applied to a data compression system of a CODEC (Coder/Decoder) apparatus will be described in detail with reference to drawings.
Fig. 1 shows a preferred embodiment of a noise eliminating circuit according to the invention. An input signal entered in a terminal 11 is supplied to the noise eliminating filter for moving image part 13 and the noise eliminating filter for stationary image part 40 at the same time. These filters 13 and 40 are switched between by a switch 20 and an output of a selected filter is sent to an output terminal 12.
For the noise eliminating filter 13, a conventional filter such as a median filter, one of spatial filters, may be used. Fig. 2 shows an example of processing of the median filter. In this example, noises n1 and n2 are introduced in the input signal (with a waveform p).
Reference numeral 30 indicates a detector for detecting a motion of the input image. An embodiment of this detector will be described later. An output of this detector is sent to the switch 20 and a stationary counter 50 by which the number of continuous stationary frames n for a stationary frame is calculated.
Fig. 3 describes the number of continuous stationary frames n. As shown in the figure, given M being a moving pixel and S being a stationary pixel, n is always 1 if a pixel in attention is the moving pixel; n = 1 also at a transition point from a moving image part to a stationary image part and vice versa. In the stationary image part, n is sequentially accumulated by the number of stationary frames. Fig. 3 indicates a n value obtained when the pixel in attention moves over three frames and following three frames are stationary.
The stationary counter 50 calculates weighting factors l/n and (n - l)/n from this number of stationary frames n and sends the weighting factors to the filter for stationary image part 40.
Referring to Fig. 1, the filter for stationary image part 40 has a frame memory 41 for averaging. An output from the frame memory 41 (an output signal with the noise in the previous frame suppressed) is supplied to a first multiplier 42 to be multiplied by the weighting factor (n - l)/n. The input signal of the current frame supplied to the terminal 11 is multiplied by the weighting factor l/n in a second multiplier 43. These two multiplication outputs are added together in an adder 44 to be used as a final output of the filter 40. The output of the adder 44 is also sent to the frame memory 41. Since the input to the frame memory 41 is: ${\bar{X}}_{i} (k)= \frac{1}{n} X_{i} (k) + \frac{n-1}{n} {\bar{X}}_{i} (k-1)$

where, X_i(k) = input data (k frame) and
$\bar{X}$
_i(k) = input to the memory 41,
output signal X̂_i to be obtained at the terminal 21 in the stationary image part is: ${\hat{X}}_{i} (k) = {\bar{X}}_{i} (k)$

where, X̂_i(k) indicates the output signal with noise suppressed.
For example, in the moving image part, the noise eliminating filter 13 is selected, so that output signal X̂(k) at the time is: ${\hat{X}}_{i} {(k) = med [X}_{i-1} {(k), X}_{i} {(k), X}_{i+1} (k)]$

where,

med[ ] :: median filter processing;
X_i(k) :: input data at current point of time;
X_i-1(k) :: input data immediately before; and
X_i+1(k) :: input data immediately after.

As shown in Fig. 2, since the numbers of frames to be averaged by the number of continuous frames n are different, a noise elimination effect gets higher as the value n gets greater. Since n = 1 at such a point as the transition point from the moving image part to the stationary image part, the output of the first multiplier 42 becomes zero to cause the input signal itself to be output from the filter 40, producing no after-image.
Since the embodiment is cyclically constituted to enter the output of the adder 44 into the frame memory 41, image information of the moving image part is cyclically averaged when the input image changes from moving to stationary. Consequently, as time passes, an influence of the moving image on the stationary image gets smaller, increasingly showing the noise elimination effect.
Fig. 4 shows the embodiment of the motion detector 30. In motion detection in the figure, threshold values for motion detection are adaptively varied between an edge potion and a flat portion of an image. Consequently, the input signal is supplied to an edge detector 31. In this embodiment, a Laplacian filter is used for edge detection. This Laplacian filter performs edge detection in blocks of 3 x 3 pixels including the pixel in attention set to a factor shown in Fig. 5. Corresponding input pixels are multiplied with Laplacian coefficients (-1 or 8) respectively. A sum of the multiplications is divided by 8 to provide a detection value (activity) L.
Referring to a 3 x 3 block (the nth frame) shown in Fig. 13, if pixels a through i are input levels, then the activity L is: $L = 1/8 {8e - (a + b + c + d + f + g + h + i)}$
This activity L is compared with a predetermined threshold value β.
If L ≧ β, then the edge portion; if L < β, then the flat portion. Based on a comparison output CA obtained by this determination, a threshold value α (in this embodiment, α1 or α2 where α1 > α2) for the motion detector 33 is selected.
In this embodiment, a coefficient ROM 32 is provided as shown in Fig. 4. This ROM stores the two threshold values α1 and α2. When the edge portion is detected for example, α1, which is greater, is selected.
Based on the threshold value α selected by the comparison output CA, a motion of the input signal is determined. That is, the motion detector 33 performs following determination processing based on the two threshold values α1 and α2:

(1) L < β (the flat portion)
S ≧ α1 ... the moving image part; and
S < α1 ... the stationary image part.
(2) L ≧ β (the edge portion)
S ≧ α2 ... the moving part; and
S < α2 ... the stationary part.

According to the case (1), it is more probable that the pixel is determined to be the stationary image part rather than the moving image part. Consequently, the filter for stationary image part 40 is selected. The filter for stationary image part 40 suppress a noise more than the filter for the moving image part 13 does, providing a higher noise elimination effect in the flat portion than the edge portion.
In the edge portion, the smaller α2 is used according to the case (2), so that it is more probable that the pixel is determined to be moving rather than stationary unlike the case (1). Consequently, the noise elimination effect in the edge portion gets somewhat lower. But, since it is the edge portion, noise introduction is not so conspicuous. Therefore, using the smaller threshold value causes little influence on noise elimination.
Rather, using the filter for stationary image part 40 in the edge portion blurs the image because the filter is a time filter. In the above-mentioned constitution, it is more probable that the pixel is determined to be moving in the edge portion, so that the noise eliminating filter for stationary image part 40 is selected less often, posing no risk of blurring the image.
Thus, adaptively changing the threshold values for motion detection based on the edge information enhances the noise elimination effect in the flat portion, mitigating a picture quality degradation in the edge portion. Values for the above-mentioned α1, α2 and β are suitably selected according to an application. Instead of the ROM 32, reference values of α1 and α2 may be selected. Any number of threshold values can be prepared.
Fig. 6 shows a variation to the embodiment of Fig. 1. In the embodiment of Fig. 1, the input to the frame memory 41 is the output itself of the adder 44. In the embodiment of Fig. 6, an output signal obtained at an output terminal 21 is used. Except it, the embodiment of Fig. 6 is the same as the embodiment of Fig. 1.
Referring to Fig. 6, the output signal at the output terminal 21 is an output signal with a noise in space processed by a noise eliminating filter for moving image part 13 or an output signal with a noise in time processed by a noise eliminating filter for stationary image part 40. Therefore, each of the outputs is an output of the previous frame with the noise sufficiently suppressed. The output appearing at the output terminal 21 is expressed in an equation as follows: ${\hat{X}}_{i} (k)= \frac{1}{n} X_{i} (k) + \frac{n-1}{n} {\hat{X}}_{i} (k-1)$
Therefore, in the filter 40 comprising a first multiplier 42, a second multiplier 43, an adder 44, and a frame memory 41, the above-mentioned output signal with the noise sufficiently suppressed is used as an input. An output signal of the previous frame weighted by the number of continuous stationary frames n is added to an input signal of the current frame weighted by the number of continuous stationary frames n. A result of the addition is used as a filter output.
Consequently, if a noise gets in the moving image part immediately before a point at which the moving image part becomes the stationary image part, the filter output with the noise suppressed by the filter 40 can be used as an output signal, resulting in a greater noise reducing effect at the above-mentioned changing point than that of the embodiment of Fig. 1.
Fig. 7 shows a second variation to the embodiment of Fig. 1. In the embodiment of Fig. 1, an input signal itself of the current frame is used as an input signal to the noise eliminating filter for moving image part 13. In the embodiment of Fig. 7, however, a filter output of a noise eliminating filter for stationary image part 40 is used as an input signal to a noise eliminating filter for moving image part 13.
As a result, the filter output on which the noise elimination processing in terms of time has been performed is entered in the noise eliminating filter for moving image part 13, so that the noise elimination effect in the moving part is greater than that obtained by entering the input signal of the current frame for the noise processing in space.
Fig. 8 shows a variation to the embodiment of the Fig. 7. In this variation, a switch 60 is provided before a noise eliminating filter for moving image part 13 to adaptively switch between the input signal of the current frame and an output signal of a previous frame according to a motion.
In the case of the moving image part, the input signal of the current frame is selected; in the case of the stationary image part, the output signal of the previous frame is selected. In the stationary image part, the noise elimination processing is performed based on the output signal of the previous frame with the noise sufficiently suppressed, providing an effective noise elimination. In addition, the an output of this noise elimination processing is supplied to a frame memory 41 for cyclic processing, totally providing a substantially significant noise suppressing effect.
In each of the above-mentioned embodiments, the filter output resulted from the noise elimination processing in time and the filter output resulted from the noise elimination processing in space are adaptively switched between according to the output of the motion detection.
Referring to Fig. 9, the outputs of both filters are adaptively mixed for use without selecting one of the outputs. To implement such a constitution, a mixer 70 is provided instead of the switch 20 as used in the above-mentioned embodiments. A mixing ratio in the mixer 70 is controlled according to a quantity of motion of an input image.
Let the quantity of motion be mq, a filter output when a median filter is used as a noise eliminating filter for moving image part 13 be dm(k), and an output of a filter 40 (an output of an adder 41) be ds(k), then a final output signal to appear at an output terminal 21 will be: ${dm(k) = med[X}_{i-1} {(k),X}_{i} {(k),X}_{i+1} (k)]$
$ds(k)= \frac{1}{n} X_{i} (k) + \frac{n-1}{n} ds(k-1)$
${\hat{X}}_{i} (k)= mq x dm(k) + (1-mq) x ds(k)$

where, 0 ≦ mq ≦ 1
Fk : an input image value at a unit pixel of 3 x 3 block in k frame;
i, j : coordinates of pixel; and
C : constant
Thus, adaptively controlling the mixing ratio of the outputs of both filters according to the motion of the input image improves a picture quality degradation at a transition point between the moving image part and the stationary image part.
The mixing coefficient mq, (1 - mq) is not linearly controlled according to the motion. Rather, it may be prepared in several types, one of which being adaptively selected.The above-mentioned embodiments can be combined with each other in any way. Some combinations provide the noise elimination in time and space more effectively than a single embodiment does. For motion detection, a single threshold value may be used as with the previously proposed constitution.
The embodiments of the invention described above provide a noise eliminating circuit wherein outputs of a filter in time and a filter in space are adaptively selected between according to a motion of an input signal.
This constitution effectively suppresses a noise introduced in a moving image or a stationary image, generating an output signal with a good picture quality.
Further, if outputs of the filters are adaptively mixed according to the motion of the input signal, a picture quality at a transition point between the moving image and the stationary image is substantially improved.
While preferred embodiments of the invention have been described, such description is for illustrative purpose only, and it is to be understood that changes and variations may be made without departing from the scope of the appended claims.

Claims

A noise eliminating circuit comprising:
a noise eliminating filter for moving image part to be used when an input image is of the moving image part and a noise eliminating filter for stationary image part to be used when the input image is of the station image part, said filters being adaptively selected between according to a motion of the input image;
a detecting circuit for detecting the motion of the input image; and
a selecting circuit for selecting between outputs of said filters based on an output of said detecting circuit;
wherein said noise eliminating circuit for stationary image part outputs an average input image of number of stationary frames n to be entered.
A noise eliminating circuit comprising:
a noise eliminating filter for moving image part to be used when an input image is of the moving image part and a filter for stationary image part to be used when the input image is of the stationary image part, said filters being adaptively selected between according to a motion of the input image;
a detecting circuit for detecting the motion of the input image; and
a mixing circuit for adaptively mixing outputs of said filters based on an output of said detecting circuit;
wherein said noise eliminating filter for stationary image part outputs averaged input image information determined by number of continuous stationary frames n in a stationary pixel to be entered.
A noise eliminating filter as defined in claim 1, further comprising:
first multiplying means for multiplying the input image by l/n;
second multiplying means for multiplying an output of a mean value memory by (n - 1)/n; and
an adding means for adding outputs of said first and second multiplying means;
wherein an output of said adding means is entered in said mean value memory.
A noise eliminating filter as defined in claim 2 wherein said detection output for adaptively mixing the outputs of said filters is generated with reference to a difference signal of an adjacent frame.
A noise eliminating circuit comprising:
a noise eliminating filter for moving image part to be used when an input image is of the moving image part and a filter for stationary image part to be used when the input image is of the stationary image part, said filters being adaptively selected between according to a motion of the input image;
a detecting circuit for detecting the motion of the input image; and
a selecting circuit for selecting between outputs of said filters based on an output of said detecting circuit;
wherein said noise eliminating filter for stationary image part outputs an average input image value of number of continuous stationary frames n in a stationary pixel to be entered.
A noise eliminating circuit comprising:
a noise eliminating filter for moving image part to be used when an input image is of the moving image part and a filter for stationary image part to be used when the input image is of the stationary image part, said filters being adaptively selected between according to a motion of the input image;
a detecting circuit for detecting the motion of the input image; and
a mixing circuit for adaptively mixing outputs of said filters based on an output of said detecting circuit;
wherein said noise eliminating filter for moving image part is selectively supplied with the output of said noise eliminating filter for stationary image part and said noise eliminating filter for stationary image part outputs averaged input image information determined by number of continuous stationary frames n in a stationary pixel to be entered.
A noise eliminating circuit as defined in claim 5, further comprising:
first multiplying means for multiplying the input image by l/n;
second multiplying means for multiplying an output of mean value memory by (n - 1)/n; and
an adding means for adding outputs of said first and second multiplying means;
wherein an output of said adding means is entered in said means value memory.