WO1999052275A1

WO1999052275A1 - Video signal processing

Info

Publication number: WO1999052275A1
Application number: PCT/GB1999/001043
Authority: WO
Inventors: Avigdor Steinberg; Liam Breathnach
Original assignee: Snell & Wilcox Limited
Priority date: 1998-04-03
Filing date: 1999-04-06
Publication date: 1999-10-14
Also published as: EP1068722A1; AU3339699A; GB9807296D0

Abstract

In a system for detecting scratches in video material originating from film material (,) values of amplitude derivatives are measured to provide local and background activity levels, and the difference between these values is formed. If the magnitude of the difference is positive and smaller than a given value (,) the value of the difference is accumulated vertically. The accumulated value is compared with a threshold to detect a scratch. Accumulation also takes place over fields. The polarity of the local activity level is carried forward to identify a detected scratch as dark or light for use in subsequent concealment processes.

Description

VIDEO SIGNAL PROCESSING

This invention relates to the processing of analogue and digital video signals and is especially concerned with the detection of visually disturbing artefacts such as film scratches.

There exists a very considerable amount of material in video archives and there are commercial imperatives in making as much as possible of this material available for broadcast or other distribution. Archive material tends, however, to suffer from a range of picture quality defects which are visually unacceptable when judged against current display standards. There exists therefore a real need for video archive restoration. A significant proportion of the material in video archives is film-originated and film scratches are a common defect.

Manual techniques exist for the correction of film scratches and certain other archive defects. It is not practicable nor economic, however, for entire archives to be restored manually. A degree of automation is essential. The volume of material requiring restoration demands that the bulk of defects are corrected automatically and at rates which are real-time or close to real-time. It is then possible for sufficient time to be devoted to the manual repair of the most heavily damaged sections. It will of course be understood that this invention would equally find use in processing which is not performed at, or close to, real-time, such as archive restoration with subsequent storing of the corrected material for later broadcasting. To enable automatic correction, a reliable method is required for the detection of film scratches and other linear picture defects. Once detected, the scratches can be repaired by a variety of well known techniques, for example by application of a horizontal median filter. The performance of scratch detection directly affects the overall performance of the scratch concealment process, as false detection will lead to unnecessary damage to the picture, whilst any significant undetected scratches will negate the principle of automation. - 2 -

It is accordingly an object of the present invention to provide an improved method of video processing for detecting the presence of a scratch.

Accordingly, in one aspect the present invention provides a method of video processing for detecting the presence of a scratch or other elongate picture artefact in a picture region, wherein the detection threshold depends on a background activity level in the picture region.

In a second aspect the present invention provides apparatus for processing video signals for detecting the presence of a scratch or other elongate picture artefact in a picture region, comprising means for analysing a background activity level in the picture region and for providing a detection threshold based on the analysis.

In a third aspect the present invention provides a method of video processing wherein the presence of a scratch or other elongate picture artefact in a picture region is detected through the steps of taking a measure of picture amplitude derivative over the region to provide a background activity level; identifying local derivatives of approximately equal magnitude and opposite polarity to provide a local activity level; subtracting said background activity level from said local activity level and accumulating positive results.

Advantageously, only positive results beneath a selected threshold are accumulated.

The present invention will now be described by way of example with reference to the accompanying drawings, in which:-

Figure 1 is a block diagram of scratch detecting video signal processing apparatus according to this invention; and

Figure 2 is a block diagram showing in greater detail certain components of the apparatus shown in Figure 1 .

In a preferred form of the invention, the following criteria for film - 3 - scratch detection are employed:

(i) Scratches are sharper than normal (background) film video, (ii) Scratches are usually narrow and isolated, (iii) Scratches are highly correlated vertically. (iv) Scratches do not necessarily come close to extreme (black or white) levels, (v) Scratches do not necessarily come to the full picture height, (vi) Scratches, while possibly persisting for several fields, may vary in contrast. (vii) For a particular film scratches may always be light, or always be dark, depending on the film processing and whether a positive or negative film was run in the telecine. As regards point (vii) above, it should be noted that usually either light scratches only or dark scratches only are present in one particular item of film material. However, film material does exist in which both types can be found.

Referring now to Figure 1 , an input "scratched" video signal 1 , which may be of any monochrome or colour standard such as analogue, serial digital, parallel digital, standard or high definition, is passed to both a local picture activity level measurement block 20 and to a background picture activity level measurement block 21 . (Note that an analogue input signal would need to be digitized prior to processing in the local and background picture activity level measurement blocks 20 and 21 . This digitization equipment is not shown in Fig. 1 ) . In this block, measures are taken of both partial single-sided (i.e. left and right) derivatives. In the presence of a scratch, these are of opposite polarity. For this processing, the magnitude of the smallest of these derivatives is then used as a measure of local activity 2. The signs 5 of either the left 1 6 or the right derivatives are also carried forward for use in certain subsequent processes, for example in scratch concealment processes. The sign 5 of the left (or right) derivative indicates whether a scratch is dark or light. Note that it does not matter whether the left or right derivative is carried - 4 - forward, as long as this is done consistently. Note also that different delays may be selected between the left and right derivatives. It has been found that for many scratches a 3 pixel delay (as indicated in Fig. 2) provides satisfactory results. Other values are possible. In particular if a certain width of scratches prevails a delay of a particular duration can be selected.

A measure of the background activity 4 is assessed by processing a number of left and right derivatives at specific horizontal distances (numbers clock intervals) from the central point of a suspect scratch and finding the largest one.

On a per pixel basis, the background activity value 4 is subtracted from the local activity value 2 to give a difference value as shown in Fig. 2. If this produces a negative value, the output relative activity difference is set to zero. This distinguishes scratches from important periodic picture textures and test signals, e.g. line sweeps.

The relative activity difference 6 is compared with a maximum allowed threshold 7, which is derived from a control parameter, defining the maximum expected scratch contrast. Only values less than this threshold 7 pass through. All other values are set to zero. The threshold 7 is increased (allowing a wider range of relative activity differences 6 to be passed) when a scratch has been detected in the preceding field . The resulting signal 8 will be referred to as y grad, which accordingly equals the relative activity difference 6 if the relative activity difference 6 is less than this threshold 7. If it is larger, then y grad 8 equals zero. The use of a maximum threshold 7 prevents the false detection of high contrast vertical lines, such as test signal pulses, and the modification of the threshold 7 in response to previously detected scratches alleviates the detection of medium contrast scratches.

The y grad value 8 for each pixel from which it was derived is accumulated vertically in a line-store 28. The store 28 has one storage location for each pixel on the line (typically 720 locations for standard definition pictures in the case of processing based on luminance only; a - 5 - different number of locations would be provided for other formats, or in the case of processing based on chrominance) and the magnitude of each y grad value 8 is added (i.e. irrespective of the sign 5) to the corresponding stored value and the result returned to the same location in the store. The accumulation process is arranged to limit the magnitude of the data to avoid overflow (so that the stored values cannot be greater than a defined maximum value) .

The sign 5 (derived from the local activity signal) is retained, as mentioned above. Note that the sign 5 would be expected to remain the same over the whole length of the scratch.

This process exploits the high vertical and temporal correlation of scratches and allows scratches to be tracked reliably.

The vertically-accumulated values 14 are spread horizontally by comparing each value with its immediate neighbours and, if it is greater than both, assigning the higher value to the two outer pixels.

This compensates for jitter, bending or movement associated with the scratch, on a field-by-field and a pixel-by-pixel basis.

At regular intervals, a preset number of frames apart, the spread, accumulated data 15 is compared (the comparison has hysteresis as will be explained below), on a pixel-by-pixel basis, with a threshold 1 1 which is derived from a control parameter. The results of the comparisons are stored as scratch flags 1 2 along with the accumulated data values.

When an accumulated data value 1 5 exceeds the threshold 1 1 , the presence of a scratch is detected at that horizontal position, the appropriate flag 12 is set, and the stored data value 10 is reduced by a constant factor (e.g. 25%) and returned to the store 28.

When an accumulated data value 1 5 does not exceed the threshold 1 1 the appropriate flag 1 2 is cleared. However, when a weak scratch (whose accumulated data value 15 for one isolated field does not exceed the threshold) persists for some fields the associated accumulated data value 15 may, due to accumulation over several fields, exceed the threshold 1 1 eventually. Thus also weak scratches are detected. In other - 6 - words, the level of confidence in a correct assessment as to whether a scratch is present is increased by accumulation over several fields. Logic may be provided to set the scratch flag for such a weak scratch only "as of the moment when the threshold is exceeded", or alternatively also retrospectively for a preceding period. The duration of such a preceding period may be determined by the exceeding of a second, lower threshold. Hysteresis is provided by lowering the threshold when the relevant flag 1 2 is already set and this prevents the scratch flag from "flickering". The sign 1 3 of the accumulated data indicates whether the scratch is light or dark and logic can be provided which prevents scratch flags being set when the sign 5 has the value corresponding to "unexpected" polarity of scratch.

Every data word (stored in the line store 28) contains the accumulated history 1 5 of a particular column of the picture over several fields. The time constant of the recursive filtering improves the noise immunity of the detection process. Further improvement may be obtained by inhibiting the detection of scratches at the edge of the picture. Similarly window(s) can be applied in which scratch detection is allowed, or alternatively forbidden. The resulting scratch flag 1 2 can now be fed downstream to enable a scratch repair (concealment) filter of any known type. This filter is not the subject of the current invention.

The next described step of the scratch detection process will not be required in many situations. It serves only for optimisation of the control parameters to be used at the other steps.

In order to optimise the confidence of detecting scratches (as opposed to mistaking sharp transitions for scratches), this method analyses the picture for the total count of different types of details within different magnitude bands of activity. The result is an activity matrix, e.g. 3 x 3 matrix of activity counts for soft, sharp and super-sharp pulses within the contrast ranges of 0%-25%, 25%-50%, and 50%-100%. Then the matrix values are checked against predefined thresholds and the - 7 - picture under test is assigned to one of the predefined groups. The result is group identification number. From this analysis, the optimal filter for the particular source of video can be selected. This analysis is also used to modify the threshold values used in the processing which follows. If this advanced technique is not used for economy reasons then the setting of optimal control parameters should be done manually by a skilled operator. In a further form of this invention there is provided an improved scratch detector which will also cope with horizontal scratches, using vertical banding and "line-suspect-counters" . This is the same method as described above, but "rotated through 90 degrees" .

This invention has been described so far taking the example of the luminance content of the video signal. It is possible to enhance the performance of this scratch detector by considering the chrominance using similar processing. It will be understood that this invention has been described by way of example and that a variety of modifications are possible without departing from the scope of the invention. Whilst reference has been made repeatedly to scratches, the invention will find application also in the detection of other elongate picture artefacts. Each feature disclosed in this specification (which term includes the claims) and/or shown in the drawings may be incorporated in the invention independently of other disclosed and/or illustrated features.

Statements in this specification of the "objects of the invention" relate to preferred embodiments of the invention, but not necessarily to all embodiments of the invention falling within the claims.

Claims

- 8 - CLAIMS

1 . A method of video processing for detecting the presence of a scratch or other elongate picture artefact in a picture region, wherein the detection threshold depends on a background activity level in the picture region.

2. The method according to Claim 1 , wherein a local activity level is provided for comparison with the threshold, the local activity level and the background activity level being obtained by means of equivalent processes, the process for obtaining the local activity level being carried out over a sub-region of said region.

3. The method according to Claim 1 or Claim 2, wherein the background activity level is provided by taking a measure of picture amplitude derivative over the picture region.

4. The method according to Claim 2 or Claim 3 as dependent on Claim 2, wherein the step of providing a local activity level comprises identifying local picture amplitude derivatives of opposite polarity.

5. The method according to Claim 2 or any claim as dependent on Claim 2, wherein a relative activity difference is provided by subtracting the local activity level from the background activity level.

6. The method according to Claim 5, wherein those values of the relative activity difference which exceed a given value are disregarded.

7. The method according to Claim 6, wherein said given value depends on whether a said artefact has been detected in a previous field.

8. The method according to any of Claims 5 to 7, wherein values of - 9 - the relative activity level are accumulated in the direction in which the artefact is assumed to extend so as to provide accumulated values.

9. The method according to Claim 8, wherein values of the relative activity level are accumulated vertically to provide said accumulated values.

10. The method according to Claim 8 or Claim 9, wherein the accumulated values are spread in a direction perpendicular to the direction in which the artefact is assumed to extend.

1 1 . The method according to any of Claims 8 to 1 0, wherein the accumulated values are compared with a further threshold.

1 2. The method according to Claim 1 1 , wherein the further threshold depends on whether a said artefact has been detected at or near a corresponding position in a previous field.

1 3. The method according to any of Claims 8 to 1 2, wherein, when the polarity associated with the accumulated values changes, a said artefact is judged not to be present.

14. The method according to any of Claims 8 to 1 2, wherein, when the polarity associated with the accumulated values is of a predetermined sign, a said artefact is judged not to be present.

15. The method according to any preceding claim, carried out at or close to real-time.

1 6. The method according to Claim 2 or any claim as dependent thereon, wherein the polarity associated with the accumulated values is carried forward for use in a subsequent artefact concealment process. - 10 -

1 7. A method of concealing a scratch or other elongate picture artefact in a picture region, comprising the method according to any preceding claim and further the step of repairing all detected artefacts.

1 8. Apparatus for processing video signals for detecting the presence of a scratch or other elongate picture artefact in a picture region, comprising means for analysing a background activity level in the picture region and for providing a detection threshold based on the analysis.

1 9. Apparatus according to Claim 1 8, further comprising means for analysing a local activity level in a sub-region of said region and for providing a local activity level based on the analysis for comparison with the threshold.

20. Archive restoration equipment comprising the apparatus according to Claim 1 8 or Claim 1 9, and further means for repairing all artefacts which the apparatus is adapted to detect.

21 . A method of video processing wherein the presence of a scratch or other elongate picture artefact in a picture region is detected through the steps of taking a measure of picture amplitude derivative over the region to provide a background activity level; identifying local derivatives of approximately equal magnitude and opposite polarity to provide a local activity level; subtracting said background activity level from said local activity level and accumulating positive results.

22. A method according to Claim 21 in which only positive results beneath a selected threshold are accumulated.