WO2010037782A1

WO2010037782A1 - Image processing to decrease the quality of images captured illegally

Info

Publication number: WO2010037782A1
Application number: PCT/EP2009/062693
Authority: WO
Inventors: Pascal Bourdon; Sylvain Thiebaud; Jean-Ronan Vigouroux
Original assignee: Thomson Licensing
Priority date: 2008-10-03
Filing date: 2009-09-30
Publication date: 2010-04-08

Abstract

The present invention lies in the field of protecting video contents and relates more particularly to a method of processing a source image so as to generate a scrambling pattern on illegal copies captured by a camcorder. According to the invention, the scrambling pattern is generated on the basis of a deformed image of the source image to be protected. For this purpose, the method comprises the following steps:- generating a deformed image of said source image (10; 40), and - processing the source image (11, 12; 41, 42) so as to modify in a global manner not perceptible to the eye, the colour of pixels of the source image as a function of the deformed source image

Description

PROCESSING IMAGES TO DECREASE THE QUALITY OF ILLEGALLY CAPTURED IMAGES

FIELD OF THE INVENTION

The present invention is in the field of video content protection. The invention relates more particularly to a source image processing method intended to fight against copying by a camcorder or a camera.

BACKGROUND OF THE INVENTION

Numerous so-called anti-camcorder techniques, based on the temporal and / or spatial modulation of the pixel color of the image, on the emission of an infrared component or on metamerism have been developed to deteriorate the quality of the images.

15 captured fraudulently by a camcorder. All these techniques consist in modifying the content of the images to be displayed in a visible manner on the illegal copy captured by the camcorder while being generally invisible to a spectator looking at the images. The contents of the images are changed to show one or more

20 reasons for scrambling the illegal copy, for example a warning message informing the viewer that the copy is illegal or information identifying the movie theater or the date of capture of the illegal copy. The scramble pattern appears superimposed on the illegal copy.

Although the quality of the illegal copy is degraded by the insertion of scrambling patterns into the images, some "illegal" viewers are not bothered by this degradation and will continue to watch such copies as long as the quality of the images behind the pattern overprinting remains acceptable.

US2008 / 095363 relates to the distortion of source images generally not perceptible to the eye but largely perceptible by a camcorder. In the image processing proposed in this document, the image is not distorted but distorted in the sense that the color of certain pixels of the image is modified in such a way that

SUBSTITUTE SHEET (RULE 26) overall not perceptible to the eye. The pixels whose color is changed then form a scrambling pattern.

Summary of the invention

An object of the invention is to provide an image processing to increase the embarrassment of a viewer looking at an illegal copy to possibly dissuade him from looking at it.

According to the invention, the scrambling pattern is generated from a distorted image of the source image to be processed.

For this purpose, the present invention relates to a method for processing a source image comprising the following steps:

generating a distorted image by spatial deformation of said source image, and

processing of the source image to modify, in a manner generally not perceptible by the eye, the color of pixels of the source image as a function of the distorted source image.

According to a particular characteristic of the invention, the spatial distortion of the source image is generated by vertically and / or horizontally shifting the pixels of the source image, the offset applied to the pixels being variable. Preferably, the offset applied to a pixel depends on the position of the pixel.

According to a first embodiment, the step of processing the source image comprises, prior to said modifying step, a step of selecting the pixels of the source image whose color is to be modified.

Advantageously, the selection step comprises an edge detection step in the deformed source image, the selected pixels of the source image for modification then corresponding to the contours detected in the distorted source image. In this first embodiment, the color can be modified according to several treatments. According to a first treatment based on metamerism, the color of each pixel of the source image comprises at least 4 primary color components and the step of modification of the pixel color of the source image comprises a step of replacing said color with a metameric color.

According to a second processing based on the temporal modulation of the color, the step of modifying the source image comprises a step of decomposing the source image into n different component images for the pixels corresponding to the selected pixels of the source image whereby, a color vector being associated with each of the pixels of said source image and component images in a predetermined color space, the color vector associated with each pixel of said source image is equal to the resultant divided by n of the vectors of color associated with the n pixels of the component images corresponding to said pixel of the source image.

According to a third processing which may optionally be combined with the other two, the spectral composition of the pixels of the source image is modified by adding an infrared component to the pixels of the source image corresponding to the selected pixels of the source image.

Thus, according to this first embodiment, the image that appears on the illegal copy includes both the content of the source image and the contours of a distorted source image of the source image. This illegal copy is very unpleasant to watch because it forces the viewer to accommodate permanently to overcome these parasitic contours.

According to a second embodiment of the invention, the pixels whose color is modified are not limited to the edge pixels of the distorted source image. In this embodiment, the processing of the source image comprises, prior to the modification step, a step of calculating the color difference between each pixel of the source image and the same pixel of the deformed image, and the color change of each pixel of the source image is a function of the calculated difference. According to a particular mode, the color of each pixel of the source image comprising at least 4 primary color components, the pixel color modification step of the source image comprises a step of replacing said color with a metameric color so that at least one of the color components of each modified pixel is proportional to said color difference between the same pixel of the source image and the same pixel of the distorted image.

Thus, according to this second embodiment, the image that appears on the illegal copy includes both the source image and a distorted image of the source image. This illegal copy is very unpleasant to watch because it forces the viewer to constantly accommodate to get rid of the distorted source image.

Brief description of the figures

The invention will be better understood, and other objects, details, features and advantages will become more clearly apparent from the following detailed explanatory description of two presently preferred particular embodiments of the invention, with reference to the accompanying drawings, among which :

FIG. 1 is a flowchart of the steps of the method of the invention according to a first embodiment; FIG. 2A represents a source image to be processed and FIGS. 2B to 2D illustrate the images obtained at the end of the steps of the method of FIG. 1 when it is applied to the source image of FIG. 2A;

- Figure 3 schematically shows a system adapted to implement the first embodiment;

FIG. 4 is a flowchart of the steps of the method of the invention according to a second embodiment;

FIG. 5A represents a source image to be processed and FIGS. 5B to 5D illustrate the images obtained at the end of the steps of the method of FIG. 4 when it is applied to the source image of FIG. 5A; and,

- Figure 6 schematically shows a system adapted to implement the second embodiment. Detailed Description of Preferred Embodiments

In the remainder of the description, I designates a source image and I (x, y) designates a spatial coordinate pixel (x, y) in image I, the x and y coordinates being normalized and between -1 and 1. , where x = 2 (p / P) -l with p representing the column number of the pixel considered and

Where P is the total number of pixel columns in the image and where y = 2 (q / Q) -l where q is the line number of the pixel and Q is the total number of pixel lines in the image considered. Note also that in the remainder of this document, any image function of the type f (x, y) will have a saturation effect at the edges, namely for x <-l and -l≤y≤l, then f (x , y) = f (-l, y), for x> l and -l≤y≤l, then f (x, y) = f (l, y), for -l≤x≤l and y <- l then f (x, y) = f (x, -l), and for -l≤x≤l and y> l then f (x, y) = f (x, l).

The source image has three primary color components, namely a red component, a green component and a blue component. Each pixel I (x, y) of the source image can be represented by the following vector: l {x, y) = [R {x, y \ V {x, y \ B {x, y)] ^τ where - R (x, y) represents the red component,

V (x, y) represents the green component, and

- B (x, y) represents the blue component.

In a general manner, the method of the invention comprises a step of generating a distorted image of the source image followed by a step of modifying the color of pixels of the source image or the spectral composition of the image. the light emitted by pixels of the source image as a function of the deformed source image, the color of the pixels of the modified source image remaining unchanged for the human eye relative to the initial source image.

FIG. 1 illustrates the steps of the method of the invention according to a first embodiment. This method comprises a step 10 of deformation of the source image in which a vertical phase shift and / or horizontal is applied to the pixels of the source image, the same phase shift being applied to the three color components of the source image, a contour detection step 11 in the distorted source image and a processing step 12. source image for modifying the color or the spectral composition of the pixels of the source image corresponding to the contours detected in the distorted source image.

The step 10 of deformation of the source image consists in shifting vertically and / or horizontally all or part of the pixels of the source image, the offset being variable according to the pixel in question. A phase θ having a horizontal component θ _x and a vertical component θ _y in the two-dimensional space of the image is applied to the pixels I (x, y) of the source image.

The pixels of the deformed source image can then be represented in the following manner: l (x + θ _x , y + θ _y ). Advantageously, so that the deformed source image is not simply a globally offset image of the source image, the phase θ _x and / or θ _y varies (nt) as a function of the position (x, y) of the pixel.

For example, to apply deformation along the vertical axis, θ _y (x, y) is varied while keeping θ _x (x, y) zero. By way of example, it is possible to vary the vertical phase θ _y according to a sinusoidal function. In this case, θ _y (x, y) can be expressed in the following way: θ _y (x, y) = m.cos (ϋ5 ₀ y) where - me [θ, l] denotes a modulation index; and

- 03 ₀ is a pulse value (cycles per pixel).

The deformed source image then comprises vertical deformation waves as illustrated in FIG. 2B compared with FIG. 2A. Figure 2A shows a source image showing the face of a woman wearing a hat. FIG. 2B represents the same image but deformed according to a vertical phase θ _y (x, y) = m.cos (ϋ3 ₀ y) and a horizontal phase θ _x zero. It should be noted that any type of function with slow and continuous evolution can be used instead of the cosine function. For example, a triangle function or a pulse width modulation (PWM) type function may be employed. It is also possible to vary the strain as a function of time by replacing for example the function m.cos (ϋ3 ₀ y) by the function m.cos (ϋ3 ₀ t) or m.cos (ϋ3 ₀ y • t).

According to the invention, the source image thus deformed is then used to modify the color of certain pixels of the source image and show, in the image captured by a camcorder, a scrambling pattern.

In the first embodiment illustrated in FIG. 1, the interference pattern generated is based on the contours detected in the distorted source image. To do this, step 11 of contour detection is carried out in the distorted source image. This detection is for example carried out by filtering the deformed source image using a Canny filter or a Canny-Deriche filter known per se. This filtering operation is combined with an additional operation of thresholding or binarization of the values obtained to retain only the main contours of the image. At the end of this step, a monochrome image is obtained, the clear points of the image corresponding to the contours of the distorted source image. According to the invention, the pixels of the source image corresponding to the detected contours constitute the pixels of the source image whose color or spectral composition is to be modified. These pixels, after processing, form the scrambling pattern that appears in the illegal copy.

Figure 2C shows the contour image of the distorted source image of Figure 2B. One can see in particular the contours of the face of the woman and her hat represented by clear dots.

The next step of the method, referenced 12 in FIG. 1, consists in modifying, in a manner generally invisible to the human eye, the color of the pixels of the source image which correspond spatially to the contours detected in the distorted source image or the composition spectral light emitted by these same pixels. For the sake of simplicity, the pixels of the source image spatially corresponding to the contours detected in the distorted source image are called pattern pixels in the remainder of the description. This modification step depends on the display or projection device that will be used to display the source image.

If the display device comprises infrared signal transmission means, the modification of the image then consists in adding an infrared component, invisible to the human eye, in the source image. When displaying the source image, a non-zero infrared component is superimposed on the other color components of the source image (red, green and blue in the case of a 3 color component image) for the considered pixels. . If the displayed image is captured by a camcorder, a red or red-dominant pattern, representing the contours of the distorted source image, appears on the captured copy because the camcorder does not filter or partially filter this infrared component.

If the display device comprises means for displaying the image with four primary colors and possibly, in the case of a primary three-color source image, means for previously converting the source image to three basic primary colors into a primary four-color image, the processing of the source image may consist of displaying the pattern pixels with the four primary colors of the device and the other pixels of the source image with the three primary primary colors, the component of the fourth primary color is then always zero for these other pixels. Thus, for the same color to be reproduced in the pattern and outside the pattern, two different primary color combinations are used which are visually equivalent for the human eye but have different spectral compositions. The colors corresponding to these combinations are called metamers. This use of metamerism to scramble the image captured by a camcorder is described in more detail in US Patent Application Publication Number US2004 / 0081318. Another possible treatment applied to the source image to change the color of the pixels in the image captured by a video camera is to temporally modulate the color of the pattern pixels of the source image. It consists in decomposing the source image into n component images which are different from each other for the part corresponding to the pixels of pattern and to define, for each of these pixels of pattern, a color vector which is different in each of the component images. The color vector of the pixel considered in the source image is equal to the sum divided by n of the color vectors associated with the n corresponding pixels of the component images. The color vectors are for example defined in the CIE XYZ color space. The n component images are displayed successively. The color of the pattern pixels of the source image is thus modulated temporally.

The image thus processed is then displayed with an appropriate device.

The result obtained on the illegal copy by any of these treatments (metamerism, infrared emission, temporal modulation of the color) on an image is illustrated by the 2D figure which represents the image produced by a camcorder when it captures the image. source image of Figure 2A. The image produced by the camcorder includes both the content of the source image and the contours of the distorted source image. The contours of the distorted source image form the scrambling pattern. This image is very disturbing because it forces the viewer to accommodate constantly to disregard contours. FIG. 3 illustrates a system 1 able to implement the first embodiment of the method of the invention represented in FIG. 1. This system 1 comprises a first image processing circuit 30 for generating a distorted image of the source image, an edge detection circuit 31 for detecting contours in the distorted source image and a second image processing circuit 32 for applying a process for changing the color or spectral composition of the pattern pixels of the source image. The source image thus modified is then displayed by a display device suitable such as a video projector with three or four primary or a projector equipped with infrared transmission means.

According to a second embodiment, the processing applied to the source image is not limited to the pixels corresponding to contours of the distorted source image.

FIG. 4 shows the steps of the method of the invention according to this second embodiment. It comprises a step 40 of deformation of the source image in which a vertical and / or horizontal phase shift is applied to the pixels of the source image, different phase shifts possibly being applicable to the three color components of the source image, a step 41 calculation, component by component, the differences between the source image and the distorted source image and a step 42 of processing the source image for modifying the spectral composition of the pixels of the source image as a function of the calculated differences .

The step 40 of deformation of the source image consists, for example, in applying to each pixel I (x, y) a phase θ which is the same for all the color components of the pixel, as described previously with reference to step 10 of FIG. 1, or to apply a different phase for each of the color components, namely a phase θ _R = (θ _Rx , θ _Ry ) for the red component, a phase θ _v = (θ _Vx , θ _Vy ) for the green component and a phase Θ _B = (θ _Bx , θ _By ) for the blue component. In the case of a vertical phase shift only, these different phases are for example

- θ _Rx = O and θ _Ry (x, y) = m _R .cos (ϋ5 _0R y)

θ _Vx = 0 and θ _Vy (x, y) = m _v ccos (ϋ5 _0V y), and

- _Bx θ = θ O and _By (x, y) = _R m cos (ϋ5 _0B y) where - _R m e [θ, t] denotes a modulation index and ϋ3 _0R pulsation value for the red component; and

- m _v e [θ, l] designates a modulation index and 03 _ov a pulsation value for the green component; and m _B G [θ, 1] designates a modulatio value of pulsation value for the blue component. As indicated previously, each pixel I (x, y) of the source image is represented by the following vector: l {x, y) = [R {x, y \ V {x, y \ B {x, y) ] ^τ where - R (x, y) represents the red component, - V (x, y) represents the green component, and

- B (x, y) represents the blue component. Each offset pixel I '(x, y) of the deformed image is then represented by the following vector:

T {x, y) = [R '(x, y), V' (x, y), B '(x, y)] ^τ .

With reference to step 41, the color differences between the deformed image and the source image, pixel by pixel, are calculated; component by component, the differences are assessed as follows: Diff _R (x, y) = R (x, y) - R '(x, y) for red,

Diff _v (x, y) = v {x, y) - V '{x, y) for green, and Diff _B (x, y) = B (x, y) -B' (x, y) for blue.

Given the definition of the distorted image (step 40 above), we have: _R Diff (x, y) = R (x, y) - R (x, y + _R m cos (y ϋ5 _0R )) for red,

Diff _v (x, y) = V (x, y) - V (x, y + m _v. Cos (ϋ3 _0V y)) for green, and Diff _B (x, y) = B (x, y) - B (x, y + m _B .cos (ϋ3 _0B y)) for blue.

According to step 42, the source image, initially based on the three primary colors R, V and B, called basic primary colors, is converted into an image based on six primary colors R1, Vl, B1, R2, V2 and B2.

The definition of the six primary colors is such that primary colors R1, Vl and B1 are respectively metamers of the primary colors R2, V2 and B2, as generally described, for example, in document EP 1971157. Instead of being represented as previously by a single component triplet, a component for each of the three basic primary colors, it is now necessary to represent each pixel I (x, y) by two triplets of components, one first triplet comprising the primary color components R1, Vl and B1 and a second triplet comprising the primary color components R2, V2 and B2.

Furthermore, the pairs of primary metameric colors (R1, R2), (V1, V2) and (B1, B2) are each associated with a basic primary color, R, V and B, respectively.

The primary colors Rl Vl and Bl being respectively metamers of the primary colors R2, V2 and B2, we have:

XYZ (S _R1 ) = XYZ (S _R2 ) = XYZ (a • S _R1 + b • S _R2 )

V (a, b) e [θ, 1] / a + b = 1 XYZ (S _V1 ) = XYZ (S _V2 ) = XYZ (a • S _V1 + b • S _V2 )

XYZ (S _B1 ) = XYZ (S _B2 ) = XYZ (a • S ₃₁ + b • S _B2 )

where - S _RI , Sγi and S _B i respectively denote the spectrum of the primary colors R1, Vl, B1;

- S _R2 , S _V2 and S _B2 respectively denote the spectrum of primary colors R2, V2, B2; and

- XYZ (S) denotes the color associated with the spectrum S in the CIE XYZ color space.

The second triplet of primary color components R2, V2, B2 is preferably defined with respect to the first component triplet of the primary colors R1, Vl, B1 so that the difference between the spectra of the primary colors is maximum while remaining globally invisible for the human eye.

The six primary colors being thus defined, again according to step 42, each pixel I (x, y) of the converted image which is based on the six primary colors R1, Vl, B1, R2, V2 and B2 is generated at from the corresponding pixel I (x, y) of the source image based on the three primary primary colors R, V and B so as to verify the following conditions:

- condition 1

where the first triplet _R i (x, y), _V i (x, y), a _B i (x, y) defines the components of the pixel I (x, y) of the primary colors R1, Vl and B1, and the second triplet aR2 (x, y), a _V 2 (x, y) and a _B 2 (x, y) defines the components of the pixel I (x, y) of the primary colors R2, V2 and B2.

- condition 2:

- a _R2 (x, y) = Diff _R (x, y) / R _max and a _R1 (x, y) = 1 - a _R2 (x, y)

- a _V2 (x, y) = Diff _v (x, y) / V _max and a _v1 (x, y) = 1 - a _V2 (x, y)

- a _B2 (x, y) = Diff _B (x, y) / B _max and a _B1 (x, y) = 1 - a _B2 (x, y) where R _max , V _max and B _max denote the maximum values basic primary color components R, V and B, for example 255 for an 8-bit coded image.

Thus, for a given basic primary color component R, V or B, the greater the difference in Diff _R , Diff _v , and / or Diff _B color between the source image and the distorted source image for the same pixel. of coordinates (x, y), plus the components aR2, a _V 2 and a _B 2 of the primary colors R2, V2 or B2 have a high value; conversely, the smaller the Diff _R , Diff _v , and / or Diff _B color difference between the source image and the deformed source image for the same pixel of coordinates (x, y), the more the components a _R i, a _V i and a _B i primary colors have a high value.

When the difference between the source image and the deformed source image is zero for a given primary color component i i (case, in particular, where m _i .cos (03 _oi y) = 0), the pixel is displayed only for example, when the difference between the source image and the deformed source image is zero for the basic primary color component R (where m _R .cos (ϋ3 _OR y) = 0), the coefficient a _R2 is zero and the coefficient a _R1 is equal to 1. The value of the primary color component belonging to the second triplet increases as the difference between the source image and the distorted source image increases.

Thus, from the source image and the deformed image obtained at the end of step 40, a converted image, ie a processed image, is obtained at the end of step 42 . Figures 5A-5D illustrate the results of the method in the context of the second embodiment. Figure 5A shows a source image to be protected. In the present case, this source image is identical to that of FIG. 2A. Figure 5B shows a distorted image of the source image in which the same deformation is applied to each of the color components of the source image. This distorted source image is identical to the image of Figure 2B. Figure 5C is a monochrome image showing the differences between the source image and the distorted source image for one of the color components. The bright spots represent the pixels for which the calculated difference is high and the dark points represent the pixels for which the difference is small or zero. Finally, Figure 5D shows the final processed image.

FIG. 6 illustrates a system 100 able to implement this second embodiment. This system 100 includes a first image processing circuit 130 for generating a distorted image of the source image, a calculation circuit 131 for calculating, component by component, the differences between the source image and the distorted source image and a second image processing circuit 132 for applying a processing modifying the spectral composition of the pixels of the source image according to the calculated difference values. The source image thus modified is then displayed by a six primary video projector.

Although the invention has been described in connection with various particular embodiments, it is obvious that it is not limited thereto and that it comprises all the technical equivalents of the means described and their combinations if they are within the scope of the invention.

Claims

A method of processing a source image comprising the steps of: - generating a distorted image by spatial deformation of said source image (10; 40), and

- processing the source image (11,12; 41,42) to modify globally not perceptible by the eye, the color of pixels of the source image as a function of the distorted image.

2. The method of claim 1, wherein said spatial deformation is obtained by vertical and / or horizontal shift of the pixels of the source image, the offset applied to the pixels being spatially variable within said source image.

The method of claim 2, wherein the offset applied to each pixel depends on the position of said pixel in said source image.

4. Method according to any one of claims 1 to 3, wherein the processing of the source image comprises, prior to said modifying step (12), a step of selecting (11) the pixels of the source image of which the color is to be modified.

5. Method according to claim 4, characterized in that the selection step (11) comprises a step of detecting contours in the distorted image, the selected pixels of the source image for modification then corresponding to the contours detected in the image. distorted image.

The method according to any one of the preceding claims, wherein the color of each pixel of the source image comprises at least four primary color components and wherein the pixel color changing step (12) of the picture source comprises a step of replacing said color with a metameric color.

The method according to claim 4 or 5, wherein the step of modifying (12) the source image comprises a step of decomposing the source image into n different component images for the pixels corresponding to the selected pixels of the image. source image such that, a color vector being associated with each of the pixels of said source image and component images in a predetermined color space, the color vector associated with each pixel of said source image is equal to the resultant divided by n color vectors associated with the n pixels of the component images corresponding to said pixel of the source image.

8. Process according to any one of claims 4, 5 and

7, in which the spectral composition of the pixels of the source image is modified by adding an infrared component to the pixels of the source image corresponding to the selected pixels of the source image.

9. Method according to any one of claims 1 to 3, wherein the processing of the source image comprises, prior to said modifying step (12), a step of calculating the color difference between each pixel of the source image and the same pixel of the deformed image, and wherein said color change of each pixel of the source image is a function of said difference.

The method of claim 9, wherein the color of each pixel of the source image comprises at least four primary color components, wherein the pixel color changing step (12) of the source image comprises a step of replacing said color with a metameric color so that at least one of the color components of each modified pixel is proportional to said color difference between the same pixel of the source image and the same pixel of the image. distorted image.

The method of any of the preceding claims, wherein the spatial distortion applied to the source image differs according to the basic primary color component.