WO2007053745A2 - Motion picture content editing - Google Patents

Motion picture content editing Download PDF

Info

Publication number
WO2007053745A2
WO2007053745A2 PCT/US2006/042798 US2006042798W WO2007053745A2 WO 2007053745 A2 WO2007053745 A2 WO 2007053745A2 US 2006042798 W US2006042798 W US 2006042798W WO 2007053745 A2 WO2007053745 A2 WO 2007053745A2
Authority
WO
WIPO (PCT)
Prior art keywords
color
values
matrix
icc
scene
Prior art date
Application number
PCT/US2006/042798
Other languages
French (fr)
Other versions
WO2007053745A3 (en
Inventor
Manish S. Kulkarni
Lars U. Borg
Original Assignee
Adobe Systems Incorporated
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Adobe Systems Incorporated filed Critical Adobe Systems Incorporated
Priority to CN2006800464565A priority Critical patent/CN101401108B/en
Priority to EP06827368A priority patent/EP1949301A2/en
Priority to JP2008539023A priority patent/JP2009514483A/en
Publication of WO2007053745A2 publication Critical patent/WO2007053745A2/en
Publication of WO2007053745A3 publication Critical patent/WO2007053745A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/253Picture signal generating by scanning motion picture films or slide opaques, e.g. for telecine

Definitions

  • a typical motion picture workflow begins with a camera recording a scene on a medium such as an original camera negative (OCN) for traditional cameras or digital media (e.g., optical or magnetic media) for digital cameras.
  • OCN original camera negative
  • digital media e.g., optical or magnetic media
  • a digitized representation of captured images may be then provided to a computer workstation, such as by scanning the film.
  • the workstation may include one or more monitors on which the appearance of the digitized data can be previewed. Editorial modifications may be made and special effects may be added to the digitized data.
  • the digitized data (with the modifications and special effects) may be transferred to film for projection on a cinema screen.
  • the digitized data maybe transferred to a digital projector for projection on the cinema theater screen.
  • Figure 1 is a block diagram illustrating a system of an embodiment of the present invention
  • Figure 2 illustrates a workflow according to an embodiment of the present invention
  • Figure 3 illustrates a workflow according to an embodiment of the present invention to create LUT curves
  • Figure 4 illustrates a workflow according to an embodiment of the present invention to create a Matrix
  • Figure 5 illustrates a workflow according to an embodiment of the present invention to code an ICC profile
  • Figure 6 illustrates an apparatus to implement processes similar to that provided in Figures 2-4.
  • DPX data obtained by scanning Negative Camera Film to recreate an original filmed scene image.
  • Parameters used to process the data can be stored as an INTERNATIONAL COLOR CONSORTIUM format color profile (ICC Profile), and the scene image used in Color Managed workflows to apply Special Effects to the processed DPX images to make the Special Effects appear more realistic.
  • ICC Profile INTERNATIONAL COLOR CONSORTIUM format color profile
  • other color profiles and other file formats may be utilized with similar characteristics.
  • embodiments of the invention are described in the general context of computer-executable instructions, such as program modules, being executed by a personal computer.
  • program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
  • Figure 1 shows a diagrammatic representation of a machine system in the exemplary form of a computer system 100 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed.
  • the machine operates as a standalone device or may be connected (e.g., networked) to other machines.
  • the machine may operate in the capacity of a server or a client machine in server- client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.
  • the machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
  • PC personal computer
  • PDA Personal Digital Assistant
  • STB set-top box
  • a Personal Digital Assistant Personal Digital Assistant
  • a cellular telephone a web appliance
  • network router switch or bridge
  • any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
  • the term "machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed
  • the exemplary computer system 100 includes a central processing unit (CPU) 110, a graphics processing unit (GPU) 120, a main memory 130 which can include a static memory, which communicate with each other via a bus 140.
  • the computer system may further include a video display unit 170 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)).
  • the computer system can also include an alphanumeric input device 160 (e.g., a keyboard), a user interface (UI) navigation device 162 (e.g., a mouse), a disk drive unit 150.
  • UI user interface
  • the disk drive unit 150 includes a machine-readable medium 152 on which is stored one or more sets of instructions and data structures (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein.
  • the software may also reside, completely or at least partially, within the main memory 130 and/or within the processor(s) 110/120 during execution thereof by the computer system 100, the main memory 130 and the processor(s) 110/120 also constituting machine-readable media.
  • the software may further be transmitted or received over a network via a network interface device utilizing any one of a number of well- known transfer protocols (e.g., HTTP).
  • machine-accessible medium 152 is shown in an exemplary embodiment to be a single medium, the term “machine-accessible medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions.
  • the term “machine-accessible medium” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present invention, or that is capable of storing, encoding or carrying data structures utilized by or associated with such a set of instructions.
  • the term “machine-accessible medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media, and carrier wave signals.
  • Embodiments of the present invention provide methods to convert
  • DPX colors to a scene space
  • an ICC Color Management Engine for the conversions. See INTERNATIONAL COLOR CONSORTIUM Specification ICC. 1 :2004-10. These methods enable manipulation, editing and application of special effects in DPX files to appear more realistic in a finished product.
  • Using an ICC Profile allows an existing ICC Color Management architecture to be leveraged for the color conversions.
  • ICC Integrated Circuit
  • ICC profiles provide a cross-platform device profile format.
  • ICC profiles can be used to translate color data created on one device into the native color space of another device.
  • Use of such color profiles allow end users to transparently move profiles and images with embedded profiles between different operating systems while maintaining color fidelity.
  • a color output from one device or software to another device or software needs to be converted (or translated) to maintain a consistent color output.
  • software executed on a computer such as image processing software, includes a color engine to determine the proper operations needed to convert color data from one ICC profile to another ICC profile. Color conversions can be performed using look-up tables (LUT) and matrix calculations.
  • the following may be based on, for illustrative purposes, digitized data obtained from negative film by a DPX system that may be illuminated by a projection system using a xenon light source.
  • film generally refers to a light sensitive material, such as but not limited to celluloid covered in a light sensitive emulsion.
  • the digitized data may be in one of many standards including DPX files. After the processing of the digitized data has been completed, the digitized data may be recorded back onto negative film which in turn is printed onto print film for projection onto a cinema screen. Alternatively, the digitized data may be provided to a digital projector and directly projected onto a cinema screen.
  • the digitized image file format may be a CINEON format file which stores data in ten bit density using printing densities.
  • file formats may be used that store data using Status M densities (which may, for example, be derived from printing densities).
  • the CINEON image file format is a subset of the ANSI/SMPTE DPX file format.
  • a DPX format file consists of four parts: (i) a generic file information header having a fixed format, predefined, general information header consisting of several sections (generic, image, data format and image origination); (ii) a motion picture and television industry specific header having a fixed format, industry (television, film) specific header; (iii) user-defined information including variable length, user defined data; and (iv) image data.
  • Figure 2 illustrates a motion picture workflow 200 in which a scene 205 may be captured by a film camera 210 on camera negative film 215.
  • the camera negative film 215 may be scanned by a scanner 220 to generate a CINEON / Digital Picture eXchange (DPX) format file 225.
  • the DPX file is converted 228 using an ICC profile as explained below.
  • the DPX file 225 may be viewed and altered in an editor 230 to generate an edited DPX file 235.
  • the edited DPX file 235 may then be transferred to negative film 250 by a recorder 240.
  • the negative film 250 may be developed into a positive print film 260 by a motion picture film printer 255.
  • the positive print film 260 may be loaded into a film projector 265 so that a picture may be projected onto a cinema screen 270.
  • OCN Original Camera Negative
  • OCN Original Camera Negative
  • the DPX file can be digitally manipulated on a computer, and recorded back onto film. The recorded film is then used for projection in the theatre.
  • the manipulations that are done on the computer include the addition of "Effects" that range from basic changes such as improving brightness/contrast to very complex effects such as high-dynamic range toning. These Effects are generally added using Computer Software such as Adobe® After Effects® software from Adobe Systems, Inc.
  • DPX files require that the content first be converted to a scene space, which is a representation of the original appearance of the content in the real world. This is necessary in order for the effects to produce the most realistic results.
  • Embodiments of the present invention convert the content of the DPX Files to the scene space.
  • Methods described herein include determining the color conversion parameters required to convert DPX files to the scene space. These color conversion parameters are saved as an ICC Profile which can be used by a Color Management Engine to convert the colors in a DPX file to the scene space. The conversion from a DPX RGB Space to a Scene Space (for editing) and the conversion from Scene Space back to a DPX RGB Space (after editing) are performed using the ICC Profile. As such, the ICC Profile contains color conversion parameters to convert from the DPX RGB Space to the Scene Space as well as parameters to convert from the Scene Space back to the DPX RGB Space.
  • scene as used herein refers to an image captured on film.
  • the film captures the scene based upon known light sensitivity principles.
  • an exposed film is scanned to convert the light-captured scene into digital data the scanned conversion does not accurately recreate or represent the light colors originally captured on film.
  • Methods described herein provide a conversion process that converts the color data obtained by scanning the film and provides scene color data that colormetrically represents the original scene colors as captured on film.
  • scene color data is data provided as an output of a color conversion.
  • FIG. 3 illustrates a method 300, according to an example embodiment, for generating three tone reproduction curves (TRCs) in a process flow diagram.
  • Figure 4 illustrates a method 400, according to an example embodiment, for generating a matrix in a process flow diagram.
  • DPX RGB color space values may be converted to Commission Internationale de L'Eclairage (CIE) XYZ color space values using TRCs and a 3x3 Matrix, to convert DPX R 1 G 1 B' to XYZ. That is, DPX R 1 G' B 1 ⁇ [TRCs] ⁇ RGB ⁇ [ 3x3 Matrix ] ⁇ XYZ.
  • CIE Commission Internationale de L'Eclairage
  • DPX R 1 G' B 1 ⁇ [TRCs] ⁇ RGB ⁇ [ 3x3 Matrix ] ⁇ XYZ.
  • the data for such a conversion may be stored in a color profile such as an ICC Profile.
  • FIG. 3 illustrates a process 300 according to an embodiment to generate three identical ID LUTs using published Kodak 5218 Characteristic Curves for KODAK VISION2 500T Color Negative Film 5218.
  • PD DPX Printing Density
  • StM Status M
  • StM 0.6144 + 0.9355 * PD.
  • StM 0.6144 + 0.9355 * PD.
  • the resulting Status M range [0.6144, 2.5284] is converted to Log Exposure using the full published Kodak 5218 Characteristic Curve data at sub- operation 304. Different film characteristic curves can be used to correspond to the film used.
  • the resultant linear TRC may be adjusted, or scaled, at sub-operation 308 to map a DPX 470 value to a reference gray value (e.g., a value from 10% gray to 20% gray, such as 18% gray).
  • TRCs three tone reproduction curves
  • a 3x3 Scene Matrix is determined 400 using estimated XYZ Color Matching Functions.
  • the Scene Matrix will be used to convert the RGB values from the TRCs to XYZ values.
  • XYZ Color As a first sub-operation 402 to create the ICC Matrix, XYZ Color
  • Matching Functions are estimated as a 3x3 Matrix transform of the RGB Sensitivities of Kodak 5218 Camera Film.
  • a set of Natural Object Spectra under D65 illuminant is used in one embodiment as a training set 404.
  • the 3x3 Matrix is chosen such that an error difference (Delta-E) between the XYZ output values from the estimated matrix using the training set and actual XYZ values is minimized 406.
  • the resultant matrix is chromatically adapted to ICC D50 Illuminant to provide the Scene Matrix 408.
  • One method of performing chromatic adaptation is described in Annex E of the INTERNATIONAL COLOR CONSORTIUM Specification. [0033] Referring to Figure 5, a V4 ICC Profile (see ICC.l:2004-10
  • the ICC profile is created by creating both A2B0 and B2A0 tags.
  • the ID LUTs values range from approximately 0.00185 to 75.83, as described above. These values are scaled 502 so that the maximum value is 1, giving a range approximately of [2.4459e-5, 1]. These scaled curves have a very small slope in the beginning making it difficult to encode as an ICC Profile, which has only 16-bit precision.
  • the curves are divided into two curves 504 to allow for encoding. The first curve is the original curves to the power 0.5, and the second curve is simply a gamma function with exponent 2.0 applied to the first curve. Raising the original curve to a power of 0.5 decreases the slope sufficiently so that the curve can be accurately encoded in the ICC Profile.
  • the A2B0 tag defines a color transform from Device to a Profile Connected Space (PCS ) using lookup table tag element structures and the B2A0 tag defines a color transform from PCS to Device using lookup table tag element structures. See INTERNATIONAL COLOR CONSORTIUM Specification ICC. 1 :2004-10.
  • the A2B0 color transform follows the following sequence: ("A" curves) ⁇ (Multidimensional lookup table - GLUT) ⁇ ("M" Curves) ⁇ (Matrix) ⁇ ("B” Curves).
  • the B2A0 color transform follows the following sequence: (“B” curves) ⁇ (Matrix) ⁇ ("M” Curves) ⁇ (Multidimensional lookup table - GLUT) ⁇ ("A" Curves).
  • A2B0 tag (RGB to XYZ) for the ICC Profile is generated 506 as:
  • Matrix Scene Matrix B Curves: Identity (null operation)
  • Using a gamma of 2.0 with the inverse Scene Curve simplifies processing, because the M-curve can then be implemented using a square root operation which is generally faster than a general power function implementation.
  • a gamma of 2.0 with the inverse Scene Curve is faster than using a different gamma, for example 4.0.
  • the Media White Point in the Profile in one embodiment is set to be 75.83 (max value of un-scaled Scene Curve) times D50.
  • the reason for doing this is that when Absolute Colorimetric rendering intent is used, the Color Management Engine does the correct scaling of the Scene Curve to the full DPX range. This is a part of ensuring that the entire DPX Range can be encoded in ICC Profiles that otherwise do not support over-range data outside of a [0,1] range.
  • the above embodiments convert DPX colors to Scene colors using parameters derived from characteristics of the Original Camera Negative.
  • An ICC color profile is used in one embodiment for the conversion.
  • the ICC color profile uses look-up tables and a matrix for the conversion.
  • a method described herein to determine ID LUT' s includes converting Printing Density to Status M and then to Linear values.
  • a method described herein to determine the matrix includes error minimization using a natural objects spectra as a training set, and estimates XYZ Color Matching Functions.
  • the ICC Profile encoding described herein splits the tone reproduction curves (TRCs) into two, specifically the Inverse Scene Curve with a Gamma of 2.0 to improve performance.
  • TRCs tone reproduction curves
  • the method described herein to encode entire DPX range includes over-range data in the ICC Profile.
  • Figure 6 illustrates an apparatus 600 that may be standalone or maybe integrated into the apparatus of Figure 1.
  • a LUT values calculation unit 610 is provided to determine LUT values as described above using characteristics of a film.
  • a matrix values calculation unit 620 is provided to determine matrix values as described above using RGB sensitivities of the film.
  • An ICC unit 630 uses the values determined for the LUTs and the matrix to encode an ICC Profile for execution by transformation unit 640.
  • the transformation unit transforms a DPX input data 650 into XYZ data 660 provided to editor unit 670. As explained above, the editor unit allows for edits to be made to the color data.
  • the edited XYZ color data 680 is provided to the transformation unit 640 which converts the edited color data to output DXP color data 690 which can be transferred to film, as explained above.
  • the accompanying drawings that form a part hereof show by way of illustration, and not of limitation, specific embodiments in which the subject matter may be practiced. The embodiments illustrated are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other embodiments may be utilized and derived there from, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. This Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.
  • inventive subject matter may be referred to herein, individually and/or collectively, by the term "invention" merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed.
  • inventive subject matter may be referred to herein, individually and/or collectively, by the term "invention" merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed.
  • inventive subject matter merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed.

Abstract

Processing Digital Picture eXchange (DPX) data obtained by scanning Negative Camera Film to colormetrically recreate an original de-rendered scene image is described. One method described includes converting a DPX color file, obtained from scanning a camera negative film used to capture a scene with a film camera, to scene color data using an ICC Color management engine. The ICC Color management engine executes an ICC color profile created using parameters representing a characteristic curve and color sensitivity of the camera negative film.

Description

MOTION PICTURE CONTENT EDITING
Claim of Priority
[0001] This PCT application claims the benefit of the filing date of U.S.
Patent Application Serial No. 11/264,879, filed November 1, 2005 entitled, "MOTION PICTURE CONTENT EDITING," which priority is hereby claimed under 35 U.S.C. § 120 or 365(c).
Technical Field
[0002] The subject matter herein relates to data processing.
Background Information
[0003] The motion picture industry is increasingly relying on computer systems to modify and preview films. A typical motion picture workflow begins with a camera recording a scene on a medium such as an original camera negative (OCN) for traditional cameras or digital media (e.g., optical or magnetic media) for digital cameras. A digitized representation of captured images may be then provided to a computer workstation, such as by scanning the film. The workstation may include one or more monitors on which the appearance of the digitized data can be previewed. Editorial modifications may be made and special effects may be added to the digitized data. Thereafter, for traditional film projectors, the digitized data (with the modifications and special effects) may be transferred to film for projection on a cinema screen. Alternatively, the digitized data maybe transferred to a digital projector for projection on the cinema theater screen.
[0004] Any manipulations to the digitized data (e.g., computer rendered special effects, combinations of multiple footage from multiple scenes, etc.) often do not appear as realistic as desired due to the film scanning operation.
Brief Description of the Drawings
[0005] Figure 1 is a block diagram illustrating a system of an embodiment of the present invention; [0006] Figure 2 illustrates a workflow according to an embodiment of the present invention;
[0007] Figure 3 illustrates a workflow according to an embodiment of the present invention to create LUT curves;
[0008] Figure 4 illustrates a workflow according to an embodiment of the present invention to create a Matrix;
[0009] Figure 5 illustrates a workflow according to an embodiment of the present invention to code an ICC profile; and
[0010] Figure 6 illustrates an apparatus to implement processes similar to that provided in Figures 2-4.
Detailed Description
[0011] In the following detailed description, reference is made to the accompanying drawings which form a part hereof, and in which is shown, by way of illustration, different embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Other embodiments may be utilized and structural, logical, and electrical changes may be made without departing from the scope of the present invention.
[0012] The following describes processing Digital Picture eXchange
(DPX) data obtained by scanning Negative Camera Film to recreate an original filmed scene image. Parameters used to process the data can be stored as an INTERNATIONAL COLOR CONSORTIUM format color profile (ICC Profile), and the scene image used in Color Managed workflows to apply Special Effects to the processed DPX images to make the Special Effects appear more realistic. It will be appreciated by the skilled artisan that other color profiles and other file formats may be utilized with similar characteristics. [0013] Although not required, embodiments of the invention are described in the general context of computer-executable instructions, such as program modules, being executed by a personal computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Those skilled in the art will appreciate that other computer system configurations, including hand-held devices, multi-processor systems, microprocessor based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like can be used to practice embodiments of the invention. [0014] An example system for implementing embodiments of the invention is illustrated in Figure 1. Figure 1 shows a diagrammatic representation of a machine system in the exemplary form of a computer system 100 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client machine in server- client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.
[0015] The exemplary computer system 100 includes a central processing unit (CPU) 110, a graphics processing unit (GPU) 120, a main memory 130 which can include a static memory, which communicate with each other via a bus 140. The computer system may further include a video display unit 170 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system can also include an alphanumeric input device 160 (e.g., a keyboard), a user interface (UI) navigation device 162 (e.g., a mouse), a disk drive unit 150.
[0016] The disk drive unit 150 includes a machine-readable medium 152 on which is stored one or more sets of instructions and data structures (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. The software may also reside, completely or at least partially, within the main memory 130 and/or within the processor(s) 110/120 during execution thereof by the computer system 100, the main memory 130 and the processor(s) 110/120 also constituting machine-readable media. [0017] The software may further be transmitted or received over a network via a network interface device utilizing any one of a number of well- known transfer protocols (e.g., HTTP).
[0018] While the machine-accessible medium 152 is shown in an exemplary embodiment to be a single medium, the term "machine-accessible medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term "machine-accessible medium" shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present invention, or that is capable of storing, encoding or carrying data structures utilized by or associated with such a set of instructions. The term "machine-accessible medium" shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media, and carrier wave signals.
[0019] Embodiments of the present invention provide methods to convert
DPX colors to a scene space, and use an ICC Color Management Engine for the conversions. See INTERNATIONAL COLOR CONSORTIUM Specification ICC. 1 :2004-10. These methods enable manipulation, editing and application of special effects in DPX files to appear more realistic in a finished product. Using an ICC Profile allows an existing ICC Color Management architecture to be leveraged for the color conversions.
[0020] Color management using the International Color Consortium
("ICC") standard is widely implemented for converting color data from one color space to another color space. That is, ICC profiles provide a cross-platform device profile format. ICC profiles can be used to translate color data created on one device into the native color space of another device. Use of such color profiles allow end users to transparently move profiles and images with embedded profiles between different operating systems while maintaining color fidelity. For example, a color output from one device or software to another device or software needs to be converted (or translated) to maintain a consistent color output. Often software executed on a computer, such as image processing software, includes a color engine to determine the proper operations needed to convert color data from one ICC profile to another ICC profile. Color conversions can be performed using look-up tables (LUT) and matrix calculations.
[0021] The following may be based on, for illustrative purposes, digitized data obtained from negative film by a DPX system that may be illuminated by a projection system using a xenon light source. The term film generally refers to a light sensitive material, such as but not limited to celluloid covered in a light sensitive emulsion. The digitized data may be in one of many standards including DPX files. After the processing of the digitized data has been completed, the digitized data may be recorded back onto negative film which in turn is printed onto print film for projection onto a cinema screen. Alternatively, the digitized data may be provided to a digital projector and directly projected onto a cinema screen. Other print films, scanners, and illuminants may be utilized depending on the desired implementation. In particular, the subject matter described herein may also be used in connection with a workflow involving a digital camera and/or a digital projector. [0022] The digitized image file format may be a CINEON format file which stores data in ten bit density using printing densities. In some variations, file formats may be used that store data using Status M densities (which may, for example, be derived from printing densities). The CINEON image file format is a subset of the ANSI/SMPTE DPX file format. A DPX format file consists of four parts: (i) a generic file information header having a fixed format, predefined, general information header consisting of several sections (generic, image, data format and image origination); (ii) a motion picture and television industry specific header having a fixed format, industry (television, film) specific header; (iii) user-defined information including variable length, user defined data; and (iv) image data.
[0023] Figure 2 illustrates a motion picture workflow 200 in which a scene 205 may be captured by a film camera 210 on camera negative film 215. The camera negative film 215 may be scanned by a scanner 220 to generate a CINEON / Digital Picture eXchange (DPX) format file 225. The DPX file is converted 228 using an ICC profile as explained below. The DPX file 225 may be viewed and altered in an editor 230 to generate an edited DPX file 235. The edited DPX file 235 may then be transferred to negative film 250 by a recorder 240. Thereafter, the negative film 250 may be developed into a positive print film 260 by a motion picture film printer 255. The positive print film 260 may be loaded into a film projector 265 so that a picture may be projected onto a cinema screen 270.
[0024] As stated above, in a Digital Motion Picture workflow the
Original Camera Negative (OCN) that contains the captured scene is scanned using film scanners and converted to a digital DPX file that is encoded in a 10- bit Printing Density. The DPX file can be digitally manipulated on a computer, and recorded back onto film. The recorded film is then used for projection in the theatre. The manipulations that are done on the computer include the addition of "Effects" that range from basic changes such as improving brightness/contrast to very complex effects such as high-dynamic range toning. These Effects are generally added using Computer Software such as Adobe® After Effects® software from Adobe Systems, Inc.
[0025] Many of the computer effects that are added to the content in the
DPX files require that the content first be converted to a scene space, which is a representation of the original appearance of the content in the real world. This is necessary in order for the effects to produce the most realistic results. Embodiments of the present invention convert the content of the DPX Files to the scene space.
[0026] Methods described herein include determining the color conversion parameters required to convert DPX files to the scene space. These color conversion parameters are saved as an ICC Profile which can be used by a Color Management Engine to convert the colors in a DPX file to the scene space. The conversion from a DPX RGB Space to a Scene Space (for editing) and the conversion from Scene Space back to a DPX RGB Space (after editing) are performed using the ICC Profile. As such, the ICC Profile contains color conversion parameters to convert from the DPX RGB Space to the Scene Space as well as parameters to convert from the Scene Space back to the DPX RGB Space.
[0027] The term scene as used herein refers to an image captured on film. The film captures the scene based upon known light sensitivity principles. When an exposed film is scanned to convert the light-captured scene into digital data the scanned conversion does not accurately recreate or represent the light colors originally captured on film. Methods described herein provide a conversion process that converts the color data obtained by scanning the film and provides scene color data that colormetrically represents the original scene colors as captured on film. As such, the terms scene color data is data provided as an output of a color conversion. Some of the described processes use an ICC profile created using parameters representing a characteristic curve and sensitivity of the actual film used to capture the scene. [0028] Figure 3 illustrates a method 300, according to an example embodiment, for generating three tone reproduction curves (TRCs) in a process flow diagram. Figure 4 illustrates a method 400, according to an example embodiment, for generating a matrix in a process flow diagram. In one embodiment, DPX RGB color space values may be converted to Commission Internationale de L'Eclairage (CIE) XYZ color space values using TRCs and a 3x3 Matrix, to convert DPX R1G1B' to XYZ. That is, DPX R1G' B1 → [TRCs] → RGB → [ 3x3 Matrix ] →XYZ. The data for such a conversion may be stored in a color profile such as an ICC Profile.
[0029] Figure 3 illustrates a process 300 according to an embodiment to generate three identical ID LUTs using published Kodak 5218 Characteristic Curves for KODAK VISION2 500T Color Negative Film 5218. As a first sub- operation 302, the DPX Printing Density (PD) Range [0, 2.046] is converted to a Status M (StM) value using a simple linear relationship: StM = 0.6144 + 0.9355 * PD. The resulting Status M range [0.6144, 2.5284] is converted to Log Exposure using the full published Kodak 5218 Characteristic Curve data at sub- operation 304. Different film characteristic curves can be used to correspond to the film used. Next at sub-operation 306, the Log Exposure data is converted to Linear values using Linear Value = 10 Λ (- Log Value). Finally, the resultant linear TRC may be adjusted, or scaled, at sub-operation 308 to map a DPX 470 value to a reference gray value (e.g., a value from 10% gray to 20% gray, such as 18% gray).
[0030] As a result of process 300, three tone reproduction curves (TRCs) are generated using the film characteristics and represented by three ID LUTs. These LUTs are used to convert the DPX R'G'B' values to RGB values. [0031] Referring to Figure 4, a 3x3 Scene Matrix is determined 400 using estimated XYZ Color Matching Functions. The Scene Matrix will be used to convert the RGB values from the TRCs to XYZ values. [0032] As a first sub-operation 402 to create the ICC Matrix, XYZ Color
Matching Functions are estimated as a 3x3 Matrix transform of the RGB Sensitivities of Kodak 5218 Camera Film. A set of Natural Object Spectra under D65 illuminant is used in one embodiment as a training set 404. Those skilled in the art recognize that the D65 illuminant is intended to represent average daylight throughout the visible spectrum. The 3x3 Matrix is chosen such that an error difference (Delta-E) between the XYZ output values from the estimated matrix using the training set and actual XYZ values is minimized 406. The resultant matrix is chromatically adapted to ICC D50 Illuminant to provide the Scene Matrix 408. One method of performing chromatic adaptation is described in Annex E of the INTERNATIONAL COLOR CONSORTIUM Specification. [0033] Referring to Figure 5, a V4 ICC Profile (see ICC.l:2004-10
(Profile version 4.2.0.0)) is created with process 500. The ICC profile is created by creating both A2B0 and B2A0 tags.
[0034] The ID LUTs values range from approximately 0.00185 to 75.83, as described above. These values are scaled 502 so that the maximum value is 1, giving a range approximately of [2.4459e-5, 1]. These scaled curves have a very small slope in the beginning making it difficult to encode as an ICC Profile, which has only 16-bit precision. In one embodiment, the curves are divided into two curves 504 to allow for encoding. The first curve is the original curves to the power 0.5, and the second curve is simply a gamma function with exponent 2.0 applied to the first curve. Raising the original curve to a power of 0.5 decreases the slope sufficiently so that the curve can be accurately encoded in the ICC Profile. [0035] Those skilled in the art recognize that the A2B0 tag defines a color transform from Device to a Profile Connected Space (PCS ) using lookup table tag element structures and the B2A0 tag defines a color transform from PCS to Device using lookup table tag element structures. See INTERNATIONAL COLOR CONSORTIUM Specification ICC. 1 :2004-10. The A2B0 color transform follows the following sequence: ("A" curves) → (Multidimensional lookup table - GLUT) → ("M" Curves) → (Matrix) → ("B" Curves). The B2A0 color transform follows the following sequence: ("B" curves) → (Matrix) → ("M" Curves) → (Multidimensional lookup table - GLUT) → ("A" Curves).
[0036] The Scene Matrix described above is encoded as-is, so the overall
A2B0 tag (RGB to XYZ) for the ICC Profile is generated 506 as: A Curves: Scene Curve, with gamma 0.5 Color Look-up Table (GLUT): Identity (null operation) M Curves: Parametric Curve, gamma 2.0 (which is 1/0.5) Matrix: Scene Matrix B Curves: Identity (null operation)
[0037] For creating the B2A0 tag (XYZ to RGB) 508 for the ICC
Profile, the process is inverted as:
B Curves: Identity (null operation)
Matrix: inverse of Scene Matrix
M Curves: Parametric Curve, gamma 0.5 (which is 1/2.0)
CLUT: Identity (null operation)
A Curves: Inverse of Scene Curve, with gamma 2.0
[0038] Two curves are required in the B2A0 tag to avoid extremely high interpolation errors in the dark regions of the curve.
[0039] Using a gamma of 2.0 with the inverse Scene Curve simplifies processing, because the M-curve can then be implemented using a square root operation which is generally faster than a general power function implementation. Thus using a gamma of 2.0 with the inverse Scene Curve is faster than using a different gamma, for example 4.0.
[0040] The Media White Point in the Profile in one embodiment is set to be 75.83 (max value of un-scaled Scene Curve) times D50. The reason for doing this is that when Absolute Colorimetric rendering intent is used, the Color Management Engine does the correct scaling of the Scene Curve to the full DPX range. This is a part of ensuring that the entire DPX Range can be encoded in ICC Profiles that otherwise do not support over-range data outside of a [0,1] range.
[0041] The above embodiments convert DPX colors to Scene colors using parameters derived from characteristics of the Original Camera Negative. An ICC color profile is used in one embodiment for the conversion. The ICC color profile uses look-up tables and a matrix for the conversion. A method described herein to determine ID LUT' s includes converting Printing Density to Status M and then to Linear values. A method described herein to determine the matrix includes error minimization using a natural objects spectra as a training set, and estimates XYZ Color Matching Functions. The ICC Profile encoding described herein splits the tone reproduction curves (TRCs) into two, specifically the Inverse Scene Curve with a Gamma of 2.0 to improve performance. Finally, the method described herein to encode entire DPX range includes over-range data in the ICC Profile.
[0042] Figure 6 illustrates an apparatus 600 that may be standalone or maybe integrated into the apparatus of Figure 1. A LUT values calculation unit 610 is provided to determine LUT values as described above using characteristics of a film. Likewise, a matrix values calculation unit 620 is provided to determine matrix values as described above using RGB sensitivities of the film. An ICC unit 630 uses the values determined for the LUTs and the matrix to encode an ICC Profile for execution by transformation unit 640. The transformation unit transforms a DPX input data 650 into XYZ data 660 provided to editor unit 670. As explained above, the editor unit allows for edits to be made to the color data. The edited XYZ color data 680 is provided to the transformation unit 640 which converts the edited color data to output DXP color data 690 which can be transferred to film, as explained above. [0043] The accompanying drawings that form a part hereof show by way of illustration, and not of limitation, specific embodiments in which the subject matter may be practiced. The embodiments illustrated are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other embodiments may be utilized and derived there from, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. This Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.
[0044] Such embodiments of the inventive subject matter may be referred to herein, individually and/or collectively, by the term "invention" merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed. Thus, although specific embodiments have been illustrated and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all adaptations or variations of various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those of skill in the art upon reviewing the above description.

Claims

What is claimed is:
1. A method of processing data comprising: receiving color data representing a scene from a film; and using parameters representing a characteristic curve and color sensitivity of an original camera negative of the film, performing a color conversion operation to convert the color data to scene color data.
2. The method of claim 1 wherein the conversion operation comprises applying an international color consortium (ICC) profile to the received color data.
3. The method of claim 2 wherein the ICC profile comprises look-up tables (LUTs), wherein values of the LUTs are determined by a method comprising converting a printing density range to a Status M density range and converting the Status M density to log exposure using the characteristic curves of the original camera negative.
4. The method of claim 2 wherein the ICC profile comprises a matrix, wherein values of the matrix are determined by a method comprising: estimating values for the matrix based upon the color sensitivity of the original camera negative; applying the matrix estimated values to a training data set to provide estimated colormetric values; and adjusting the values of the matrix to minimize an error between the estimated colormetric values and known colormetric values.
5. A method of processing data comprising: determining values for three one-dimensional look-up tables (LUTs) to convert Red Green Blue (RGB) color values of the data to interim RGB color values, wherein the values of the LUTs are determined by a method comprising converting a printing density range to a Status M density range and converting the Status M density to log exposure using the characteristic curves of an original camera negative; determining matrix values for a matrix to convert the interim RGB values to XYZ color values, wherein the matrix values are determined by a method comprising estimating values for the matrix based upon the color sensitivity of the original camera negative, applying the matrix estimated values to a training data set to provide estimated colormetric values, and adjusting the values of the matrix to minimize an error between the estimated colormetric values and known colormetric values; creating an international color consortium (ICC) profile using the LUTs and the matrix; and applying the ICC profile to color data representing a scene captured on a film.
6. The method of claim 5 wherein creating an international color consortium (ICC) profile comprises: applying a gamma value of one-half to the LUT values; and encoding the LUT values after applying the gamma value.
7. The method of claim 5 wherein applying the ICC profile comprises encoding over-range data.
8. A method of processing data comprising: exposing a camera negative film to light to capture a scene; scanning the camera negative film to generate a Digital Picture eXchange (DPX) format color file representing the captured scene; and converting the DPX color file to scene color data using an ICC Color management engine, wherein the ICC Color management engine executes an ICC color profile created using parameters representing a characteristic curve and color sensitivity of the camera negative film.
9. The method of claim 8 wherein the ICC color profile comprises look-up tables (LUTs), wherein a content of the LUTs is determined by a method comprising converting a printing density range to a Status M density range and converting the Status M density to log exposure using the characteristic curves of the camera negative film.
10. The method of claim 8 wherein the ICC profile comprises a matrix, wherein values of the matrix are determined by a method comprising: estimating values for the matrix based upon a known color sensitivity of the camera negative film; applying the matrix estimated values to a training data set to provide estimated colormetric values; and adjusting the values of the matrix to minimize an error between the estimated colormetric values and known colormetric values.
11. The method of claim 8 wherein converting the DPX color file to the scene color data using an ICC Color management engine comprises: encoding three tone reproduction curves (TRCs); and applying the TRCs to a matrix operation.
12. The method of claim 11 wherein encoding the three TRCs comprises adjusting a slope of the TRCs to encode the TRCs with at least sixteen-bit precision.
13. The method of claim 11 wherein encoding the three TRCs comprises applying two gamma operations to the TRCs, wherein a first gamma operation is an inverse of the second gamma function.
14. The method of claim 8 further comprising: manipulating the scene color data; converting the manipulated scene color data into a second DPX color file using the ICC Color management engine; and recording the second DPX color file onto a film.
15. The method of claim 14 wherein converting the manipulated scene color data into the second DPX color file comprises: applying an inverse matrix to the manipulated color data into matrix output data in a DPX color format; and applying a gamma with a gamma value of two and applying a gamma operation to the matrix output data with a gamma value of one-half.
16. The method of claim 8 wherein converting the DPX color file to the scene color data using an ICC Color management engine maintains over-range data.
17. An article including a machine-accessible medium having associated information, wherein the information, when accessed, results in a machine: receiving color data representing a scene from a film; and using parameters representing a characteristic curve and color sensitivity of an original camera negative of the film, performing a color conversion operation to convert the color data to scene color data.
18. The article of claim 17 wherein converting the color data to scene colors comprises applying an international color consortium (ICC) profile to the received color data.
19. The article of claim 18 wherein the ICC profile comprises look-up tables (LUTs), wherein values of the LUTs are determined by a method comprising converting a printing density range to a Status M density range and converting the Status M density to log exposure using the characteristic curves of the original camera negative.
20. The article of claim 18 wherein the ICC profile comprises a matrix, wherein values of the matrix are determined by a method comprising: estimating values for the matrix based upon the color sensitivity of the original camera negative; applying the matrix estimated values to a training data set to provide estimated colormetric values; and adjusting the values of the matrix to minimize an error between the estimated colormetric values and known colormetric values.
21. An article including a machine-accessible medium having associated information, wherein the information, when accessed, results in a machine: converting a DPX color file, obtained from scanning a camera negative film used to capture a scene with a film camera, to scene color data using an ICC Color management engine, wherein the ICC Color management engine executes an ICC color profile created using parameters representing a characteristic curve and color sensitivity of the camera negative film.
22. The article of claim 21 wherein the ICC profile comprises look-up tables (LUTs), wherein values of the LUTs are determined by a method comprising converting a printing density range to a Status M density range and converting the Status M density to log exposure using the characteristic curves of the original camera negative film.
23. The article of claim 21 wherein the ICC profile comprises a matrix, wherein values of the matrix are determined by a method comprising: estimating values for the matrix based upon the color sensitivity of the camera negative film; applying the matrix estimated values to a training data set to provide estimated colormetric values; and adjusting the values of the matrix to minimize an error between the estimated colormetric values and known colormetric values.
24. An apparatus comprising: a transformation unit to converting a DPX color file, obtained from scanning a camera negative film used to capture a scene with a camera, to scene color data using an ICC Color management engine, wherein the ICC Color management engine executes an ICC color profile created using parameters representing a characteristic curve and color sensitivity of the camera negative film; and an editor unit to edit color data of the scene colors, wherein the transformation unit converts the edited color data to output color data.
25. The apparatus of claim 24 further comprising an ICC Profile unit to create the ICC Profile for execution by transformation unit.
26. The apparatus of claim 25 further comprising: a LUT values calculation unit to determine LUT values using characteristics of the camera negative film, the LUT values being used by the ICC Profile unit; and a matrix values calculation unit to determine matrix values using the sensitivities of the camera negative film, the matrix values being used by the ICC Profile unit.
PCT/US2006/042798 2005-11-01 2006-11-01 Motion picture content editing WO2007053745A2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN2006800464565A CN101401108B (en) 2005-11-01 2006-11-01 Data processing method of motion picture content
EP06827368A EP1949301A2 (en) 2005-11-01 2006-11-01 Motion picture content editing
JP2008539023A JP2009514483A (en) 2005-11-01 2006-11-01 Editing motion picture content

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/264,879 US7636469B2 (en) 2005-11-01 2005-11-01 Motion picture content editing
US11/264,879 2005-11-01

Publications (2)

Publication Number Publication Date
WO2007053745A2 true WO2007053745A2 (en) 2007-05-10
WO2007053745A3 WO2007053745A3 (en) 2008-08-07

Family

ID=37996354

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2006/042798 WO2007053745A2 (en) 2005-11-01 2006-11-01 Motion picture content editing

Country Status (5)

Country Link
US (1) US7636469B2 (en)
EP (1) EP1949301A2 (en)
JP (1) JP2009514483A (en)
CN (1) CN101401108B (en)
WO (1) WO2007053745A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7636469B2 (en) 2005-11-01 2009-12-22 Adobe Systems Incorporated Motion picture content editing

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7667880B2 (en) * 2006-02-10 2010-02-23 Adobe Systems, Incorporated Method and apparatus for previewing digital motion picture content
US20080007748A1 (en) * 2006-06-28 2008-01-10 Borg Lars U Compositing image data in a linear-light color space that preserves a desired look
US8144983B1 (en) * 2007-08-28 2012-03-27 Adobe Systems Incorporated Image editing workflow with color management
US8463109B2 (en) * 2008-01-07 2013-06-11 Black Mariah, Inc. Editing digital film
US20090207998A1 (en) * 2008-01-07 2009-08-20 Angus Wall Determining unique material identifier numbers using checksum values
EP2109304A1 (en) * 2008-04-07 2009-10-14 Océ-Technologies B.V. Color management method, module, and program product, and printer ussing said method
JP4506886B2 (en) * 2008-08-29 2010-07-21 ソニー株式会社 Video conversion apparatus, video conversion method, and program
TW201230817A (en) * 2010-10-15 2012-07-16 Thomson Licensing Method and system for producing video archive on film

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6028623A (en) * 1996-09-11 2000-02-22 U.S. Philips Corporation Film scanner with data interface
US6628823B1 (en) * 1997-03-24 2003-09-30 Jack M. Holm Pictorial digital image processing incorporating adjustments to compensate for dynamic range differences

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1126989C (en) * 1999-02-26 2003-11-05 贡晔 Digital exposure machine
CN1497327A (en) * 2002-10-11 2004-05-19 伊斯曼柯达公司 Camera system and method for coding image using filter
US7636469B2 (en) 2005-11-01 2009-12-22 Adobe Systems Incorporated Motion picture content editing

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6028623A (en) * 1996-09-11 2000-02-22 U.S. Philips Corporation Film scanner with data interface
US6628823B1 (en) * 1997-03-24 2003-09-30 Jack M. Holm Pictorial digital image processing incorporating adjustments to compensate for dynamic range differences

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7636469B2 (en) 2005-11-01 2009-12-22 Adobe Systems Incorporated Motion picture content editing

Also Published As

Publication number Publication date
US20070098256A1 (en) 2007-05-03
CN101401108B (en) 2011-12-21
CN101401108A (en) 2009-04-01
WO2007053745A3 (en) 2008-08-07
EP1949301A2 (en) 2008-07-30
US7636469B2 (en) 2009-12-22
JP2009514483A (en) 2009-04-02

Similar Documents

Publication Publication Date Title
US7636469B2 (en) Motion picture content editing
TWI580275B (en) Encoding, decoding, and representing high dynamic range images
US6285784B1 (en) Method of applying manipulations to an extended color gamut digital image
US8847976B2 (en) Converting a colorimetric transform from an input color space to an output color space
US7053927B2 (en) System for optimizing the display and rendering of digital images for digital mastering
JP5784541B2 (en) Video conversion apparatus, movie system shooting system using the same, video conversion method, and video conversion program
EP1237379A2 (en) Image processing for digital cinema
US20070058183A1 (en) Gamut compression method, program, and gamut compression device
JP2000152279A (en) System using one remaining image or over to represent digital image in expanded color area
JP4523193B2 (en) Extended color gamut digital image processing method and computer storage product
US20030035653A1 (en) Storage and processing service network for unrendered image data
US20080007748A1 (en) Compositing image data in a linear-light color space that preserves a desired look
JP4210397B2 (en) Method for operating on extended color gamut digital images
US20140035944A1 (en) Rendering and Un-Rendering Using Profile Replacement
US20060181721A1 (en) Motion picture content preview
Selan Cinematic color: From your monitor to the big screen
US20040042025A1 (en) Image processing method, image processing apparatus, image recording apparatus, program and recording medium
JP2001036762A (en) Device and method for preparing color correction definition
Okun et al. Post-Production/Image Manipulation
Krasser Post-Production/Image Manipulation
CN114693544A (en) Wide-color-gamut-based image toning method, device and storage medium
McElvain et al. Direct Display of Integer ACES Content for Post-Production Environments
JP2002016821A (en) Method and device for processing image and recording medium having recorded program for it
Bakke et al. Simulation of film media in motion picture production using a digital still camera
Kim et al. Colour correction and matching between scenes using 3D LUT

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200680046456.5

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application
ENP Entry into the national phase

Ref document number: 2008539023

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2006827368

Country of ref document: EP