CN101897194B

CN101897194B - System and method for generating and reproducing image file including 2D image and 3D stereoscopic image

Info

Publication number: CN101897194B
Application number: CN2008801200582A
Authority: CN
Inventors: 黄栖煐; 李健日; 宋在涓; 金容台
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2007-12-10
Filing date: 2008-12-05
Publication date: 2012-12-12
Anticipated expiration: 2028-12-05
Also published as: AU2008336448A1; ATE546017T1; JP2011509552A; CA2707428A1; CN101897194A; AU2008336448B2; KR101480186B1; JP5158727B2; KR20090060922A; ES2382057T3; CA2707428C

Abstract

An apparatus includes a storage unit to receive and store an image file, a processor to parse a media data field of the image file including one or more image data samples and to parse a media header field including an image type data field indicating whether each of the one or more image data samples is one of 2 dimensional (2D) image data and 3 dimensional (3D) stereoscopic image data to generate an image corresponding to one of a 2D image and a 3D stereoscopic image based on the image type data field of the image file, and a display unit to display the generated image according to the image type data field of the image file.

Description

Be used to produce and reproduce the system and method for the image file that comprises 2 dimension images and 3 dimension stereo-pictures

Technical field

The present invention relates to produce and reproduction comprises the system and method for the image file of two dimension (2D) image and three-dimensional (3D) stereo-picture based on 2D image media standard.More particularly, the present invention relates to alternately to produce and to reproduce the file format of 2D image and 3D stereo-picture and the system and method that use this document form comes alternately to produce and reproduce 2D image and 3D stereo-picture.

Background technology

The file format standard that is used to store the 2D image is being known in the art.Generally, the Motion Picture Experts Group (MPEG) as the International Standards Organization in multimedia field has issued MPEG-2, MPEG-4, MPEG-7 and MPEG-21 standard since its standardization first time to MPEG-1 in 1988.Because developed various standards, needs have been produced through a combination various criterion profile of technique for generating (profile).Need in response to this, carried out MPEG-A (MPEG uses: ISO/ICE 230000) multimedia application standardization activity and be used for storage and reproduce the 2D image.

Yet up to the present, the file format that is used to store the 3D stereo-picture is not as yet by standardization.In addition, in general portable terminal device, comprise 2D and 3D stereo-picture the two file format or use the structure of this class file format to produce and the system and method that reproduces this type of image is not implemented as yet.This is very important, because when the form with the 3D stereo-picture produced image file, the user had to the non-3D stereo-picture in the image file is watched as the 3D stereo-picture, thereby causes user's eye fatigue.Here, for example, this type of image can be that wherein entire image is disposed the image that forms by character.

Summary of the invention

One side of the present invention is to solve above-described at least problem and/or not enough and the advantage of the following stated at least is provided.Therefore, one side of the present invention provides the file format that is used to produce, store and reproduce the 3D stereo-picture.

Another aspect of the present invention provides based on the file format that is used for the 3D stereo-picture that is used to produce, store and reproduce the file format of existing 2D image.

Another aspect of the present invention provides the system and method that the file format that is used for the 3D stereo-picture through use produced and reproduced 3D stereo-picture file.

Particularly, the present invention provides and comprises that the two file format of 3D stereo-picture and 2D image is so that the user can watch 3D stereo-picture and 2D image according to this document form.According to file format of the present invention be provided in an image file storage 2D and 3D stereo-picture the two.For example, the 3D stereo-picture generally can be provided in a 3D stereo-picture that is used for news content, and for example, and the 2D image can be provided in only to comprise in the image of captions, thereby facilitates to the user.

According to one side of the present invention, a kind of device comprises: memory cell is used for receiving and memory image file; Processor; Be used to resolve the media data field of comprising of this image file of one or more view data samples; Whether and to resolve each that comprise in the said one or more view data samples of indication be one of them the medium header of image type data field of 2D view data and 3D stereoscopic image data, produces with 2 with the image type data field based on this image file and tie up (2D) images and 3 and tie up one of them corresponding image of (3D) stereo-picture; And display unit, be used for showing the image that is produced according to the image type data field of this image file.

According to another aspect of the present invention, a kind of computer implemented method comprises: receive image file; Resolve the media data field of comprising of this image file of one or more view data samples; Parsing comprises that whether indication each in said one or more view data samples is one of them the medium header of image type data field of 2 dimension (2D) view data and 3 dimension (3D) stereoscopic image datas; And produce and 2D image and one of them corresponding image of 3D stereo-picture based on the image type data field of this image file.

According to another aspect of the present invention, a kind of computer-readable medium, data structure of storage comprises on it: the media data field comprises one or more view data samples; And the medium header, comprise that whether indication each in said one or more view data samples is one of them image type data field of 2 dimension (2D) view data and 3 dimension (3D) stereoscopic image datas.

The following detailed description of example embodiment of the present invention is disclosed in conjunction with the drawings, other aspects of the present invention, advantage and characteristic for those skilled in the art with more than you know.

Description of drawings

Following description in conjunction with the drawings, above and other aspects of example embodiment of the present invention, feature and advantage are with more than you know, wherein:

Fig. 1 is the block diagram of explanation according to the storage format of the 2D image file of prior art;

Fig. 2 A is the block diagram of explanation according to the storage format of the image file of example embodiment of the present invention;

Fig. 2 B is the block diagram of explanation according to the storage format of the image file of another example embodiment of the present invention;

Fig. 2 C is the block diagram of explanation according to the storage format of the image file of another example embodiment of the present invention;

Fig. 2 D is the block diagram of explanation according to the storage format of the image file of another example embodiment of the present invention;

Fig. 2 E is the block diagram of explanation according to the storage format of the image file of another example embodiment of the present invention;

Fig. 2 F is the block diagram of explanation according to the storage format of the image file of another example embodiment of the present invention;

Fig. 2 G is the block diagram of explanation according to the storage format of the image file of another example embodiment of the present invention.

Fig. 2 H is the block diagram of explanation according to the storage format of the image file of another example embodiment of the present invention;

Fig. 3 is the block diagram of explanation according to the image file generation device of example embodiment of the present invention;

Fig. 4 is the block diagram of explanation according to the image file reproduction device of example embodiment of the present invention;

Fig. 5 is the flow chart of explanation according to the method for the generation image file of example embodiment of the present invention;

Fig. 6 is the flow chart of explanation according to the method for the reproduced image file of example embodiment of the present invention;

Fig. 7 is the flow chart of explanation according to the method for the reproduced image file of another example embodiment of the present invention;

Fig. 8 is the flow chart of explanation according to the method for the reproduced image file of another example embodiment of the present invention;

Fig. 9 is the flow chart of explanation according to the method for the reproduced image file of another example embodiment of the present invention;

Figure 10 is the flow chart of explanation according to the method for the reproduced image file of another example embodiment of the present invention; And

Figure 11 is the flow chart of explanation according to the method for realization random access of the present invention.

Run through accompanying drawing, should notice that similar reference number is used to represent same or similar assembly, characteristic and structure.

Embodiment

Provide following explanation to help to make much of the example embodiment of the present invention that limits claims and equivalent thereof with reference to accompanying drawing.It comprises and is used to help the various details understood, but should to be counted as only be demonstration for these.Therefore, those of ordinary skill in the art will recognize, can carry out various changes and modification to embodiment described here and not deviate from the spirit and scope of the present invention.

Before the form of describing according to example embodiment of the present invention that is used for three-dimensional (3D) stereo-picture of storage, with the storage format of describing based on two dimension (2D) image file of traditional International Standards Organization (ISO) standard.Fig. 1 is the block diagram of explanation based on the file format of the 2D image of traditional I SO 14496-12 standard.With reference to figure 1,2D image file format 100 comprises file area ftyp 110, moov zone 120 and the mdat zone 130 on upper strata.Mdat zone 130 is data areas of file format, and comprises the speech data 134 in actual image data 132 and the voice track 133 in the picture track 131.Each track is included in other view data of branch and the speech data of storing in the frame unit.

The header region of moov zone 120 and file format is corresponding and have an object-based structure.Moov zone 120 comprises reproduces the required full detail of file, comprise content information (as, frame rate, bit rate, picture size etc.) and be used to support the synchronizing information of the representational role of fast forward/back (FF/REW).Particularly, the information that moov zone 120 comprises such as the quantity of frame in view data and the speech data, the size of each frame etc., thus make and might recover and reproduced image data and speech data through resolving moov zone 120 at reproduction period.

Be different from prior art, example embodiment of the present invention comprise be provided for 2D and 3D stereo-picture the two image file storage format and use storage format of the present invention to produce and the system of reproduced image file.Particularly, example embodiment of the present invention is characterised in that, can each part of image file be embodied as the form of 2D image or 3D stereo-picture according to the characteristic of content.For example, in comprising the part of many characters, this part is shown as the eye fatigue that the 3D stereo-picture causes the user.Therefore, this part is stored and reproduction as the 2D image.The part that requires eurythmy or 3-D effect is stored and reproduction as the 3D stereo-picture.Thereby, realize being suitable for the form of image file of the characteristic of content.

Hereinafter, will the storage format that comprise the image file of 2D image and 3D stereo-picture according to being adapted for of example embodiment of the present invention be described with reference to figure 2A and 2B.As stated, according to example embodiment of the present invention, the

image file

201 and 202 that comprises 2D image and 3D stereo-picture comprises the frame (box) (that is field) that has about the information of the image file of relevant 2D image and 3D stereo-picture.

According to the present invention; Can be with comprising that the frame about the information of the image file of relevant 2D image and 3D stereo-picture directly inserts file area, moov zone or orbital region; Perhaps as the part of first frame; Perhaps can insert the sample bezel, cluster (like, " stbl " frame) of the information that comprises sample in the orbital region.Sample is meant the elementary cell that is used for partitioned image in file format, such as frame.

The storage format of Fig. 2 A explanation image file wherein is included in 3D stereo-picture file in the image streams.Shown in Fig. 2 A, the data structure 201 of image file comprises topmost paper zone Ftyp 210, the Moov zone 220 corresponding with header region, Mdata zone 240 and the metadata area 230 corresponding with the data area.Here, Mdata zone 240 comprises picture track 241 and voice track 245, and wherein image data storage is in picture track 241, and speech data is stored in voice track 245.Picture track 241 comprises first view data (note is done " 1 ") that is used for the 2D image and the second and the 3rd view data (note is done " 2 " and " 3 ") that is used for the 3D stereo-picture.Here, second view data and the 3rd view data can be respectively the left view view data and the right view view data of single object (subject).For example, if expression is shown by alternation sum that from the left view and the right view view data of the single object of left view and right view shooting then the user can see 3-D effect.

Fig. 2 A explains an example, and wherein each fragment 242,243 and 244 comprises the data sample of 3D stereoscopic image data, 2D view data and 3D stereoscopic image data respectively.Is 2D image or 3D stereo-picture according to the characteristic of each part of certain content with the sequential definition of view data.In addition, if confirmed to be stored in second view data of the 3D stereo-picture in

fragment

242 and 244 and the storage scheme of the 3rd view data in advance, then can produce according to any desired mode and the reproduced image file according to the present invention.For example, Fig. 2 A illustrates a kind of example of method, and wherein

fragment

242 and 244 comprises each second view data (that is, sample 2) and the 3rd view data (that is, sample 3) of alternately storage, and wherein each sample is a frame unit.Alternatively, a kind of scheme can be arranged, wherein second view data and the 3rd view data are stored as frame unit side by side, perhaps can view data be divided into small data so that be stored as frame unit with the mode that replaces.

The speech data 246,247 and 248 that is included in the voice track 245 is the speech datas that are respectively applied for each fragment 242,243 and 244.Speech data is synchronous with the view data of the fragment 242,243 that will reproduce and 244.

Moov zone 220 is corresponding with the header region of data structure, and comprises about the information 221 of picture track with about the information 222 of voice track.Information 221 about picture track comprises the general information that is used to reproduce file, comprises content information, such as frame per second, bit rate, picture size etc., and synchronizing information, be used for supporting representational role such as fast forward/back (FF/REW).Particularly, Moov zone 220 comprises the information such as the total quantity of the frame of speech data in view data in the picture track 241 and the voice track 245, the size of each frame etc.Therefore, might recover and reproduced image data and speech data through resolving Moov zone 220 at reproduction period.

The present invention includes a frame, it each frame that comprises that indication is produced by first view data, second view data and the 3rd view data is used for the identifying information of 2D image or 3D stereo-picture.Shown in Fig. 2 A, Moov zone 220 comprises frame 231, and each view data of storing in the frame unit in its presentation video file comprises that the view data that is used for the 2D image still is used for the view data of 3D stereo-picture.In example embodiment, can sign be distributed to each frame and setting so that the picture characteristics of expression frame.Identifying information for example comprises the information that is used for the 2D image and the number of fragments of the successive frame that is used for the 3D stereo-picture about comprising.Thereby, can use such information to recover and the reproduced image file with the form of 2D image or 3D stereo-picture.Illustration is recovered the explanation with the reproduced image file with the form of 2D image or 3D stereo-picture in the following table 1 and 2.

[table 1]

[table 2]

Like table 1 and 2; Shown in Fig. 2 A; First fragment 242 comprises second view data and the 3rd view data that is used for the 3D stereo-picture, and second fragment 243 comprises first view data that is used for the 2D image, and the 3rd fragment 244 comprises second view data and the 3rd view data that is used for the 3D stereo-picture.Here, sample count shown in identifying information 231 dial gauges 2 and sign (Stereo Flag).Therefore, can be that the view data that is stored in fragment 242,243 and 244 is recovered and reproduced to the 2D image or the information 231 of 3D stereo-picture through the image that is stored in the data structure 201 with reference to indication.Identifying information comprises the information of be used to decode second view data and the 3rd view data and is used for the information of synchronous second view data and the 3rd view data, and at reproduction period reference information 231.

To another example embodiment of the present invention be described with reference to figure 2B.Fig. 2 B is the block diagram of explanation according to the storage format of the 3D stereo-picture file of another example embodiment of the present invention.In the example embodiment shown in Fig. 2 B, two picture track are arranged but not a picture track shown in Fig. 2 A.The data structure 202 of 3D stereo-picture file comprises topmost paper zone Ftyp 250, the Moov zone 260 corresponding with header region, Mdata zone 280 and the metadata area 270 corresponding with the data area.For succinctly, do not repeat essentially identical explanation with Fig. 2 A.

Briefly; About the

information

261 and 262 of first picture track and second picture track, the

information

221 and 222 with Fig. 2 A is identical basically about the information 263 of voice track; Fragment 293,294 is identical with fragment 242,243 and 244 basically with 295; Comprise that speech data 290,291 is identical with the voice track 245 that comprises speech data 246,247 and 248 basically with 292 voice track 289, and information 272 is substantially the same with information 231.First picture track 281 comprise with from corresponding second view data 282 of the image of a view angle (like the left view image) and 284 and first view data 283 corresponding with the 2D image.Second picture track 285 comprise with from corresponding the 3rd view data 286 of the image of different view angle (like the right view image) and 288 and first view data 287 corresponding with the 2D image.Also promptly, left view image and right view image are stored in respectively in the pictures different track, and first view data 283 and first view data 287 are corresponding to identical image.Therefore; Except aforesaid information; Indicating image is that which of two 2D view data (that is, between first view data 283 and first view data 287) the 2D image or the identifying information 272 of 3D stereo-picture also comprise about and will be used to the information of 2D image.Also promptly, can confirm to be used to the view data of 2D image according to the information that is set to the master image track about which picture track between first picture track 281 and second picture track 285.

Table 3 expression is as the message box according to the standard of the storage format of the image file that is used to comprise 2D image and 3D stereo-picture of the present invention.Usually follow the basic media formats of ISO/IEC 14496-12 ISO and define said standard.

[table 3]

[definition]
	Box?Type：′svmi′
Container：Meta?Box(′meta′)or?Sample?Table?Box(‘stbl’)
	Mandatory：Yes
Quantity：Exactly?on
	?
[grammer]
	aligned(8)class StereoscopicVideoInformationBox extends FullBox(′svmi′，version＝0，0)

[0052]

{
	//stereoscopic visual type information
Unsigned int (8) stereoscopic_composition_type;
	Unsigned int (1) is_left_first;
Unsigned int (7) reserved;

//stereoscopic fragment information
	Unsigned int (1) is_all_stereo;
Unsigned int (7) reserved;

If (is_all_stereo==0)
	{
Unsigned int (1) is_main_media;
	Unsigned int (7) reserved;
Unsigned int (32) entry_count;

For (i=0; I＜entry_count; I++)
	{
Unsigned int (32) sample_count;
	Unsigned int (8) stereo_flag;
}
	}
}

[semanteme]
	Stereoscopic_composition_type: the frame form of the composition of stereoscopic video content (0: side by side, 1: vertical row is staggered, and 2: frame is continuous, and 3: single as left image, 4: single as right image)
Is_left_first: represent that which is at first encoded between left image and the right image
	Is_all_stereo: the fragment in the expression ES is whole three-dimensional fragments (0: single picture, 1: solid)
Is_main_media: the list in the expression ES is main medium (0: sub-medium, 1: main medium) as content
	Entry_count: the number that comprises the fragment of sample with successive value
Sample_count: number with sample of successive value
	Stereo_flag: the expression present frame is solid or single picture (0: single picture, 1: solid)

Image information frame (like, " svmi " frame) can be storage about the solid/list of each sample that comprises in the image file (ES) frame as information.Therefore, the container that comprises image information frame (like, " svmi " frame) can be first frame or sample bezel, cluster (like, " stbl " frame).Container is meant the more high-rise frame that comprises current frame.Therefore, comprise that the container of image information frame (like, " svmi " frame) can be the metadata area 270 shown in Fig. 2 B and the following table 4A that illustrates, and also can be included in the sample bezel, cluster shown in following table 4B (like, " stbl " frame).Therefore, be illustrated in the part that increases newly in the container in the table 3,6,8,9 shown in below.According to the present invention, the container that comprises image information frame (like, " svmi " frame) is meant first frame or sample bezel, cluster (like, " stbl " frame)., should be appreciated that it can be free to travel in the more suitably position about any one table in the frame of ISO/IEC 14496-12ISO base media file form.

[table 4A]

meta	?	?	?	metadata
					?	Hdlr	?	?	handler
?	Dinf	?	?	data?information?box
					?	?	dref	?	dta?reference?box
?	ipmc	?	?	PMP?control?box
					?	iloc	?	?	item?location
?	ipro	?	?	item?protection
					?	?	sinf	?	protection?scheme?information?box
?	?	?	frma	original?format?box
					?	?	?	imif	IPMP?information?box
?	?	?	schm	scheme?type?box
					?	?	?	schi	scheme?information?box
?	linf	?	?	item?information
					?	xml	?	?	XML?container
?	bxml	?	?	binary?XML?container
					?	pitm	?	?	primary?item?reference
?	svmi	?	?	stereoscopic video media information box

Fig. 4 B representes, wherein image information frame (like, " svmi " frame) can be inserted into the form of the frame of son " stbl " frame container, for example, and in the file structure of ISO/IEC 23000-11 three-dimensional video-frequency application format.In addition, can be included in file area, moov zone or the orbital region according to each image information frame of example embodiment of the present invention (like, " svmi " frame).

[table 4B]

Ftyp

?

file?type?and?compatibility

pdin

?

Progressive?download?Information

[0059]?

moov

?

container?for?all?the?metadata

?

mvhd

?

movie?header，overall?declarations

?

trak

?

container for an individual track or stream

?

tkhd

?

track header，overall information about the track

?

tref

?

track?reference?container

?

edts

?

edit?list?container

?

elst

?

an?edit?list

?

mdia

?

container for the media information in a track

?

mdhd

?

media header，overall information about the media

?

hdlr

?

handler，declares?the?media(handler)type

?

minf

?

media?information?container

?

vmhd

?

video media header，overall information (video track only)

?

smhd

?

sound media header，overall information(sound track only)

?

hmhd

?

hint?media?header，overall?information(hint track?only)

?

nmhd

?

Null media header，overall information(some tracks only)

?

dinf

?

data?information?box，container

?

dref

data?reference?box，declares?soure(s)of media?data?in?track

?

stbl

?

sample table box，container for the time/space map

?

stsd

sample?descriptions(codec?types， initialization?etc.)

?

stts

(decoding)time-to-sample

?

stsc

sample-to-chunk，partial data-offset information

?

stsz

sample?sizes(framing)

?

stz2

compact?sample?sizes(framing)

?

stco

chunk offset，partial data-offset information

?

co64

64-bit?chunk?offset

?

stss

sync?sample?table(random?access?point)

[0060]?

?

svmi

stereoscopic?video?media?information

?

ipmc

?

IPMP?Control?Box

mdat

?

media?data?container

meta

?

metadata

?

hdlr

?

handler，declares?the?metadata(handler) type

?

iloc

?

item?location

?

iinf

?

item?information

?

xml

?

XML?container

?

bxml

?

binary?XML?container

?

scdi

?

stereoscopic camera and display information

Fig. 2 B is to the storage format of 2F explanation according to the image file of various example embodiment of the present invention.

Fig. 2 B explain wherein 3D stereo-picture file comprise two image streams (as; Left image of in the image streams of separating, storing and right image) the storage format of situation hypograph file; Wherein the frame according to the information of the 2D of comprising image of the present invention and 3D stereo-picture is added into moov zone (that is, Moov 260).

Fig. 2 C explains that 3D stereo-picture file wherein comprises the storage format of the situation hypograph file of an image streams, and wherein the frame according to the information of the 2D of comprising image of the present invention and 3D stereo-picture is added into file area (that is, Ftype 210).

Fig. 2 D explain wherein 3D stereo-picture file comprise two image streams (as; Left image of in the image streams of separating, storing and right image) the storage format of situation hypograph file; Wherein the frame according to the information of the 2D of comprising image of the present invention and 3D stereo-picture is added into file area (that is, Ftype250).

Fig. 2 E explains that 3D stereo-picture file wherein comprises the storage format of the situation hypograph file of an image streams, and wherein the frame according to the information of the 2D of comprising image of the present invention and 3D stereo-picture is added into orbital region (that is, track 221).

Fig. 2 F explain wherein 3D stereo-picture file comprise two image streams (as; Left image of in the image streams of separating, storing and right image) the storage format of situation hypograph file; Wherein the frame according to the information of the 2D of comprising image of the present invention and 3D stereo-picture is added into each orbital region (that is, track 261 and track 262).

Fig. 2 A explains example embodiment of the present invention to 2F; Comprising about comprise 2D image and 3D stereo-picture the two image file information the image information frame (as; " svmi " frame) is included in first frame, thereby is inserted into file area, moov zone and orbital region.

Fig. 2 G and 2H explanation example embodiment; Comprising about comprise 2D image and 3D stereo-picture the two image file information the image information frame (as; " svmi " frame) is inserted into the sample bezel, cluster (like, " stbl " frame) of the sample information that comprises image file in the orbital region.For the purpose of explaining, sample is meant the elementary cell that is used for partitioned image in file format, such as frame.

Fig. 2 G explains that 3D stereo-picture file wherein comprises the storage format of the situation hypograph file of an image streams, wherein is added into sample bezel, cluster (like, " stbl " frame) according to the frame of the information of the 2D of comprising image of the present invention and 3D stereo-picture.

Fig. 2 H explain wherein 3D stereo-picture file comprise two image streams (as; Left image of in the image streams of separating, storing and right image) the storage format of situation hypograph file; Wherein be added into sample bezel, cluster (like, " stbl " frame) according to the frame of the information of the 2D of comprising image of the present invention and 3D stereo-picture.

To shown in the 2H, image information frame (like, " svmi " frame) can be added to the file layers and the rail layer of conventional images file format like Fig. 2 A, but not the moov layer, thereby can produce image files through various image file formats.

Hereinafter, the example embodiment of the replacement of the present invention that introduction and table 2 is different with 3 example embodiment and the new image information frame of revising (as, " svmi " frame).

Content comprises that the syntax and semantics of back image information frame is revised in description as shown in table 6 in the example embodiment under the two the situation of 2D image and 3D stereo-picture therein.

[table 5]

Type	S1 (solid)	S2 (solid)	S3 (solid)	M1 (single picture)	M2 (single picture)	S4 (solid)
							The quantity of frame	100	100	100	200	300	100

[table 6]

[definition]
	Box?Type：′svmi′
Container：Meta?Box(′meta′)or?Sample?Table?Box(′stbl′)
	Mandatory：Yes
Quantity：Exactly?one
	?
[grammer]
	aligned(8)class StereoscopicVideoInformationBox extends FullBox(′svmi′， version＝0，0)

[0078]

{
	//stereoscopic visual type information
Unsigned int (8) stereoscopic_composition_type;
	Unsigned int (1) is_left_first;
Unsigned int (7) reserved;

//stereoscopic fragment_information
	Unsigned int (32) entry_count;

	For (i=0; I＜entry_count; I++)
{
	Unsigned int (32) sample_count;
Unsigned int (1) stereo_flag;
	Unsigned int (7) reserved;
Unsigned int (32) item_count;
	}
}

[semanteme]
	Stereoscopic_composition_type: the frame structure types of stereoscopic video content (0: side by side, 1: vertical row is staggered, and 2: frame is continuous, and 3: single as left image, 4: single as right image)
Is_left_first: represent that which is at first encoded between left image and the right image
	Entry_count: wherein clip types is from solid to single picture or from the number of the fragment that singly looks like the solid change
Sample_count: number with sample of successive value
	Stereo_flag: the expression present frame is solid or single picture (0: single picture, 1: solid)
Item_count: the number that is included in the fragment in the clauses and subclauses

The grammer that uses table 6 is table 7 with the content representation of table 5.

[table 7]

If Entry_Count is defined as the semanteme of table 3, then there is the problem of the fragment structure that can't discern in the current clauses and subclauses.Therefore, in example embodiment, comprise the syntax values of Item_Count so that address the above problem.Also promptly, when according to the semantical definition Entry_Count of table 6, only distinguish three-dimensionally when being first value of statistical indicant for first still single picture when content comprises, just can save Stereo_Flag, it is as giving a definition.

[table 8]

[definition]
	Box?Type：′svmi′
Container：Meta?Box(′meta′)or?Sample?Table?Box(′stbl′)
	Mandatory：Yes
Quantity：Exactly?one
	?
[grammer]
	aligned(8)class StereoscopicVideoInformationBox extends FullBox(′svmi′， version＝0，0)
{
	?//stereoscopic?visual?type?information
?unsigned?int(8)stereoscopic_composition_type；
	?unsigned?int(1)is_left_first；
?unsigned?int(7)reserved；
	?
?//stereoscopic?fragment?information
	?unsigned?int(32)entry?count；
?unsigned?int(1)is_stereo_first；
	?unsigned?int(7)reserved；
?
	?for(i＝0；i＜entry_count；i++)
?{
	?unsigned?int(32)sample_count；
?unsigned?int(32)item_count；
	?}
}
	?
[semanteme]
	Stereoscopic_composition_type: the frame structure types of stereoscopic video content (0: side by side, 1: vertical

[0085]

Line interlacing, 2: frame is continuous, and 3: single as left image, 4: single as right image)
	Is_left_first: represent that which is at first encoded between left image and the right image
Entry_count: wherein clip types is from solid to single picture or from the number of the fragment that singly looks like the solid change
	Is_stereo_flag: the image that is illustrated in which kind of type in the content of mixing three-dimensional and single picture is at first illustrated (0: singly look like at first, 1: solid at first)
Sample_count: number with sample of successive value
	Item_count: the number that is included in the fragment in the clauses and subclauses

If the value of image sequence information (is_stereo_first) is 1; Then content press S → M → S → M → ... order construct; Wherein " S " representes three-dimensional and the single picture of " M " expression; And if the value of image sequence information (is_stereo_first) is 0, then content press M → S → M → S → ... order construct.

In another example embodiment, the sample_count grammer is disallowable.In this case, can discern each fragment is solid or single picture, but the frame that can't discern how many numbers is three-dimensional or single picture.Therefore, can use the value of grammer of sub-frame of grammer and the sample bezel, cluster (like, " stbl " frame) of the item location frame that defines in the ISO media file format to confirm the number of solid or single picture frame.

[table 9]

[definition]
	Box?Type：′svmi′
Container：Meta?Box(′meta′)or?Sample?Table?Box(′stbl′)
	Mandatory：Yes
Quantity：Exactly?one
	?
[grammer]
	aligned(8)class StereoscopicVideoInformationBox extends FullBox(′svmi′， version＝0，0)
{
	?//stereoscopic?visual?type?information
?unsigned?int(8)stereoscopic_composition_type；
	?unsigned?int(1)is_left_first；
?unsigned?int(7)reserved；
	?
?//stereoscopic?fragment?information
	?unsigned?int(32)entry_count；
?
	?for(i＝0；i＜entry_count；i++)

[0090]

{
	Unsigned int (1) stereo_flag;
Unsigned int (1) reserved;
	}
}

[semanteme]
	Stereoscopic_composition_type: the frame structure types of stereoscopic video content (0: side by side, 1: vertical row is staggered, and 2: frame is continuous, and 3: single as left image, 4: single as right image)
Is_left_first: represent that which is at first encoded between left image and the right image
	The number of fragment in the entry_count:ES
Stereo_flag: the expression present frame is solid or single picture (0: single picture, 1: solid)

In addition, when the random access to content was carried out at the terminal, it moved to the position of expectation when the value that starts anew to use sample-sized frame (like " stsz " frame) reads the size of frame in proper order.If the sample count syntax values of the image information frame that defines among use the present invention (like, " svmi " frame) then can be discerned the number of the frame of each fragment in the item location frame (like " iloc " frame) and the start address and the size of each fragment.Therefore, use those values more effectively to accomplish random access to the precalculated position.

Then, data structure 201 to 208 that describe to use Fig. 2 A to the image file shown in the 2H is produced the demonstration system with the reproduced image file.This system generally comprises image file generation device and image file reproduction device.At first, will be with reference to the image file generation device of figure 3 descriptions according to example embodiment of the present invention.

As shown in Figure 3, image file generation device according to the present invention comprises first camera 311, second camera 312, input unit 320, image signal processing unit 330, memory cell 340, coding unit 350 and file generation unit 360.First camera 311 is from left view or right view reference object and then export second view data.Second camera 312 is also then exported the 3rd view data from the view reference object that is different from first camera 311.Should be appreciated that, can use a plurality of views of different angles and do not depart from the scope of the present invention.First view data that then, will be used for the 2D image is imported through input unit 320 with second view data and the 3rd view data.

Come preliminary treatment first view data, second view data and the 3rd view data through image signal processing unit 330.Here, pretreatment operation comprises and converts simulation external image value (that is the light that, is for example produced by charge-coupled device (CCD) or complementary metal oxide semiconductors (CMOS) (CMOS) type transducer and the analogue value of color component) into digital value.

Image signal processing unit 330 pretreated first view data, second view data and the 3rd view data are passed through in memory cell 340 storages, and the image stored data are offered coding unit 350.Fig. 3 illustrates memory cell 340, but the storage construct that is used for the buffering between the assembly shown in Fig. 3 that can comprise is not shown separately.Coding unit 350 codings are from first view data, second view data and the 3rd view data of memory cell 340.The encoding operation of being carried out by coding unit 350 is the coding of data, can skip because of the occasion demand.

File generation unit 360 produces image file 370 through using first view data, second view data and the 3rd view data by coding unit 350 codings.In this case; With first view data, second view data and the 3rd image data storage in the data area (as; The mdata zone) in; And the information stores that will be used to produce first view data (being the 2D image) and second view data and the 3rd view data (being the 3D stereo-picture) header region (as, moov zone and metadata area) in.The image file 370 that is produced is transfused to and sends to the stereo-picture file reproduction device, and then image file reproduction device generation and reproduction are from the 2D image and the 3D stereo-picture of image file 370.Hereinafter, with the image file reproduction device of describing demonstration.

Fig. 4 is the block diagram of explanation according to the image file reproduction device of example embodiment of the present invention.As shown in Figure 4, image file reproduction device comprises document analysis unit 420, decoding unit 430, memory cell 440, reproduction units 450 and display unit 460.

Document analysis unit 420 receptions and for example resolve the image file 410 (like the image file 370 of Fig. 3) that the file generation unit 360 by the image file generation device produces.In this case, the information that is stored in respectively in moov zone and the metadata area is resolved in document analysis unit 420, extracts first view data, second view data and the 3rd view data that is stored in the mdata zone then.

First view data, second view data and the 3rd view data that decoding unit 430 decodings are extracted.In example embodiment, carry out decoding under the situation of the coding unit of image file generation device use therein 350 coded datas.Also promptly, if the file generation device is skipped coding, then file reproduction device is skipped decoding.Subsequently, with decoded storage in memory cell 440.

Reproduction units 450 reproduces the 2D image that first view data from be stored in memory cell 440 produces, and according to second view data and the three view data synthetic 3D stereo-picture of identifying information from be stored in memory cell 440.Then, display unit 460 shows 2D image and the 3D stereo-picture that is reproduced.Display unit 460 can adopt and block (barrier) LCD (LCD).In example embodiment, if the fragment of image file be the 2D image then close and block LCD, if the fragment of image file be the 3D stereo-picture then open and block LCD, thereby display image correctly.

Then, use according to the data structure generation of image file of the present invention and the demonstration methods of reproduced image file describing.

Fig. 5 is the flow chart of explanation according to the method for the generation image file of example embodiment of the present invention.As shown in Figure 5, this method comprises that input step S510, pre-treatment step S520, coding step S530 and file produce step S540.

At step S510, input is used to second view data and the 3rd view data that produce first view data of 2D image and be used to produce the 3D stereo-picture.For example, from left view and/or right view reference object and export second view data and the 3rd view data.At step S520, first view data that preliminary treatment is imported in step S510, second view data and the 3rd view data, and will convert digital value into from the analogue value by the view data that CCD or CMOS type transducer produce.At step S530, according to predetermined encoding scheme coding process pretreated first view data, second view data and the 3rd view data.Can answer occasion to need skips steps S530.At step S540, produce image file through first view data, second view data and the 3rd view data of using coding in coding unit 350.In this case, can produce image file to any one of the data structure of the image file described in the 2H according to Fig. 2 A.

Fig. 6 is the flow chart of explanation according to the method for the reproduced image file of example embodiment of the present invention.As shown in Figure 6, the method that is used for the reproduced image file comprises document analysis step S610, decoding step S620, reproduces step S630 and step display S640.

At step S610, be stored in the moov zone of the image file that produces according to the present invention and the information in the metadata area is extracted first view data, second view data and the 3rd view data through use.Particularly, use above-mentioned identifying information to extract view data.At step S620, first view data of decoding, second view data and the 3rd view data.If when producing image file, skip coding step, then decoding step S620 is also skipped.At step S630, will synthesize the 2D image and the 3D stereo-picture that will reproduce in first view data, second view data and the 3rd view data of step S620 decoding.Then, at step S640, on display unit 460, be presented at 2D image and 3D stereo-picture that step S630 produces.Equally, display unit 460 can adopt and block LCD, if wherein the fragment of image file is the 2D image then closes and block LCD, and if the fragment of image file be the 3D stereo-picture then open and block LCD, thereby display image correctly.

Fig. 7 to 10 explanation is according to the operation from the terminal that is resolved to the reproduced image file of various example embodiment of the present invention.

Fig. 7 describes and comprises the example embodiment of resolving and reproducing the image file that produces according to the present invention.The embodiment of Fig. 7 relates to the image information frame that comprises shown in the table 3 image file format of (like, " svmi " frame).Image information frame (like, " svmi " frame) comprises a plurality of fields.Each frame that the function of tonic chord of said field provides the indicating image file is the information of 2D image or 3D stereo-picture, and this information provides value of statistical indicant to be used to control the activation of display (like LCD) or do not activate.

As shown in Figure 7, resolve the file frame (like " ftyp " frame) in the image file at step S710.In example embodiment, ftyp is provided frame according to traditional I SO/IEC 14496-12 standard.In S740, resolve moov frame, track box and first frame of image file at step S720 respectively.In example embodiment, also can moov frame and track box be provided according to traditional I SO/IEC 14496-12 standard.At step S750, each field of resolving image information frame according to image file of the present invention (like, " svmi " frame) is 2D image or 3D stereo-picture with what confirm that each frame in the picture track comprises.This information spinner will provide through the field of sample_count and entry_count.

Here, entry_count is meant the number of fragment in the image file.For example, 6 images can be according to following sequential storage in image file: 3D stereo-picture (1), 3D stereo-picture (2), 3D stereo-picture (3), 2D image (1), 2D image (2) and another 3D stereo-picture (4).In this example, each image is called as fragment., the unit that should be appreciated that fragment can be frame, have a framing of successive value, or so as to dividing the interval of 3D stereo-picture and 2D image.Sample_count is meant the number that is included in the successive frame in each fragment.

Therefore, identification entry_count is with the number of fragment in definite image, and identification sample_count is to confirm to be included in the total number of the frame in each fragment.Then, identification stereo_flag, and identification is included in the flag information of the set of the frame in the present frame (promptly corresponding fragment).Through flag information, can confirm that corresponding fragment is 3D stereo-picture or 2D image.Then, in step S760 according to each frame of having discerned of the form of 3D stereo-picture or 2D image decoding.

In step S770, according to the resolving information of the interior stereo_flag of image information frame (like, " svmi " frame), control is blocked LCD so that if value is " 1 ", then activates and block LCD, and if value is " 0 " then not activate and block LCD.Also be; Under the situation of 3D stereo-picture, value that can stereo_flag is set to " 1 " so that activate and block LCD, and under the situation of 2D image; Value that can stereo_flag is set to " 0 " so that do not activate and block LCD, makes that thus to block LCD controlled.Simultaneously, activate or the unactivated LCD of blocking on reproduce and show that frame through decoding is so that the user can see image.

Fig. 8 is the flow chart of explanation according to the method for the reproduced image file of another example embodiment of the present invention.As shown in Figure 8, in step S810, resolve the file frame (like " ftyp " frame) in the image file.Then, in step S820, S830 and S840, resolve moov frame, track box and first frame of image file respectively.Then, in step S850, each field of resolving image information frame according to image file of the present invention (like, " svmi " frame) is 2D image or 3D stereo-picture with what confirm that each frame in the picture track comprises.

Entry_count in this example embodiment is different from last example embodiment.The entry_count of this example embodiment is meant that the type of fragment (being 2D or 3D) wherein is from solid to single picture or from the number of the fragment that singly looks like the solid change.Use previous image file example; Even if 6 images of 3D stereo-picture (1), 3D stereo-picture (2), 3D stereo-picture (3), 2D image (1), 2D image (2) and 3D stereo-picture (4) are included in the single image file successively, also divide this image based on the change of the type of 3D stereo-picture and 2D image.Therefore, entry_count is 3 (that is, 3D stereo-picture (1)-(3), 2D image (1)-(2) and 3D stereo-pictures (4)).Identification entry_count is with the number of fragment in definite image, and identification sample_count is to confirm to be included in the total number of the frame in each fragment.Then, identification stereo_flag, and discern flag information about the set that is included in the frame in the present frame (promptly corresponding fragment).Through this flag information, can confirm that corresponding fragment is 3D stereo-picture or 2D image.Then, identification item_count is so that be identified in the number of each clauses and subclauses of the image of discerning among the entry_xount interior (in each interval of three-dimensional and single picture) fragment.The unit of fragment can be frame, have a framing of successive value, or so as to dividing the interval of 3D stereo-picture and 2D image.The step (step S860 and S870) of decoding and display image is identical with the operation at the terminal of last example embodiment shown in Figure 7.

Fig. 9 is the flow chart of explanation according to the method for the parsing of another example embodiment of the present invention and reproduced image file.As shown in Figure 9, resolve the file frame (like " ftyp " frame) in the image file at step S910.Then, resolve moov frame, track box and first frame of image file respectively at step S920, S930 and S940.Then, in step S950, each field of resolving image information frame according to image file of the present invention (like, " svmi " frame) is 2D image or 3D stereo-picture with what confirm that each frame in the picture track comprises.

Identical in entry_count in this example embodiment and the example embodiment of Fig. 8.Also promptly, the entry_count in this example embodiment also be meant fragment wherein type from solid to single picture or from the number of the fragment that singly looks like the solid change.Identification entry_count, the number of fragment in the recognition image, and recognition coding sequence (is_left_first), thus which image spacing is at first constructed in the image of correspondence between identification 3D stereo-picture and the 2D image.For example, the value of coded sequence (is_left_first) can be set to " 1 " with instruction content according to S → M → S → M sequence arrangement, and this value can be set to " 0 " and comes instruction content according to M → S → M → S sequence arrangement.Then, identification sample_count is to confirm to be included in the total number of the frame in each fragment.Then, thus the number of identification item_count identification (each of three-dimensional and single picture is interior at interval) fragment in each clauses and subclauses of the image of entry_count identification.The unit of fragment can be frame, have a framing of successive value, or so as to dividing the interval of 3D stereo-picture and 2D image.Then, in step S960, be 3D stereo-picture or 2D image with the frame decoding of each identification.Then, utilize the information that obtains as the coded sequence in the message box (is_left_first) through analysis diagram to control and block LCD.In step S970, thus activate or the unactivated LCD of blocking on reproduce and show that decoded frame allows the user to watch image.

Figure 10 is the flow chart of explanation according to the method for the reproduced image file of another example embodiment of the present invention.Shown in figure 10, resolve the file frame (like " ftyp " frame) in the image file at step S1010.Then, resolve moov frame, track box and first frame of image file respectively at step S1020, S1030 and S1040.Then, at step S1050, each field of resolving image information frame according to image file of the present invention (like, " svmi " frame) is 2D image or 3D stereo-picture with what confirm that each frame in the picture track comprises.

Identical in entry_count in current example embodiment and the example embodiment of Fig. 7, wherein it refers to the number of fragment in the image file.Use previous image file example; Even if 6 images of 3D stereo-picture (1), 3D stereo-picture (2), 3D stereo-picture (3), 2D image (1), 2D image (2) and 3D stereo-picture (4) are included in the single image file successively; Each image also comprises a plurality of frames, and wherein each image is called as fragment.As stated, the unit of fragment can be frame, have a framing of successive value, or so as to dividing the interval of 3D stereo-picture and 2D image.Sample_count is meant the number that is included in the successive frame in each fragment.Identification entry_count is with the number of fragment in definite image, and identification stereo_flag, and identification is included in the flag information of the set of the frame in each fragment.Through this flag information, can confirm that corresponding fragment is 3D stereo-picture or 2D image.Then, in the start address and the size of step S1060 identification item location frame (like " iloc " frame), identification fragment, and the sample-sized in the recognition sample size frame (like " stsz " frame), thereby discern how many frames be included in each fragment.The step (step S1070 and S1080) of decoding and display image is consistent with the operation at the terminal of first example embodiment shown in Figure 7.

Figure 11 is the flow chart of explanation according to the method for the random access of realization image file of the present invention.Figure 11 explanation wherein during decoding and reproduced image (such as when play bar reproducing be moved to that expectation watches in one hour during the image time the district image on the time) operation at the generation random access terminal of ordering.

At step S1100, stab information from the frame recognition time that comprises timestamp (like " TimeStamp ") information, so that the frame (that is, random access point, hereinafter referred to as " RAP ") of random access is wanted in identification.At step S1100, the entry_count of recognition image message box (like, " svmi " frame), and the number of fragment in the recognition image.Here, the unit of fragment can be frame, have a framing of successive value, or so as to dividing the interval of 3D stereo-picture and 2D image.In step S1120 and step S1130, discern fragment and identification item location frame (like " iloc " frame) that sample_count, identification comprise RAP respectively, thus the start address through the corresponding fragment of this information (such as the skew of the fragment of correspondence) identification.In step S1140 and step S1150 respectively the sample_size in the recognition sample size frame (like " stsz " frame) and from item location frame (like " iloc " frame) thus the start address of fragment of the correspondence discerned add that one by one sample_size finds RAP.Then, in the decoding of step S1160 according to random access order beginning RAP.If produced the random access order, then conventional method is confirmed RAP through the sample_size that calculates entire image.Yet,, only need calculate the sample in the fragment that comprises RAP according to the present invention.

Figure 11 only explains the example embodiment of using entry_count and sample_count., should be appreciated that random access operation of the present invention goes for other example embodiment.In addition, depend on the example embodiment in the scope of the logic flow of the operation at terminal, can add or reject step of being used to discern item_count etc. and do not deviate from scope of the present invention.In addition, the details of the operation at interpretation of images message box (like, " svmi " frame) or terminal can depend on changes such as the position of image information frame, the position of frame intrinsic parameter and not deviate from scope of the present invention.

Here do not have the details of the specifically described step that is used for the resolution file form and the operation at terminal to realize, and can combine various embodiment of the present invention to use based on ISO/IEC 14496-12 and ISO/IEC 23000-11 standard.

As stated, the present invention has defined the data structure of image file, and the two simplifies the proof procedure that is used as new standard thereby the standard technique of the change that it can be through using the 2D image comprises 2D image and 3D stereo-picture.Therefore, the present invention allow in the single image file, to realize in case of necessity 2D image and 3D stereo-picture the two.Particularly, use system and method permission demonstration not require that the image of watching with the form of 3D stereo-picture (that is the 2D image in the image file of demonstration 3D stereo-picture) is so that avoid user's eye fatigue according to image file format of the present invention.In addition, the present invention has such advantage, wherein can block opening or closing and reproducing the content of having mixed 3D stereo-picture and 2D image effectively of LCD through what use image information frame (like, " svmi " frame) control terminal.

Although illustrate and describe the present invention with reference to concrete example embodiment of the present invention; But it will be apparent to those skilled in the art that and to carry out various changes in form and details here and do not deviate from the spirit and scope of the present invention that limit accompanying claims and equivalent thereof.

Claims

1. computer implemented method comprises:

Receive image file;

Resolve the media data field of comprising of this image file of one or more view data samples;

Parsing comprises that whether indication each in said one or more view data samples is one of them the medium header of image type data field of 2 dimension (2D) view data and 3 dimension (3D) stereoscopic image datas; Wherein the image type data field comprises the sample counting field, the digital counting of the view data sample of same type in the indicator sequence; And

Image type data field based on this image file produces and 2D image and one of them corresponding image of 3D stereo-picture.

2. computer implemented method as claimed in claim 1, wherein the medium header comprises the track field, and the image type data field is arranged in the track field.

3. computer implemented method as claimed in claim 2, its middle orbit field comprises the rail layer metadata fields, and the image type data field is arranged in the rail layer metadata fields.

4. computer implemented method as claimed in claim 2, its middle orbit field comprises the schedule of samples field, and the image type data field is arranged in the schedule of samples field.

5. computer implemented method as claimed in claim 1, wherein the image type data field comprises in said one or more view data samples each is appointed as one of them sign of 2D view data and 3D stereoscopic image data.

6. computer implemented method as claimed in claim 5 also comprises:

Indicate to control based on this and block LCD (LCD), if this sign is set to the 3D stereoscopic image data then opens, if this sign is set to the 2D view data then closes.

7. computer implemented method as claimed in claim 1, wherein the image type data field comprises the clauses and subclauses count area, indication is as the digital counting of the transformation between the view data sample of 2D view data and 3D stereoscopic image data.

8. computer implemented method as claimed in claim 1, wherein the media data field comprises at least one picture track.

9. computer implemented method as claimed in claim 8, wherein picture track comprises a plurality of 3D stereoscopic image data samples, each in these a plurality of 3D stereoscopic image data samples is represented the image of different views angle.

10. computer implemented method as claimed in claim 9; In wherein said a plurality of 3D stereoscopic image data sample each represent left view image and right view image one of them, and said a plurality of 3D stereoscopic image data sample is alternately arranged between left view image and right view image.

11. computer implemented method as claimed in claim 10, each in wherein said a plurality of 3D stereoscopic image data samples is represented a frame.

12. computer implemented method as claimed in claim 10 is represented a frame for two in wherein said a plurality of 3D stereoscopic image data samples, so that each frame is made up of left view image and the right view image arranged in a side-by-side manner.

13. computer implemented method as claimed in claim 8, wherein the media data field comprises first picture track and second picture track.

14. computer implemented method as claimed in claim 13, wherein

First picture track comprises a plurality of 3D stereoscopic image data samples, and each in these a plurality of 3D stereoscopic image data samples is represented the left view angular image, and

Second picture track comprises a plurality of 3D stereoscopic image data samples, and each in these a plurality of 3D stereoscopic image data samples is represented the right view angular image.

15. a computer implemented device comprises:

Be used to receive the device of image file;

Be used to resolve the device of the media data field of comprising of this image file of one or more view data samples;

Be used for resolving each that comprise the said one or more view data samples of indication and whether be one of them the device of medium header of image type data field of 2 dimension (2D) view data and 3 dimension (3D) stereoscopic image datas; Wherein the image type data field comprises the sample counting field, the digital counting of the view data sample of same type in the indicator sequence; And

Be used for based on the image type data field generation of this image file and the device of 2D image and one of them corresponding image of 3D stereo-picture.

16. computer implemented device as claimed in claim 15, wherein the image type data field comprises in said one or more view data samples each is appointed as one of them sign of 2D view data and 3D stereoscopic image data.

17. computer implemented device as claimed in claim 16; Further comprise and be used for indicating and control the device that blocks LCD (LCD) based on this; If this sign is set to the 3D stereoscopic image data then opens and block LCD, if this sign is set to the 2D view data then closes and block LCD.