CN100440117C - Large screen non contact type control mode - Google Patents

Large screen non contact type control mode Download PDF

Info

Publication number
CN100440117C
CN100440117C CNB031079008A CN03107900A CN100440117C CN 100440117 C CN100440117 C CN 100440117C CN B031079008 A CNB031079008 A CN B031079008A CN 03107900 A CN03107900 A CN 03107900A CN 100440117 C CN100440117 C CN 100440117C
Authority
CN
China
Prior art keywords
screen
giant
laser
control mode
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CNB031079008A
Other languages
Chinese (zh)
Other versions
CN1534544A (en
Inventor
彭海良
许光銮
周强
洪隆昌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Electronics of CAS
Original Assignee
Institute of Electronics of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Electronics of CAS filed Critical Institute of Electronics of CAS
Priority to CNB031079008A priority Critical patent/CN100440117C/en
Publication of CN1534544A publication Critical patent/CN1534544A/en
Application granted granted Critical
Publication of CN100440117C publication Critical patent/CN100440117C/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Abstract

The present invention relates to a man-machine interaction technique of large screen display, particularly to a man-machine interaction technique of a large screen non contact type control mode, which has the main technique that a laser pen is used for controlling the operation of a screen and achieving the non contact type control mode; a red laser pen with wireless mouse left and right keys is used for controlling the left and the right keys and the cursor position of the operation of a large screen, a cursor of a system can track a laser point at any time, and all the functions of the ordinary mouse are completed; the information of the laser point is obtained by a camera with a red filter, signal-to-noise ratio is increased, and a tracking process with high precision is achieved; the data of the camera is received by a video capture card and is processed in real time, the position of the laser point is obtained, and therefore, the position of the cursor of the system is set; the interpolation of the laser point is carried out by a forward prediction interpolation algorithm, and the cursor can be smoothly moved.

Description

The contactless control mode of giant-screen
Technical field
The present invention relates to the human-computer interaction technique field of large screen display, relate generally to the straight technology of giant-screen being carried out contactless control mode that connects of red laser of utilizing band wireless mouse right and left key.
Background technology
Arrival along with the information age, the interaction technique of people and information is just being played the part of more and more important role, wherein the large screen projection display system is exactly one of current the most modern video signal instrument, because intuitive, the super large characteristics of itself, be widely applied to every field, especially military combat commander, virtual reality etc.
Large screen display system mainly comprises display control program, projector system, screen wall system.
Display control program is the control center of whole large screen projection display system, and displaying contents and display mode on the Projection Display wall are directly controlled in its response people's various control behaviors.Display control program generally is made up of computing machine, multiple interactive device (as keyboard, mouse, laser pen etc.), multi-channel video display device (during multi-screen splicing).Wherein the multi-screen splicing display control program mainly is divided into two classes: a class is directly the picture on several the computer monitors to be received to amplify on the giant-screen to show; Another kind of is to be presented on the whole giant-screen by the various application windows of multi-channel video display device with same computing machine.Preceding a kind of quantity of information that do not increase just allows everybody can see the information that shows on several the computing machines simultaneously; The back is a kind of then to be that the large screen display wall is shielded as virtual logical, can open any a plurality of application window on screen wall, makes full use of the display area of screen wall, and a large amount of information is provided simultaneously.
When realizing large screen display system, human-computer interaction technology is absolutely necessary, and this directly influences efficient, the difficulty of operation and the applicability of intuitive, different occasions that display system is used.Because the intrinsic super large characteristics of giant-screen itself, make the red laser pen become the first-selection interactive device commonly used of large screen display system, laser pen is used for the red point of direct directive screen generation high brightness, thereby plays indicative function.
There have been some giant-screen contact control modes to obtain application at present.[The Writing ' the s on the Wall:Large of Mark Ashdown and Peter Robinson, Remotely ControlledDisplays] introduced the positional information that detects input pen with the magnetic detection head of two intersections, when pen magnetic transmitter in the pen by on screen the time is just launched electromagnetic wave one time, the detecting head that intersects receives electromagnetic wave respectively and passes to computing machine at once, the reception program of running background calculates the current location that both receive definite pen of electromagnetic mistiming in real time on the computing machine, is converted into screen coordinate again the current location of system's cursor to pen is set.[the Multi-UserLaser-Based Interaction on Large Tiled Displays] of James Davis and Xing Chen also mentions with two intersection detecting heads and obtains the input point position.3M company has made integrated demonstration conference system, has realized functions such as interactive electronic white board, Web conference, can directly write on screen, edition function.But contact control mode itself has just limited its range of application and screen size.At first, pen must contact screen, this all is unallowed for most of screen, screen can be hindered by stroke and leave a trace, if the polylith Mosaic screen can influence the track of pen in splicing place, splicing place simultaneously can be subjected to badly damaged, and it needs special wear-resisting material to be additional to screen surface with the protection screen; Secondly, a range of movement is directly received the restriction that staff can reach scope, and screen size just can not be stretched out one's hand scope above common people like this.So the contact control mode generally is applied to the screen interactive display system of the common size of monolithic screen.
Summary of the invention
The object of the invention is to provide the human-computer interaction technology of the contactless control mode of a kind of giant-screen, this technology make the man-machine interaction of large screen display system become quite simple effectively, thereby simultaneously again the simplified system equipment needed thereby reduce the cost of display system.Use the large screen display system of this technology to possess very strong operation simplification, equipment interoperability, system's adjustability.Only need a hand-held small and exquisite red laser pen, control cursor arbitrarily and be with the LR-button operation with laser spots, realization is to all mouse action functions of giant-screen, use very intuitive and convenient, laser points to where cursor is arrived there at once, cooperates LR-button to get final product the Any Application of operating system again.Maximum characteristics are exactly no longer to rely on the physics roll mouse to control cursor position, but depend merely on the red laser assigned address, so just adapt to different far and near distances, contact or contactless any operation.
For achieving the above object, technical solution of the present invention provides the contactless control mode of a kind of giant-screen, form by laser pen, giant-screen, projector, camera and high-performance computer, its techniqueflow is: adopt band wireless mouse right and left key red laser pen emission red laser to be directed on the giant-screen, form high azarin point on screen; The a plurality of cameras that are positioned at screen optical filter are redly behind gathered the image/video on the corresponding screen area respectively in real time; Be installed in multi-channel video capture card in the high-performance computer and import image/video respectively from each road camera; High-performance computer is handled the every road of capture card image/video in real time, accurate red some position of fast detecting, and be converted into corresponding computing machine and show and picture element position on the output video again the computer system cursor is set to this position, realize that system's mouse moves; The wireless push of laser pen is realized mouse button functions with the receiver that wireless mode connects on the computing machine in addition; Computing machine outputs on many back projecting projectors by the bull video card again, projects the unified logic giant-screen of reformulation on the screen by projector.
The contactless control mode of described giant-screen, its laser pen can point at a distance and not be subjected to staff can reach scope to limit, can not influence the participant yet and watch screen, and can on rear projection screen, form high azarin point, make rear-mounted camera can record down the red locus of points in real time.
The contactless control mode of described giant-screen, it has adopted the camera of a plurality of rear-mounted optical filters redly to come to be responsible for respectively than small screen area, the pointing accuracy and the sensitivity of laser pen have effectively been guaranteed, laser pen not only can point to and can also carry out the operation of mouse right and left key simultaneously, possesses complete mouse function.
The present invention does not just need pen to contact with screen and realizes operating function owing to adopted laser pen, makes position and size that giant-screen is placed no longer be subjected to user's hand can reach the restriction of scope, and only needing that red laser is pointed to giant-screen can operate.Make giant-screen do not rubbed or scratch simultaneously, improved screen serviceable life and display effect.Certainly hold laser pen and contact too with screen and can operate, as long as red laser is mapped on the giant-screen.
Description of drawings
Fig. 1 is the contactless control mode system construction drawing of giant-screen of the present invention.
Embodiment
Technical scheme system construction drawing of the present invention mainly comprises as shown in Figure 1: laser pen 1, giant-screen 2, projector 3, camera 4 and high-performance computer 5.
Wherein, bull video frequency collection card 51, multichannel display card 52 and network interface card 53 are arranged in the high-performance computer 5; Computing machine 5 also can be operated by Wireless Keyboard 6.
Giant-screen 2 links to each other with high-performance computer 5; Multichannel display card 52 in projector 3 and the high-performance computer 5 links to each other, and camera 4 and high-performance computer 5 interior bull video frequency collection cards 51 link to each other; Network interface card 53 in the computing machine 5 connects high-speed local area network 7.
Red filter plate (not illustrating among the figure) is housed before camera 4 camera lenses.
Laser pen 1 can be launched red laser to giant-screen 2, produces the red luminous point of high brightness.
The contactless control mode of giant-screen of the present invention system also needs the support of software program, and comprise two important procedure: 1, laser spots detects background program in real time; 2, laser spots drafting symbol background program.
The working condition of the contactless control mode of giant-screen of the present invention is as follows:
(a) laser pen 1 and giant-screen 2.Giant-screen 2 can be any display screen, also can be that monolithic screen or polylith screen splicing form, it only influences the display effect of projection, does not influence red laser control accuracy of the present invention and sensitivity, so can choose at random pre-projecting type or rear projection screen.The red laser pen 1 of band wireless mouse right and left key function is realized mouse function, utilizes the left and right press key radio communication function between it and the computing machine to realize the left and right press key operation, utilizes red laser shine the red point of generation on giant-screen 2 and changes system's cursor position.Feature of the present invention is controlled cursor with laser spots exactly, and finishes all mouse control functions with left and right sides wireless push, and more simple, intuitive is convenient to allow giant-screen 2 operations become, and realizes giant-screen 2 contactless controls.Owing to realized the jumbotron demonstration, the mobile of conventional mouse becomes difficult unusually, at first the user need see current cursor position clearly, only found current cursor position to follow current relative direction ability rolling mouse according to the position of wanting to move on to then, next position that wants to move on to is far with current cursor position distance, conventional mouse needs mobile range very big, and operation has also just become trouble.The advantage that laser pen 1 replaces conventional mouse just is this.Right and left key on the laser pen 1 is finished the right and left key function that realizes conventional mouse simultaneously, thereby has increased substantially intuitive, the simplification used.
(b) multichannel display card 52 and projector 3.If the monolithic display screen has not just needed multichannel display card 52, as long as single projector 3 is received in the output of the video card of common computer, general resolution can arrive 1024 * 768.The words of bigger display area if desired, just need the polylith screen splicing to form, so just need the demonstration of computing machine 5 to be processed into multi-channel video signal and pass to a plurality of projectors 3 by multichannel display card 52, each projector's 3 projection goes out partial content reformulate the entire display screen curtain on giant-screen 2 then, for example 1 * 3 arrangement screen and projector 3 just can obtain 3072 * 768,2 * 3 arrangements and just can obtain 3072 * 1536.
(c) camera 4 of filter plate and bull video frequency collection card 51 redly.In order to obtain the positional information of laser spots in real time, directly over each projector 3, be close to and place a camera 4, obtain the interior laser spots information of corresponding screen scope in real time with camera 4.Because giant-screen 2 systems may be applied in the multiple different complex environment, noise just may flood laser spots information and can't obtain correct laser spots position like this, so the present invention on each camera 4 is pre-installed red filter plate, so just the signal to noise ratio (S/N ratio) of laser spots can be brought up to about 10DB from 2DB, approximately improved more than 5 times.Video frequency collection card 51 is contained on the computing machine 5, receives the image that each camera 4 obtains in real time, and deposits buffer zone so that computing machine 5 reads processing in real time.Here camera 4 can be in giant-screen 2 fronts also can be in giant-screen 2 back, if prepositionly certainly just might be blocked and can't obtain laser spots, so preferably rearmounted by people or thing.Adopt rearmounted projector 3 and camera 4, the shortcoming of thoroughly having avoided screen content in general giant-screen 2 systems to be blocked by health or object, the user can walk about arbitrarily before giant-screen 2, makes the user operate like this and more is not subjected to environmental restraint.Also can add that 1394 capture cards realize better acquisition function with digital camera head 4.
(d) high-performance computer 5.It is the processing maincenter of total system, and major function has: handle the realtime graphic acquisition laser spots position that collects and put cursor again to laser spots; The left and right press key of response wireless mouse; Drive multichannel display card 52 outputting video signals; Operational system and each application program; Laser spots among operation the present invention detects background program and laser spots drafting symbol background program in real time.
The present invention also comprises two critical software programs:
(1) laser spots detects background program in real time.Every two field picture that this program is finished every road capture card calculates, therefrom detect the picture element that surpasses certain thresholding with laser spots sample matches degree, calibrate display screen and camera again and obtain the position scaled ratio of image, thereby the picture element coordinate transform is become system's displaing coordinate initialization system cursor position.Here Edge extraction and image matching algorithm have been adopted.Program also provides automatically accurately calibration control interface.Adopt the forward prediction interpolation algorithm that laser spots is carried out interpolation simultaneously, realize smooth moving cursor.
(2) laser spots drafting symbol background program.Under suitable certain situation, the user need directly draw some symbols or write some things with laser spots on giant-screen, and this just needs laser spots drafting symbol background program to respond these operations.After back has obtained the laser spots position, write down each positional information and just form curve, but the video that obtains owing to camera generally all had only about 30 frame/seconds, so the curve shape that can the broken line type curve obviously occur even can not obtain wanting when the laser spots fast moving.For sort this problem out the present invention has adopted the forward prediction interpolation algorithm, promptly on basis, more preceding some position, carry out interpolation and obtain better curve, again this curve is predicted that extension calculates next point and detects some somes positions may arriving before, pointwise is predicted forward successively, finally obtains curve more true to nature.Simultaneously if the piece image that on giant-screen, shows, program fictionalizes all symbols that come the recording laser point-rendering with its empty graph layer of a size so, and drafting, selection, delete function can be arranged, and these symbols are followed the picture roll convergent-divergent with display image coordinate correspondence simultaneously.
Embodiment
The present invention has obtained specific implementation.Contrast accompanying drawing 1, the equipment of selection such as following table:
Title Concrete equipment Major parameter
Screen The back projection type LCD screen 60 inches/1024*768,3 * 2,3072*1536,6
Laser pen Sieve skill-blue utmost point flies ermine Red laser, right and left key, bluetooth/10 meter, USB interface receiver
Wireless Keyboard Benq-no shadow hand Wireless Keyboard, the USB interface receiver
The bull display card Matrox-Parhelia 512 3 outputs, DirectX9, OpenGL, 2
Projector SONY 1078*768,6
The multi pass acquisition card Beautiful as picture-PV988 4 the road gather simultaneously, PAL/NTSC, 2
Camera The simulation camera The PAL/NTSC standard, 384*288,25FPS, 6
Red filter plate Red filter plate Common grade
Server High-performance computer 2.4G, 1G DDR, 120G hard disk
Network interface card TP-Link The 100M/10M self-adaptation
The leading indicator that obtains on above hardware foundation is as follows:
● the coincidence error of cursor and laser spots: every screen edge area coincidence error is less than 5 pixels (display resolution), and error is less than 4 pixels in 2/3 screen area of center, and center 1/3 screen area is interior less than 2 pixels;
● the relative laser spots of cursor postpones:<40ms visually postpones not obvious;
● the real-time verification and measurement ratio of laser spots:>90%, real-time is better;
● real-time trace routine of laser spots and the direct plotter program of laser pen normally move CPU usage: 1~4% (P42.4G);
● laser pen operating distance: in 10 meters; 15 meters (receiver adds the USB extended line);
● screen splicing error:<1 pixel.

Claims (3)

1, the contactless control mode of a kind of giant-screen is characterized in that, techniqueflow is: adopt band wireless mouse right and left key red laser pen emission red laser to be directed on the giant-screen, form high azarin point on screen; The a plurality of cameras that are positioned at screen optical filter are redly behind gathered the image/video on the corresponding screen area respectively in real time; Be installed in multi-channel video capture card in the high-performance computer and import image/video respectively from each road camera; High-performance computer is handled the every road of capture card image/video in real time, detects red some position, and is converted into the picture element position on the corresponding computing machine demonstration output video, again the computer system cursor is set to this position, realizes that system's mouse moves; The wireless push of laser pen is realized mouse button functions with the receiver that wireless mode connects on the computing machine in addition; Computing machine outputs on many back projecting projectors by the bull video card again, projects the unified logic giant-screen of reformulation on the screen by projector.
2, the contactless control mode of giant-screen as claimed in claim 1, it is characterized in that, laser pen can point at a distance and not be subjected to staff can reach scope to limit, can not influence the participant yet and watch screen, and can on rear projection screen, form high azarin point, make rear-mounted camera can record down the red locus of points in real time.
3, the contactless control mode of giant-screen as claimed in claim 1, it is characterized in that, adopted the camera of a plurality of rear-mounted optical filters redly to come to be responsible for respectively than small screen area, the pointing accuracy and the sensitivity of laser pen have effectively been guaranteed, laser pen not only can point to and can also carry out the operation of mouse right and left key simultaneously, possesses complete mouse function.
CNB031079008A 2003-04-01 2003-04-01 Large screen non contact type control mode Expired - Fee Related CN100440117C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNB031079008A CN100440117C (en) 2003-04-01 2003-04-01 Large screen non contact type control mode

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNB031079008A CN100440117C (en) 2003-04-01 2003-04-01 Large screen non contact type control mode

Publications (2)

Publication Number Publication Date
CN1534544A CN1534544A (en) 2004-10-06
CN100440117C true CN100440117C (en) 2008-12-03

Family

ID=34283051

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB031079008A Expired - Fee Related CN100440117C (en) 2003-04-01 2003-04-01 Large screen non contact type control mode

Country Status (1)

Country Link
CN (1) CN100440117C (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102707817A (en) * 2012-04-24 2012-10-03 广州市唯昕电子科技有限公司 Laser writing system

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1299192C (en) * 2005-04-08 2007-02-07 清华大学 Remote-wireless controller for computer
CN101807111A (en) * 2009-02-13 2010-08-18 夏普株式会社 Information apparatus, control method and system thereof
CN101520326B (en) * 2009-03-24 2012-04-25 湖北恒颖超科技有限公司 Non-contact type positioning device for lighting body capable of utilizing different color sequences
CN101551664B (en) * 2009-05-04 2010-12-29 大道计算机技术(上海)有限公司 Big screen commanding and dispatching system and realizing method thereof
CN101833385B (en) * 2009-08-21 2011-11-02 深圳先进技术研究院 Remote control interactive pen and receiver thereof
CN102262733B (en) * 2010-05-28 2014-03-19 株式会社理光 Laser point detection method and apparatus thereof
CN102024446B (en) * 2010-11-23 2013-01-09 广东威创视讯科技股份有限公司 Large-screen spliced wall display device and method
CN102073395B (en) * 2011-02-25 2012-08-29 上海交通大学 Wireless laser pen interaction system based on field programmable gate array (FPGA)
CN102760011B (en) * 2011-04-28 2016-06-29 东友科技股份有限公司 The interactive approach of interactive projection system and optical projection system
CN102253737B (en) * 2011-05-31 2013-07-17 王石 Realizing method of screen-vision mouse system
CN103092432B (en) * 2011-11-08 2016-08-03 深圳市中科睿成智能科技有限公司 The trigger control method of man-machine interactive operation instruction and system and laser beam emitting device
CN102778980B (en) * 2012-07-05 2015-07-08 中国电子科技集团公司第二十八研究所 Fusion and interaction system for extra-large-breadth display contact
CN102799375B (en) * 2012-07-05 2015-08-19 中国电子科技集团公司第二十八研究所 A kind of extra-large-breadth display contact merges the image processing method of interactive system
CN103472825A (en) * 2013-08-23 2013-12-25 黄涌 Camera-based bright spot tracking system and method
CN103760998A (en) * 2014-02-19 2014-04-30 赵威 Method for achieving laser pen ultra-bright cursor identification with CCD sensor over exposure
CN104346126B (en) * 2014-10-31 2017-04-05 京东方科技集团股份有限公司 Method, method for displaying image and system that image is shown in real time with mobile spot
CN106445195B (en) 2015-08-11 2019-05-24 华为技术有限公司 The method for detecting position, apparatus and system of laser point in screen
CN106445090B (en) * 2015-08-12 2021-02-23 中兴通讯股份有限公司 Method and device for controlling cursor and input equipment
CN105183190B (en) * 2015-10-25 2018-03-20 刘书智 A kind of laser pen with mouse function
CN107967078B (en) * 2016-10-20 2021-03-12 南京仁光电子科技有限公司 Calibration method for touch points of spliced screen
CN109144375B (en) * 2018-10-09 2022-08-19 中天智领(北京)科技有限公司 Screen control method and device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5835078A (en) * 1993-12-28 1998-11-10 Hitachi, Ltd. Information presentation apparatus and information display apparatus
CN1281209A (en) * 1999-07-20 2001-01-24 浙江大学 'Photoelectric virtual blackboard' system based on large-screen display
KR20020037740A (en) * 2002-05-07 2002-05-22 권대웅 Laser Signal Control For Digital Presentation Device Laser Receiver

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5835078A (en) * 1993-12-28 1998-11-10 Hitachi, Ltd. Information presentation apparatus and information display apparatus
CN1281209A (en) * 1999-07-20 2001-01-24 浙江大学 'Photoelectric virtual blackboard' system based on large-screen display
KR20020037740A (en) * 2002-05-07 2002-05-22 권대웅 Laser Signal Control For Digital Presentation Device Laser Receiver

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102707817A (en) * 2012-04-24 2012-10-03 广州市唯昕电子科技有限公司 Laser writing system
CN102707817B (en) * 2012-04-24 2015-09-30 广州市唯昕电子科技有限公司 Laser inscription system

Also Published As

Publication number Publication date
CN1534544A (en) 2004-10-06

Similar Documents

Publication Publication Date Title
CN100440117C (en) Large screen non contact type control mode
US6501461B2 (en) Retrofittable apparatus for converting a substantially planar surface into an electronic data capture device
US7486281B2 (en) Coordinate input apparatus, control method thereof, and program
US5926168A (en) Remote pointers for interactive televisions
CN101238428B (en) Free-space pointing and handwriting
CN101315586B (en) Electronic pen for interactive electronic white board and interaction control method thereof
KR101247991B1 (en) Camera gestures for user interface control
US20060012567A1 (en) Minature optical mouse and stylus
EP0686935A1 (en) Pointing interface
US20070200791A1 (en) Locating display items on multiple display devices
US20090309841A1 (en) Eraser for use with optical interactive surface
CN106055177B (en) A kind of scan method and device of infrared touch panel
CN101714044B (en) Camera positioning-based touch screen system
CN201562264U (en) Self-adaptive intelligent interactive projector
JP4455185B2 (en) Presentation system, control method therefor, program, and storage medium
CN103294284B (en) A kind of electronic equipment and a kind of writing device
KR101057620B1 (en) Multi touch input processing device
CN101008877A (en) Image device with optical recognition screen and control method therefor
CN110851025B (en) Frameless touch interactive system and method based on radar induction
JPH0326855B2 (en)
CN208240003U (en) A kind of orientable spherical displaying device
CN101751161A (en) Optical pen, optical pen position decision method and interactive display system
AU718394C (en) Retrofittable apparatus for converting a substantially planar surface into an electronic data capture device
CN201266355Y (en) Positioning multi-optical-spot system for rectangle plane
CN114690977A (en) Elastic wave-based interaction evoking method and device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20081203

Termination date: 20150401

EXPY Termination of patent right or utility model