CN102799344A - Virtual touch screen system and method - Google Patents
Virtual touch screen system and method Download PDFInfo
- Publication number
- CN102799344A CN102799344A CN2011101400794A CN201110140079A CN102799344A CN 102799344 A CN102799344 A CN 102799344A CN 2011101400794 A CN2011101400794 A CN 2011101400794A CN 201110140079 A CN201110140079 A CN 201110140079A CN 102799344 A CN102799344 A CN 102799344A
- Authority
- CN
- China
- Prior art keywords
- patch
- touch operation
- unicom
- operation zone
- component
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention provides a virtual touch screen method used for a touch screen system and a virtual touch screen system using the virtual touch screen method. The virtual touch screen method comprises the following steps: initially obtaining depth information of an environment containing a touch operation area, establishing an initial depth map based on the initially obtained depth information, and determining the position of the touch operation area based on the initial depth map; continuously obtaining images of the environment of the determined touch operation area; detecting a candidate plaque of at least an object in a preset distance in front of the touch operation area from each frame of the obtained images; and classifying each plaque to a corresponding point sequence according to a relationship of a centroid point of the plaque obtained from front and back two adjacent frames of images in time and space.
Description
Technical field
The present invention relates to a kind of field of human-computer interaction, especially the design digital image processing field.Particularly, the present invention relates to a kind of virtual touch method of touchscreen system and virtual touch screen system that adopts this method of being used for.
Background technology
Touchscreen technology is used for portable equipment (for example smart phone) and the PC (for example Desktop PC) as HMI equipment now more and more widely.Through touch screen, the user can operate this equipment more comfortable and easily and bring favorable experience.Although touchscreen technology is extremely successful in handheld device, but still there are some problems and chance for the touch screen of large-sized monitor
Belong to Canesta; Which key assignments the Inc title selects be designated as the method for current key assignments for the U.S. Pat 7151530B2 of " System and Method for Determining an Input Selected By a User through a Virtual Interface (confirming the system and method for user-selected input through virtual interface) " has proposed a kind of being used at one group of key assignments, therefore be able to provide with virtual interface in the object that intersects of zone.This virtual interface can be implemented in the single key assignments of selection in the key assignments group and use depth transducer to confirm that position, this depth transducer can confirm the degree of depth of the position relevant with the position of depth transducer.In addition, the style characteristic of the placement property of object or object one of at least can be able to confirm.Positional information can be similar to the degree of depth of object with respect to position transducer or other RPs.When the existing of the pixel denoted object of sufficient amount in the pel array of camera, just think to detect this object.Confirm to compare with the shape of the object of the surface crosswise of virtual input region and with multiple known shape (for example finger or stylus).
Belong to Canesta equally; The Inc title comprises two optical system OS1 and OS2 for the U.S. Pat 6710770B2 of " Quasi-Three-Dimensional Method And Apparatus To Detect And Localize Interaction Of User-Object And Virtual Transfer Device mutual standard three method for position and the equipment of consumer positioning-object and virtual conversion equipment (be used to detect and) " has disclosed a kind of virtual bench input or transmission information of adopting to the system of auxiliary equipment.In light structure embodiment, OS1 is on virtual bench and be parallel to the luminous energy on this virtual bench emission fan beam plane ().When user object penetrated interested beam plane, OS2 registered this incident.Triangulation method can be located virtual contact, and with the consumer premise information transmission to subsystem.In non-structure active light structure, OS1 is preferably a kind of digital camera, and its visual field has defined interested plane, and this plane is illuminated by an active luminous energy source.
Belong to Touchtable, the title of Inc has disclosed a kind of by the interactive display that detects user's attitude of the being discerned control on the display surface in touch for the U.S. Pat 7728821B2 of " Touch Detecting Interactive Display (touch and detect interactive display) ".Image is projected on the horizontal projection surface from the projector that is positioned on the projection surface.Use is arranged in one group of infrared transmitter and the position that receiver detects user's contact projection surface of the periphery of projection surface.For each contact position, computer software application is stored the history of contact position information, and confirms the speed of each contact position according to this position history.Based on this position history and velocity information, the identification attitude.The attitude of being discerned is associated with display command, and correspondingly, these display commands are performed the image that is shown to upgrade.Therefore, this United States Patent (USP) make the user can through and image between direct physical control display alternately.
The title that belongs to Microsoft is that the U.S. Pat 7599561B2 of " Compact Interactive Tabletop with Projection-Vision (the small-sized mutual desktop with projection visual field) " has disclosed a kind of system and method that projects to any surface of being convenient to any image (static or mobile) based on vision.The interaction surface system of a kind of orthogonal projection computer based in vision especially is provided, and it uses a kind of novel commercial obtainable shadow casting technique to obtain the factor (factor) of a kind of small-sized independence (self-contained) form.The configuration of this system solved great majority in based on the table top system of vision main installation, calibration and the portable problem of paying close attention to.Wherein, the image of input by binaryzation producing shadow image, and the shadow shapes in this image of subsequent analysis with the original of confirming this shade be " touchs " to surperficial go up or " hovering " from the teeth outwards just.
It seems that from top these prior aries of mentioning most of large scale touch screen all are based on magnetic board (such as electronic whiteboard), IR border (such as interactive large-sized monitor) etc.For the current technical scheme of large scale touch screen, still had a lot of problems at that time, for example: in general, the equipment of these types is big and heavy owing to its volume that its hardware causes usually, therefore is difficult to carry, and does not have portability.And the device screen size of these types receives restriction and the fixed size of hardware and can not freely regulate according to the needs of environment, but also needs a kind of special time writer or a kind of IR pen to operate.
For some virtual whiteboard projector, the user must control the on/off switch of laser pen, and this is very loaded down with trivial details, therefore has the unmanageable problem of laser pen.In addition, in this virtual whiteboard projector,, just be difficult to accurately laser point navigated to next position, therefore have the difficult problem in laser point location in case laser pen is closed.Adopt finger mouse to substitute laser pen in some virtual whiteboard projector, still, adopted the virtual whiteboard projector of finger mouse can not detect touch beginning (touch on) or touch end (touch up).
Summary of the invention
For these problems of the prior art of being mentioned above solving, the present invention proposes and a kind ofly be used for the virtual touch method of touchscreen system and make virtual touch screen system in this way.
Particularly; This virtual touch method that is used for touchscreen system comprises: the initial depth information that obtains to comprise the regional environment of a touch operation; Depth information based on said initial acquisition is created ID figure, and confirms the position in said touch operation zone based on said ID figure; Obtain continuously the image of the environment in said definite touch operation zone; From the every two field picture that is obtained, detect the candidate's patch be positioned at least one object of preset distance before the said touch operation zone; The relation of centroid point on time and space according to the patch that obtains in adjacent two two field pictures in front and back is included into corresponding point sequence with each patch.The step of the position in wherein said definite said touch operation zone comprises: the UNICOM's component among detection and the said ID figure of mark; Whether UNICOM's component of mark comprises two intersection of diagonal of said ID figure to confirm to detect institute also; Detect and UNICOM's component of mark comprises under the situation of two intersection of diagonal of said ID figure, the diagonal line that calculates said ID figure detects the also intersection point of UNICOM's component of mark with institute; And connect the intersection point that is calculated successively, and institute's connection acquisition convex polygon is confirmed as said touch operation zone.
According to the virtual touch method that is used for touchscreen system of the present invention, the described virtual touch method that is used for touchscreen system, wherein, said touch operation zone is overlapping with the view field of said touchscreen system.
According to the virtual touch method that is used for touchscreen system of the present invention, said touch operation zone is not overlapping with the view field of said touchscreen system.
According to the virtual touch method that is used for touchscreen system of the present invention, the step of the UNICOM's component among the said ID figure of wherein said detection and mark comprises: calculate each its area of UNICOM's component; And whether the area of confirming to be calculated greater than a predetermined area threshold, and abandons the connected component of its area less than said predetermined area threshold value.
According to the virtual touch method that is used for touchscreen system of the present invention, 1/4th of the area that said predetermined area threshold value is said ID figure.
According to the virtual touch method that is used for touchscreen system of the present invention; The wherein said intersection point that is calculated that connects successively; And with connect and obtain convex polygon and confirm as the step in said touch operation zone and comprise: confirm whether connect the formed shape of being calculated of intersection point successively is convex polygon; If not convex polygon, then abandon formed UNICOM component.
According to the virtual touch method that is used for touchscreen system of the present invention, the said relation of centroid point on time and space according to the patch that obtains in adjacent two two field pictures in front and back comprises the step that each patch is included into corresponding point sequence: receive the centroid point that from a frame new images, obtains a plurality of new patches; Retrieve a plurality of existing point sequences, to each existing point sequence, the searching distance should have the centroid point that point sequence closes on, and the pairing new patch of nearest centroid point is had the nearest new patch of sequence of points as being somebody's turn to do; And the new patch nearest apart from this existing point sequence be included into this existing point sequence.
According to the virtual touch method that is used for touchscreen system of the present invention; The said relation of centroid point on time and space according to the patch that obtains in adjacent two two field pictures in front and back also comprises the step that each patch is included into corresponding point sequence: to any one existing point sequence; If there is not any new patch that closes on it, then provide notice and this existing point sequence of deletion that this existing point sequence has finished.
According to the virtual touch method that is used for touchscreen system of the present invention; The said relation of centroid point on time and space according to the patch that obtains in adjacent two two field pictures in front and back also comprises the step that each patch is included into corresponding point sequence: be directed against any new patch in a plurality of new patch in the two field picture of being imported; If there is not any existing point sequence that closes on it; Then providing this new patch is the notice of the starting point of a new point sequence, and creates a new point sequence.
According to the virtual touch method that is used for touchscreen system of the present invention; Said step of being somebody's turn to do the nearest centroid point of existing point sequence to each existing point sequence searching distance comprises: import an existing point sequence, and seek in a plurality of new patch from a two field picture of being imported apart from having the new patch that point sequence closes on; Do not find under the situation of the new patch that the existing point sequence imported of distance closes at a plurality of new patch from a two field picture of being imported, then provide the notice that the existing point sequence imported will be deleted; The new patch that the existing point sequence that has found distance to be imported at a plurality of new patch from a two field picture of being imported closes on; And the new patch that closes on that is found only is the new patch that closes on of the existing point sequence imported, confirms that then this new patch belongs to the existing point sequence of being imported; And the new patch that closes on of the existing point sequence that has found distance to be imported at a plurality of new patch from a two field picture of being imported; And the new patch that closes on that is found still is the new patch that closes on of other existing point sequences; If the distance of the new patch that closes on and the existing point sequence of being imported that are found is than little with the distance of other existing point sequences; Confirm that then this new patch belongs to and the nearest new patch of being imported of existing point sequence; Otherwise, provide the notice that the existing point sequence imported will be deleted; Examine and whether all existing sequence of points have been carried out above-mentioned steps.
According to the virtual touch method that is used for touchscreen system of the present invention; The step of seeking the new patch that distance should existing point sequence closes in the said a plurality of new patch from a two field picture of being imported comprises: import a new patch, calculate new patch of being imported and the distance of being imported between the point sequence that has; Calculated import new patch and the existing point sequence imported between the situation of distance less than a predetermined distance threshold Td under; The new patch of import is inserted into the candidate who belongs to the existing track of being imported to be closed in the new patch tabulation; Otherwise, confirm whether all new patches have been carried out above-mentioned steps; Confirming that the candidate who belongs to the existing point sequence of being imported closes under the situation of size less than a predetermined size threshold value Tsize of new patch tabulation; Confirm whether all new patches have been carried out above-mentioned steps; Otherwise; The candidate that deletion belongs to the existing point sequence of being imported close in the new patch tabulation with the existing point sequence of being imported new patch farthest after, confirm whether all new patches have been carried out above-mentioned steps; Confirming that new patch has been carried out under the situation of above-mentioned steps to all; Close on the tabulation of new patch for empty if belong to the candidate of the existing point sequence of being imported, then the candidate from said tabulation closes on the recently newly patch of the nearest new patch of the existing point sequence finding out in the new patch and imported as the existing point sequence of being imported.
According to the virtual touch method that is used for touchscreen system of the present invention, also comprise: adopt following formula to carry out coordinate optimization carrying out smoothing processing to the point sequence of final acquisition,
Wherein
is the point in the point sequence; K is the iteration sign; N is the point sequence sign, and m is the radix of iteration point.
According to another aspect of the present invention, a kind of virtual touch screen system is provided, has comprised: projector, with image projection to in the projection surface; Depth cameras, acquisition comprise the depth information of the environment in touch operation zone; The depth map processing unit is created ID figure based on the depth information that depth cameras obtains under initial conditions, and confirms the position in said touch operation zone based on said ID figure; Subject detecting unit after initial conditions the continuous every two field picture that obtains, detects the candidate's patch that is positioned at least one object of preset distance before the determined touch operation zone from depth cameras; Tracking cell is included into corresponding point sequence according to the relation of centroid point on time and space of the patch that obtains in adjacent two two field pictures in front and back with each patch.Wherein, said depth map processing unit is confirmed the position in said touch operation zone through following process: the UNICOM's component among detection and the said ID figure of mark; Whether UNICOM's component of mark comprises two intersection of diagonal of said ID figure to confirm to detect institute also; Detect and UNICOM's component of mark comprises under the situation of two intersection of diagonal of said ID figure, the diagonal line that calculates said ID figure detects the also intersection point of UNICOM's component of mark with institute; And connect the intersection point that is calculated successively, and institute's connection acquisition convex polygon is confirmed as said touch operation zone.
Description of drawings
Shown in Figure 1 is according to the synoptic diagram of the framework of virtual touch screen of the present invention system.
Shown in Figure 2 is the overview flow chart of performed object detection of control module according to the present invention and object tracking processing.
Shown in Fig. 3 (a)-(b) is according to the synoptic diagram of confirming the touch operation zone in the regional virtual touch screen system of hand touch operation that comprises of the present invention.
Shown in Figure 4 is according to the process flow diagram of confirming the touch operation zone in the virtual touch screen of the present invention system.
Shown in Figure 5 is according to the process flow diagram of confirming summit, touch operation zone in the virtual touch screen of the present invention system.
Shown in Figure 6 is according to the synoptic diagram of confirming summit, touch operation zone in the virtual touch screen of the present invention system.
Shown in Fig. 7 (a)-(c) is the synoptic diagram of from current depth map, disposing background depth map
Shown in Figure 8 is carries out the synoptic diagram of binary conversion treatment with the patch that obtains candidate target to the depth map of the current scene imported.
Shown in Fig. 9 (a) is the synoptic diagram that is used for patch is added the connected domain of numbering
Shown in Fig. 9 (b) is the synoptic diagram of the bianry image of the patch that has the connected domain numbering that generates according to depth map.
Shown in Figure 10 (a)-(d) is the synoptic diagram of enhancing process of the bianry image of patch.
Shown in Figure 11 is detects the synoptic diagram of the process of the coordinate of the centroid point of patch in the binary image of the patch shown in Figure 10 (d).
The synoptic diagram of the track that shown in Figure 12 is user's finger or stylus moves on the screen of virtual touch screen.
The process flow diagram that is used to follow the tracks of the object that is detected shown in Figure 13.
Shown in Figure 14 is to seek the process flow diagram of the recently new spot of every existing track according to the present invention is directed to all existing tracks.
Shown in Figure 15 is to be directed against the existing track of being imported to seek the process flow diagram apart from its nearest new patch.
The point sequence of shown in Figure 16 the is a kind of motion track of detected object on virtual touch screen that a strip adoption the present invention is obtained carries out the method for smoothing processing.
Shown in Figure 17 (a) is the synoptic diagram of the motion track of detected object on virtual touch screen that obtain of a kind of the present invention of employing.
Shown in Figure 17 (b) is the synoptic diagram through the object motion track after the smoothing processing.
Shown in Figure 180 is the synoptic diagram of the concrete configuration of control module.
Embodiment
Below, will specify specific embodiment of the present invention with reference to accompanying drawing.
Shown in Figure 1 is according to the synoptic diagram of the framework of virtual touch screen of the present invention system.As shown in Figure 1; Comprise projector equipment 1, optical device 2, control module 3 and pseudo operation zone (when this zone and projection screen are overlapping, also can be referred to as also can be referred to as projection screen or virtual screen below the projection surface) 4 according to virtual touch screen of the present invention system.In embodiment of the present invention, projector equipment can be a projector, and it is used for the needs images displayed is projected in the projection surface 4 as a kind of virtual screen, so that the user is at the enterprising line operate of this virtual screen.Optical device 2 for example can be any equipment that can obtain image, depth cameras for example, and it obtains the depth information of the environment in pseudo operation zone 4, and generates depth map based on this depth information.Control module 3 detects at least one object in the said surperficial preset distance of distance and follows the tracks of detected object along the direction away from said surface, to generate level and smooth point sequence.Interactive command or the like is for example painted, made up to said point sequence to be used for further interactive task on virtual screen.
Projector equipment 1 with image projection to the projection surface 4 as a kind of virtual screen, so that the user at the enterprising line operate of this virtual screen, for example draws or the combination interactive command.Optical device 2 is caught the environment that includes projection virtual screen and the object that is positioned at the place ahead of projection surface 4 (such as the user's finger or the stylus that touch in this projection surface 4).This optical device 2 obtains the depth information of the environment of projection surface 4, and generates depth map based on this depth information.So-called depth map is exactly; Depth cameras is positioned at the environment before the camera lens through shooting; And calculate in the captured environment each pixel apart from the distance of depth cameras; And for example adopt that 16 numerical value writes down the object of each pixel representative and the distance between the depth cameras, thereby there are 16 bit value of the incidental expression distance of these each pixels to form the figure that a width of cloth is represented distance between each pixel and the camera.Depth map is sent to control module 3 subsequently, and control module 3 detects at least one object in the said surperficial preset distance of distance along the direction away from said projection surface 4.When detecting this object, then follow the tracks of the touch action of this object in projection surface to form the touch point sequence.Subsequently, 3 pairs of formed touch points of control module sequence is carried out smoothing processing, thereby realizes the painting function on this virtual interacting screen.And these touch point sequences can be combined and generate interactive command, thereby realize the interactive function of virtual touch screen, and final virtual touch screen can change according to the interactive command that is generated.The present invention also can adopt the common foreground object detection of other ordinary cameras and other system to carry out.Be convenient to understand for tracking mode of the present invention, at first introduce the testing process of some foreground object below, multi-object is followed the tracks of necessary enforcement means but this testing process is not realization, and only is the prerequisite of following the tracks of a plurality of objects.Just, the detection of object does not belong to the content to image tracing.
Shown in Figure 180 is the synoptic diagram of the concrete configuration of control module.Said control module 3 comprises depth map processing unit 31, subject detecting unit 32, image enhancing unit 33 and coordinate Calculation and converter unit 34 and tracking cell 35 and smooth unit 36 usually.Depth map processing unit 31 is at first created ID figure based on the depth information that depth cameras obtains under initial conditions; And confirm the position in said touch operation zone based on said ID figure; And as input and handle this depth map from this depth map, and subsequently the connected domain on this depth map is numbered so that background is disposed with its depth map of being caught that depth cameras was sent.Subject detecting unit 32 is based on the depth information from the said depth map of said depth map processing unit 31; Based on a predetermined depth threshold; Depth map is carried out binary conversion treatment, form a plurality of patches, subsequently as candidate target; Based on the size of relation between each patch and the connected domain and plaque area, confirm patch as object.Coordinate Calculation and converter unit 34 calculate the centroid point coordinate of the patch of the object of confirming as, and target coordinate system, the i.e. coordinate system of virtual interacting screen are arrived in its centroid point coordinate transform.Follow the tracks of with smooth unit 35 and follow the tracks of detected a plurality of patches in the multiple image of shooting continuously, generating the coordinate point sequence after a plurality of centroid point conversion, and subsequently the coordinate point sequence that is generated is carried out smoothing processing.
Shown in Fig. 3 (a)-(b) is according to the synoptic diagram of confirming the touch operation zone in the regional virtual touch screen system of hand touch operation that comprises of the present invention.Touch operation region overlapping shown in Fig. 3 (a) is on projection screen.Particularly, the part of projection screen is exactly the touch operation zone, this means that they are on same physical plane.Thus, image frame is projected on the metope simultaneously that the user can carry out touch operation on metope.Coloured image about view field is used to the senses touch operating area.About this situation many known methods have been arranged, just do not described in detail at this.It is overlapping that projection screen is then refused in touch operation zone shown in Fig. 3 (b).In actual use, can be user's desktop on hand with the nonoverlapping operating area of projection screen.System must define a zone that is used for touch operation so that the senses touch operation, then is used to detect this touch operation zone about the depth map of current scene.In the virtual touch screen system shown in Fig. 3 (a)-(b), projector equipment, projector for example, projection frame out on a physical surface is to show and output interface.Optical device, degree of depth camera is for example caught the depth information of scene around the touch operation zone and can be generated depth map based on this depth information.Depth map processing unit 31 comprises a touch operation zone detecting unit; Coloured image that this touch operation zone detecting unit is caught said optical device and depth map are as input; Detect a suitable operating area and calculate its coordinate, this zone be used for the Display projector image with carry out touch operation.This touch operation zone detecting unit system that makes can detect automatically and carry out touch operation for regional extent of user definition, rather than needs user oneself manually to define.Said touch operation zone detecting unit comprises that the coloured image about projection screen of being caught by degree of depth camera as input reaches about the depth map of current scene and the regional apex coordinate of touch operation of calculating virtual touch screen system.The output of this unit is the coordinate on 4 summits in touch operation zone, and these coordinates are used for the system calibration and the coordinate transform of control module.
Shown in Figure 4 is according to the process flow diagram of confirming the touch operation zone in the virtual touch screen of the present invention system.Initial in the system start-up of virtual touch screen; Optical device 2; For example the initial shooting of depth cameras comprises the depth information of the environment in touch operation zone; Depth information based on said initial acquisition is created ID figure, and this ID figure is imported the touch operation zone detecting unit in the depth map processing unit 31.Touch operation zone detecting unit detects the UNICOM's component among said ID figure and the said ID figure of mark at step S41 place after receiving this ID figure.Subsequently, at step S42 place, confirm to state whether have UNICOM's component among the ID figure.If in stating ID figure, there is not UNICOM's component, then finish the testing process that the touch operation zone is detected.If in stating ID figure, there is UNICOM's component, the UNICOM's component among the said ID figure of mark then.Subsequently, get into step S43, at step S43 place all UNICOM's components that are labeled are examined with the UNICOM's component which finds be labeled and comprised two intersection of diagonal of said ID figure.Judge in UNICOM's component of institute's mark, whether there is the UNICOM's component that has comprised two diagonal line intersection points of said ID figure at step S44 place afterwards.If there is not the UNICOM's component that has comprised two diagonal line intersection points of said ID figure; Then finish the testing process that the touch operation zone is detected, otherwise, step S45 got into; At step S45 place; To each UNICOM's component, calculate said UNICOM's component of two diagonal line intersection points of said ID figure and the intersection of diagonal of ID figure of having comprised, obtain the coordinate of these intersection points.Then, at step S46 place,, calculate these intersection points of acquisition and connect the polygon that the back forms through connecting the intersection point that is calculated successively.At step S47 place, judge whether the polygon that is obtained is convex polygon then.If not convex polygon, then finish the testing process that the touch operation zone is detected, otherwise, getting into step S48, the convex polygon that output is obtained is as said touch operation zone.
The detail flowchart that in process flow diagram shown in Figure 4, calculates the intersection of diagonal of said UNICOM's component that has comprised two diagonal line intersection points of said ID figure and ID figure among the step S45 shown in Figure 5.As shown in Figure 5, at first,, calculate said profile and the area that has comprised UNICOM's component of two diagonal line intersection points of said ID figure at step S51 place.The means that this step can adopt the ability those of ordinary skill to be known realize.Subsequently, at step S52 place, the area and a predetermined area threshold value of the UNICOM's component that is relatively calculated.This predetermined area threshold value is generally 1/4 of the ID area of pictural surface, also can be 1/3,1/5,2/7 etc., and the set basis user's of this threshold value concrete needs are selected.Through setting said predetermined area threshold value, can filter out the influence of some less UNICOM's components.Be that desired target area can not be too little.If the area of UNICOM's component is less than described predetermined area threshold value; Then finish further processing to this UNICOM's component; Otherwise; Get into step S53,, calculate said area more than or equal to the intersection of diagonal of UNICOM's component of predetermined area threshold value and ID figure and add up the number of said intersection point at step S53 place.To the situation of different intersection point quantity, handle respectively afterwards at step S54 place.If the quantity of intersection point of judging said statistics at step S54 promptly is not polygonal situation less than 4, then get into step S541, finish further processing subsequently to this UNICOM's component.If judge that at step S54 the quantity of the intersection point of said statistics equals 4, then get into step S542, get into step S56 subsequently.If the quantity of intersection point of judging said statistics at step S54 then gets into step S543 greater than 4, get into step S55 subsequently.At step S55 place, to each summit of ID figure, from the acquisition intersection point, find the nearest intersection point in the said summit of distance, thereby obtain the coordinate of four intersection points, and abandon remaining intersection point.Get into step S56 then.At step S56 place, UNICOM's component of being obtained of output with cornerwise four intersection points of ID figure.Fig. 6 has schematically shown out summit, definite touch operation zone process.A among the figure, b, c, d are exactly UNICOM's component and ID figure intersection of diagonal.
After having obtained UNICOM's component and ID figure intersection of diagonal, just obtained the position in touch operation zone.Just can carry out the simulated touch operation subsequently on this basis.In the process of carrying out touch operation, depth cameras is obtained the depth image in touch operation zone continuously.At step S22 place, depth map processing unit 31 is handled received depth map so that background is disposed from this depth map, and only keeps the depth information of foreground object, and the connected domain in the depth map that is kept is numbered subsequently.Shown in Fig. 7 (a)-(c) is the synoptic diagram of from current depth map, disposing background depth map.The depth map that this employing 16 bit value of institute's diagrammatic sketch show is the convenience in order to explain, is not in the process of embodiment of the present invention, must demonstrate.Shown in Fig. 7 (a) is a kind of synoptic diagram of instance of depth map of background, only is background depth map in the depth map shown in wherein, i.e. the depth map of projection surface, and it does not comprise the depth image of any foreground object (being object).A kind of mode that obtains the depth map of background is; In the starting stage of the method for virtual touch screen system implementation of the present invention virtual touch screen of the present invention function; The snapshots in time of at first obtaining the depth map of current scene and preserving this depth map, thereby the depth map of acquisition background by optical device 2.When obtaining the depth map of this background, in current scene, the place ahead of projection surface 4 (between optical device 2 and the projection surface 4) can not have any dynamic object that is used to touch projection surface.The another kind of mode that obtains the depth map of background is not to use the transient state photo and is to use a series of continuous transient state photos to generate a kind of average background depth map.
Shown in Fig. 7 (b) is the instance of the depth map of being caught of a frame current scene, wherein has an object (for example user's hand or stylus) to touch in projection surface.
Shown in Fig. 7 (c) is the wherein instance of the depth map of background after being eliminated of a frame.A kind of possible mode of removing background depth is to use the depth map of the depth map subtracting background of current scene, and another kind of mode is to scan the depth map of current scene and the every bit and the corresponding point in the background depth map of this depth map are carried out depth value relatively.If the absolute value of the degree of depth difference of the pixel that these are paired is similar and within a predetermined threshold value; Then that the absolute value of the degree of depth difference in the current scene is similar respective point is disposed from the depth map of current scene, does not carry out any change otherwise reservation is put accordingly.Subsequently the connected domain of removing in the background depth map current depth map is afterwards numbered.Connected domain of the present invention refers to such zone: supposing that depth cameras is captured has two 3D points; If their the XY plane (shot picture) that is projected in is gone up adjacent one another are and difference its depth value is not more than given threshold value D, then claim their each other D-be communicated with.If have the D-communication path between any 2 in one group of 3D point, then claim this group 3D point D-connection.If for each the some P in the 3D point that one group of D-is communicated with, each some P does not exist adjacent point under the situation of not interrupting this connection condition, to add in said group on the XY plane, claim that then the 3D point that this group D-is communicated with is communicated with for maximum D-.Connected domain of the present invention is that one group of D-connectivity points and its in the depth map is that maximum D-is communicated with.The connected domain of said depth map is corresponding with continuous blocks (mass) zone that said depth cameras is caught, and connected domain is that the D-connected set of points in the said depth map merges and its maximum D is associated.Therefore, in fact connected domain is numbered is exactly to having 3D point that above-mentioned D-the is communicated with identical numbering of annotating, that is to say, for the pixel that belongs to same connected domain add pour down with numbering, therefore, the numbering matrix of generation connected domain.The continuous block (mass) that the connected domain of said depth map is caught corresponding to said depth cameras.
The numbering matrix of said connected domain is whether a kind of which point in can the said depth map of mark is the data structure of connected domain.Each element in said numbering matrix is corresponding to a point in the depth map, and the value of this element is exactly the numbering (numbering of a connected domain) of the connected domain under this point.
Then, at step S23 place, each point in the depth map is carried out binary conversion treatment, thereby generate some patches, and the pixel of the patch that belongs to same connected domain is added the connected domain numbering as alternative objects based on a depth conditions.To describe concrete binary conversion treatment process below in detail.Shown in Figure 8 is carries out the synoptic diagram of binary conversion treatment with the patch that obtains alternative objects to the depth map of the current scene imported.。As shown in Figure 8, the relative depth information that the present invention is based between the corresponding pixel points of each pixel and background depth map in the depth map in the current scene is carried out the binary conversion treatment process.In an embodiment of the present invention, the depth value of a pixel of retrieval, i.e. distance between the object-point of depth cameras and the pixel representative retrieved from the depth map of said current scene.In Fig. 8; To travel through the mode of all pixels; The depth d of a pixel of retrieval from the depth map of the current scene imported; The depth value b of the pixel that the retrieval from background depth map afterwards is corresponding with the pixel of being retrieved from the depth map of current scene calculates difference (subtraction value) s, the i.e. s=b-d of degree of depth b of depth d and the background pixel point of two target points then.If the difference that is obtained is greater than zero and less than a predetermined distance threshold t, i.e. 0＜s＜t, then the gray-scale value of the pixel that is retrieved in the current scene is set to 255, otherwise then is made as 0.Certainly, this binaryzation also can be directly with two kinds of these positions 0 or 1 of situation difference, as long as can these the two kinds binaryzation modes that make a distinction can be adopted.Through above-mentioned binaryzation mode, can obtain the patch shown in Fig. 9 (b) with a plurality of alternative objects.The precision that the controlled amount system of threshold value t detects object.The threshold value t also hardware specification with depth cameras is relevant.The value of threshold value t is generally the thickness size of a finger or the diameter of common stylus, for example 0.2-1.5 centimetre, is preferably 0.3 centimetre, 0.4 centimetre, 0.7 centimetre, 1.0 centimetres.Threshold value t can adjust according to the environment that wherein uses virtual touch screen system; For distance threshold t, the setting of concrete size can be according to the present invention in application process the concrete size of the diameter of the people's before the screen finger thickness or employed stylus carry out corresponding setting.Usually the possible values of this threshold value t is 1 centimetre.
Shown in Fig. 9 (a) is the synoptic diagram that is used for patch is added the connected domain of numbering.Behind the bianry image that obtains patch, scan search contains the pixel of connected domain numbering, and adds this connected domain numbering to pixel corresponding in the binaryzation patch image, thereby makes some patch have the connected domain numbering, shown in Fig. 9 (b).Patch in the said bianry image (white portion or point) is the candidate that touches the possible destination object in projection surface.According to noted earlier, the binaryzation patch that has connected domain numbering among Fig. 9 (b) possess following two conditions: 1. patch belongs to connected domain.2. difference (subtraction value) s of the depth d of corresponding each pixel of patch and background depth b must be less than threshold value t, i.e. s=b-d＜t.
Then,, the binaryzation patch image of acquisition depth map is carried out enhancement process, to reduce noise unnecessary in the binaryzation patch image and to make the shape of patch become clearer and stable at step S24 place.This step has image enhancing unit 33 to carry out.Particularly, through following steps carry out said enhancement process.
At first, remove the patch that does not belong to connected domain, just with the patch that does not add connected domain numbering at step S23 place directly with its gray-scale value from the highest vanishing, for example with its gray values of pixel points from 255 vanishing.In another kind of mode, become 0 with 1.Thereby obtain the patch binary image shown in Figure 10 (a).
Secondly, remove and belong to the patch of its area S less than the connected domain of an area threshold value Ts.In an embodiment of the present invention, patch belong to a certain connected domain mean this patch at least one the point in connected domain.If the area S of the connected domain under this patch is less than an area threshold value Ts, then corresponding patch then is regarded as noise and from the bianry image of patch, is removed.Otherwise patch then is considered to the candidate of destination object.Area threshold Ts can regulate according to the employed environment of virtual touch screen system.Area threshold Ts is generally 200 pixels.Thus, obtain the binary image of the patch shown in Figure 10 (b).
Then, exactly the patch in the binary image of the patch shown in Figure 10 (b) that obtained is carried out some morphology (morphology) operation.In this embodiment, adopted expansion (dilation) to operate and close (close) operation.At first carrying out an expansive working is to carry out closed operation iteratively then.The iterations that carries out closed operation is a predetermined value, and this predetermined value can be regulated according to the employed environment of virtual touch screen system.This iterations for example can be set to 6 times.Finally obtain the binary image of the patch shown in Figure 10 (c).
At last, if there are a plurality of patches belong to same connected domain, promptly these patches have identical connected domain numbering, then keep to have a patch of maximum area in the patch with identical connected domain numbering and remove other patches.In an embodiment of the present invention, a connected domain can include a plurality of patches.In these patches, the patch that only has maximum area just is considered to destination object, and other patch then is the noise that need be removed.Finally obtain the binary image of the patch shown in Figure 10 (d).
At step S25 place, detect the profile of the patch that obtains, the coordinate of the centroid point of calculating patch also arrives coordinates of targets with the centroid point coordinate transform.This detection, calculating and map function are carried out by coordinate Calculation and converter unit 34.Shown in Figure 11 is detects the synoptic diagram of the process of the coordinate of the centroid point of patch in the binary image of the patch shown in Figure 10 (d).Referring to Figure 11, calculate the coordinate of the centre of form of patch according to the geological information of patch.This computation process comprises: the profile of detection of plaque, calculate the Hu square of this profile and use said Hu square to calculate the coordinate of centroid point.In an embodiment of the present invention, can use multiple known mode to come the profile of detection of plaque.Also can use known algorithm to calculate the Hu square.After the Hu square that obtains said profile, can calculate the coordinate of centroid point through following formula:
(x
0，y
0)＝(m
10/m
00，m
01/m
00)
(x wherein
0, y
0) be the coordinate of centroid point, and m
10, m
01, m
00It is exactly the Hu square.
Coordinate transform is exactly with the coordinate of the centroid point coordinate system from the coordinate system transformation of the bianry image of patch to user interface.The conversion of coordinate system can be adopted known method.
In order to obtain the continuous motion track of touch point; Can be through the touch point in the successive frame depth map captured in the continuous detecting virtual touch screen system of the present invention; Thereby follow the tracks of detected a plurality of patch to produce the sequence of a plurality of points, obtain the movement locus of touch point thus.
Particularly; Be exactly at step S26 place; Centroid point coordinate on user interface to obtaining the patch of every two field picture behind every frame depth map execution in step S21-S25 of continuous shooting is followed the tracks of, and generates centroid point sequence (being track), and acquisition centroid point sequence is carried out smoothing processing.This tracking and smooth operation are undertaken by tracking cell 35.
The synoptic diagram of the track that shown in Figure 12 is user's finger or stylus moves on the screen of virtual touch screen.The movement locus that has wherein shown two objects (finger).This only is an instance.In other cases, can a plurality of objects, for example 3,4,5 objects are decided according to the actual requirements.
The process flow diagram that is used to follow the tracks of the object that is detected shown in Figure 13.Carry out trace flow shown in figure 13 repeatedly, finally can obtain the preceding motion of objects track arbitrarily of screen.Particularly, carrying out tracking operation is exactly the centroid point coordinate of the patch in the new detected depth map in user interface to be included into before obtain in any track.
According to the centroid point coordinate of a plurality of institutes detection of plaque in user interface, follow the tracks of a plurality of new detected patches, thereby produce many tracks and trigger relevant touch event about these tracks.To follow the tracks of patch, just need classify and the centroid point coordinate of patch is placed in one have a few on time and space in the relevant point sequence patch.Have only the point in the identical sequence just can merge into a track.Shown in figure 12; If virtual touch screen system is supported painting function; Then the point in the sequence shown in Figure 12 is just represented the paint command on the pseudo operation zone, and the point in same sequence just can couple together and form a curve as shown in Figure 12 so.
In the present invention, can follow the tracks of three kinds of touch events: touch beginning, touch to move and touch and finish.Touching object that beginning just is meant that institute will detect, to touch pseudo operation regional and track begun.Touch to move and to refer to the detected object of wanting and just touching on the pseudo operation zone and during track just continuing in projection surface.And touch to finish to refer to surface and the motion track end that object that institute will detect leaves the pseudo operation zone.
Shown in figure 13, at step S91, receive based on the centroid point coordinate of new patch in user interface of a frame depth map according to the detected object of step S21-S25, this is the output of coordinate Calculation and converter unit 34.
Subsequently, at step S92 place, to before the patch of each frame depth map carried out tracking processing after each point sequence in all point sequences that obtain (i.e. all existing tracks, below all become existing track), this has the nearest new patch of track computed range.All tracks that touch the object on touch screen (being the pseudo operation zone) all are retained in the virtual touch screen system.Each track keeps the patch that quilt is followed the tracks of, and this patch of being followed the tracks of is the last patch that is endowed this track.The distance of new patch of the present invention and existing track refers to the distance between the last patch in a new patch and the existing track.
Then,, new patch is included into apart from its nearest existing track, and triggers the touch moving event at step S93 place.
Then, at step S94 place, if to an existing track; Do not exist any new patch approaching with it, in other words, if all new patches have been included into other existing tracks respectively; Then delete the existing track of this bar, and trigger touch End Event to this existing track.
At last, at step S95 place, if to each new patch; Do not exist any existing track approaching with it; In other words, obtain all existing tracks before and all deleted owing to trigger the touch End Event, the distance of perhaps new patch and all existing tracks is not within certain distance threshold scope; Then confirm the starting point that this new patch is new track, and trigger the beginning incident that touches.
Carry out above-mentioned steps S91-S95 repeatedly, realize tracking, thereby will belong to the track that consists of a little of same point sequence centroid point coordinate in user interface of the patch in the successive frame depth map.
When having many existing tracks, will be to every existing track execution in step S92 repeatedly.The particular flow sheet of tracking cell of the present invention 35 execution in step S92 shown in Figure 14.
At first, at step S101 place, examine and traveled through all existing tracks.This can just can realize through a simple counter.If all carried out step S92, then end step S92 to all existing tracks.If no, then advance to step S102.
At step S102 place, import the existing track of next bar.At step S103 place,, seek subsequently apart from its new patch that closes on to the existing track of being imported.Get into step S104 then.
At step S104 place, determine whether to be directed against the existing track of being imported and found the new patch that closes on apart from it.If the new patch that the existing track that has found distance to be imported closes on then advances to step S105, otherwise, step S108 then got into.
At step S108 place, there is not the new patch that closes on owing to be directed against the existing track of being imported, therefore, be " the existing track that will delete " with this existing Trajectories Toggle of importing.Turn back to step S101 subsequently.Thus, will trigger the touch End Event to this " the existing track that will delete " at step S94 place.
At step S105 place, whether the new patch that the existing track of confirming apart from imported closes on also is the new patch that closes on apart from other existing tracks.In other words, confirm this new patch whether belong to simultaneously apart from two or more existing track close on new patch.If judge this new patch belong to simultaneously two or more existing track close on new patch, then handle getting into step S106, otherwise, handle getting into step S109.
At step S109 place, because this new patch only is the new patch that closes on of the existing track imported, therefore, this new patch is included into the existing track imported as its nearest new patch, promptly become one of point in the point sequence of this existing track.Handle afterwards and turn back to step S102.
At step S106 place and since this new patch belong to simultaneously two or more existing track close on new patch, then calculate this new patch and under the distance of every existing track in many existing tracks.Then at step S107 place; Compare size in step S106 place calculated distance; And confirm between this new patch and the existing track of being imported distance whether in institute's calculated distance for minimum, promptly confirm this new patch and the existing track imported between apart from whether than and other existing tracks between distance all little.If distance for minimum, then handles getting into step S109 between the existing track of confirming this new patch and being imported in the calculated distance of step S106 place, otherwise, handle getting into step S108.
Carry out above-mentioned steps S101-109 repeatedly, thus the processing that performing step S92 is carried out, thus can travel through the patch of the new detection of all existing tracks and input.
Shown in Figure 15 is to be directed against the existing track of being imported to seek the process flow diagram apart from its new patch that closes on.Shown in figure 15, at step S111 place, examine the distance of closing on that whether is directed against between the existing track that all new patches of being imported have all calculated and imported.If close on distance between the existing track that has all calculated and imported to all new patches, then handle getting into step S118, otherwise, handle getting into step S112.
At step S118 place, confirm whether the tabulation as the new patch that closes on of the existing track of being imported is empty.If empty, end process then, otherwise, get into step S119.At step S119 place, in all the new patches that close on tabulations, find the nearest new patch with this existing track of importing, and will this nearest new patch be included into the point sequence of the existing track of being imported.End step S103 afterwards.
At step S112 place, the next new patch of input.Subsequently, at step S113 place, calculate next new patch and the existing track imported between distance.Then, at step S114 place, confirm that whether distance between the next new patch calculated and the existing track of being imported is less than a predetermined threshold value.Less than a predetermined distance threshold Td, then handle entering step S115 if confirm distance between the next new patch calculated and the existing track of being imported, otherwise, turn back to step S111.Distance threshold Td is set to the distance of 10-20 pixel usually here, is preferably the distance of 15 pixels.This threshold value Td adjusts according to the employed environment of virtual touch screen system.In the present invention, if the distance between a new patch and the existing track less than described distance threshold Td, then is referred to as this new patch and should closes on by existing track.
At step S115 place, the new patch of the said next one is inserted in the new patch tabulation of the candidate who belongs to the existing track of being imported.Subsequently at step S116 place, whether the size of the new patch tabulation of candidate of confirming to belong to the existing track of being imported is less than a predetermined size threshold value Tsize.If the size of confirming to belong to the new patch tabulation of the candidate of the existing track of being imported is then handled and is turned back to step S111 less than a predetermined size threshold value Tsize, otherwise, handle getting into step S117.At step S117 place, the longest new patch of candidate of distance that will belong between the existing track in the new patch tabulation of the candidate of the existing track of being imported and that imported is deleted from said tabulation, turns back to step S111 afterwards.
Carry out the step shown in Figure 15 repeatedly, thus completing steps S103.
The above flow process of following the tracks of patch coordinate in user interface to successive image frame of having described with reference to accompanying drawing 13-11.Through above-mentioned tracking operation, the touch that triggers institute's detected object begins incident, touches moving event or touches End Event.Thus, obtain the motion track of institute's detected object on virtual touch screen at last.Shown in Figure 17 (a) is the synoptic diagram of the motion track of detected object on virtual touch screen that obtain of a kind of the present invention of employing.
Obviously, this motion track of detected object on virtual touch screen shown in Figure 17 (a) that tentatively obtains seems comparatively mixed and disorderly.This track also need carry out smoothing processing, so that obtain level and smooth object motion track.Shown in Figure 17 (b) is the synoptic diagram through the object motion track after the smoothing processing.The point sequence of shown in Figure 16 the is a kind of motion track of detected object on virtual touch screen that a strip adoption the present invention is obtained carries out the method for smoothing processing.
The point sequence smoothing processing is exactly that coordinate to the point in this sequence is optimized so that point sequence is level and smooth.Shown in figure 16; Input constitutes the first round input of the original point sequence
(n is a positive integer) of a track as iteration, the i.e. output of patch tracking.In accompanying drawing 16, original point sequence
is first row that sit up.Formula below using then is according to the sequence of calculating the next round iteration from the result of last round of iteration
Wherein
is the point in the point sequence; K is the iteration sign; N is the point sequence sign, and m is the radix of iteration point.
Repeat this iterative computation, up to reaching predetermined iteration threshold.In an embodiment of the present invention, the m parameter can be 3-7, is set to 3 in an embodiment of the present invention, and this means this, and each next stage point has 3 some iteration of upper level to obtain.This iteration threshold is 3.
Through above-mentioned iterative computation, finally obtaining is through the object motion track after the smoothing processing shown in Figure 17 (b).
Here, in this manual, the processing of being carried out by computing machine according to program not needs according to carrying out with time series like the order of flowchart text.That is the processing of, being carried out by computing machine according to program comprises processing (for example parallel processing and target processing) parallel or that carry out separately.
Likewise, program can go up at a computing machine (processor) and carry out, and perhaps can be carried out by many computer distribution types.In addition, program can be transferred to the remote computer at the there executive routine.
Will be understood by those skilled in the art that,,, various modifications, combination, part combination and alternative can occur as long as it falls in the scope of accompanying claims or its equivalent according to designing requirement and other factors.
Claims (7)
1. virtual touch method that is used for touchscreen system comprises:
The initial depth information that obtains to comprise the regional environment of a touch operation is created ID figure based on the depth information of said initial acquisition, and confirms the position in said touch operation zone based on said ID figure;
Obtain continuously the image of the environment in said definite touch operation zone;
From the every two field picture that is obtained, detect the candidate's patch be positioned at least one object of preset distance before the said touch operation zone;
The relation of centroid point on time and space according to the patch that obtains in adjacent two two field pictures in front and back is included into corresponding point sequence with each patch,
The step of the position in wherein said definite said touch operation zone comprises:
UNICOM's component among detection and the said ID figure of mark;
Whether UNICOM's component of mark comprises two intersection of diagonal of said ID figure to confirm to detect institute also;
Detect and UNICOM's component of mark comprises under the situation of two intersection of diagonal of said ID figure, the diagonal line that calculates said ID figure detects the also intersection point of UNICOM's component of mark with institute; And
Connect the intersection point that is calculated successively, and institute's connection acquisition convex polygon is confirmed as said touch operation zone.
2. the virtual touch method that is used for touchscreen system according to claim 1, wherein, said touch operation zone is overlapping with the view field of said touchscreen system.
3. the virtual touch method that is used for touchscreen system according to claim 1, wherein, said touch operation zone is not overlapping with the view field of said touchscreen system.
4. according to any described virtual touch method that is used for touchscreen system of claim 1-3, the step of the UNICOM's component among the said ID figure of wherein said detection and mark comprises:
Calculate each its area of UNICOM's component;
Whether the area of confirming to be calculated greater than a predetermined area threshold, and abandons the connected component of its area less than said predetermined area threshold value.
5. according to any described virtual touch method that is used for touchscreen system of claim 4, wherein, 1/4th of the area that said predetermined area threshold value is said ID figure.
6. according to any described virtual touch method that is used for touchscreen system of claim 4, the wherein said intersection point that is calculated that connects successively, and connects acquisition convex polygon is confirmed as the regional step of said touch operation comprise:
Confirm that whether connect the formed shape of being calculated of intersection point successively is convex polygon, if not convex polygon, then abandons formed UNICOM component.
7. virtual touch screen system comprises:
Projector, with image projection to in the projection surface;
Depth cameras, acquisition comprise the depth information of the environment in touch operation zone;
The depth map processing unit is created ID figure based on the depth information that depth cameras obtains under initial conditions, and confirms the position in said touch operation zone based on said ID figure;
Subject detecting unit after initial conditions the continuous every two field picture that obtains, detects the candidate's patch that is positioned at least one object of preset distance before the determined touch operation zone from depth cameras;
Tracking cell is included into corresponding point sequence according to the relation of centroid point on time and space of the patch that obtains in adjacent two two field pictures in front and back with each patch,
Wherein, the position in said touch operation zone is confirmed in definite unit, said touch operation zone through following process: the UNICOM's component among detection and the said ID figure of mark; Whether UNICOM's component of mark comprises two intersection of diagonal of said ID figure to confirm to detect institute also; Detect and UNICOM's component of mark comprises under the situation of two intersection of diagonal of said ID figure, the diagonal line that calculates said ID figure detects the also intersection point of UNICOM's component of mark with institute; And connect the intersection point that is calculated successively, and institute's connection acquisition convex polygon is confirmed as said touch operation zone.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110140079.4A CN102799344B (en) | 2011-05-27 | 2011-05-27 | Virtual touch screen system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110140079.4A CN102799344B (en) | 2011-05-27 | 2011-05-27 | Virtual touch screen system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102799344A true CN102799344A (en) | 2012-11-28 |
| CN102799344B CN102799344B (en) | 2014-11-19 |
Family
ID=47198462
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110140079.4A Active CN102799344B (en) | 2011-05-27 | 2011-05-27 | Virtual touch screen system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102799344B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104281381A (en) * | 2013-07-05 | 2015-01-14 | 现代自动车株式会社 | Apparatus and method for controlling of user interface equipped touch screen |
| CN104571513A (en) * | 2014-12-31 | 2015-04-29 | 东莞市南星电子有限公司 | Method and system for simulating touch instructions by shielding camera shooting area |
| CN104951200A (en) * | 2014-03-26 | 2015-09-30 | 联想(北京)有限公司 | Method and device for executing interface operations |
| CN112306331A (en) * | 2020-10-26 | 2021-02-02 | 广州朗国电子科技有限公司 | Touch penetration processing method and device, storage medium and all-in-one machine |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050193349A1 (en) * | 2004-03-01 | 2005-09-01 | Fuminori Sato | Image display system and information processing apparatus |
| CN101393497A (en) * | 2008-10-30 | 2009-03-25 | 上海交通大学 | Multi-point touch method based on binocular stereo vision |
| US20100302145A1 (en) * | 2009-06-01 | 2010-12-02 | Microsoft Corporation | Virtual desktop coordinate transformation |
| CN101952818A (en) * | 2007-09-14 | 2011-01-19 | 智慧投资控股67有限责任公司 | Processing based on the user interactions of attitude |
| CN101963840A (en) * | 2009-07-22 | 2011-02-02 | 罗技欧洲公司 | Be used for system and method long-range, the virtual screen input |
-
2011
- 2011-05-27 CN CN201110140079.4A patent/CN102799344B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050193349A1 (en) * | 2004-03-01 | 2005-09-01 | Fuminori Sato | Image display system and information processing apparatus |
| CN101952818A (en) * | 2007-09-14 | 2011-01-19 | 智慧投资控股67有限责任公司 | Processing based on the user interactions of attitude |
| CN101393497A (en) * | 2008-10-30 | 2009-03-25 | 上海交通大学 | Multi-point touch method based on binocular stereo vision |
| US20100302145A1 (en) * | 2009-06-01 | 2010-12-02 | Microsoft Corporation | Virtual desktop coordinate transformation |
| CN101963840A (en) * | 2009-07-22 | 2011-02-02 | 罗技欧洲公司 | Be used for system and method long-range, the virtual screen input |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104281381A (en) * | 2013-07-05 | 2015-01-14 | 现代自动车株式会社 | Apparatus and method for controlling of user interface equipped touch screen |
| CN104281381B (en) * | 2013-07-05 | 2019-08-23 | 现代自动车株式会社 | The device and method for controlling the user interface equipped with touch screen |
| CN104951200A (en) * | 2014-03-26 | 2015-09-30 | 联想(北京)有限公司 | Method and device for executing interface operations |
| CN104951200B (en) * | 2014-03-26 | 2018-02-27 | 联想(北京)有限公司 | A kind of method and apparatus for performing interface operation |
| CN104571513A (en) * | 2014-12-31 | 2015-04-29 | 东莞市南星电子有限公司 | Method and system for simulating touch instructions by shielding camera shooting area |
| CN112306331A (en) * | 2020-10-26 | 2021-02-02 | 广州朗国电子科技有限公司 | Touch penetration processing method and device, storage medium and all-in-one machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102799344B (en) | 2014-11-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102841733B (en) | Virtual touch screen system and method for automatically switching interaction modes | |
| US10895868B2 (en) | Augmented interface authoring | |
| CN102799318B (en) | A kind of man-machine interaction method based on binocular stereo vision and system | |
| CN100407798C (en) | Three-dimensional geometric mode building system and method | |
| CN102566827A (en) | Method and system for detecting object in virtual touch screen system | |
| EP2790089A1 (en) | Portable device and method for providing non-contact interface | |
| EP3137979A1 (en) | Air and surface multi-touch detection in mobile platform | |
| CN104254817A (en) | Rapid gesture re-engagement | |
| CN102799317A (en) | Smart interactive projection system | |
| CN102754047A (en) | Methods and systems for position detection using an interactive volume | |
| CN102073414B (en) | Multi-touch tracking method based on machine vision | |
| Katz et al. | A multi-touch surface using multiple cameras | |
| Geer | Will gesture recognition technology point the way? | |
| CN102799344B (en) | Virtual touch screen system and method | |
| CN102541417B (en) | Multi-object tracking method and system in virtual touch screen system | |
| Clark et al. | Seamless interaction in space | |
| KR20150112198A (en) | multi-user recognition multi-touch interface apparatus and method using depth-camera | |
| KR101461145B1 (en) | System for Controlling of Event by Using Depth Information | |
| CN102004584B (en) | Method and device of positioning and displaying active pen | |
| CN111354018B (en) | Object identification method, device and system based on image | |
| CN109871178A (en) | A kind of virtual touch screen system based on image recognition | |
| CN116301551A (en) | Touch identification method, touch identification device, electronic equipment and medium | |
| Ahsan et al. | Interactive white board using gestures with KINECT | |
| Liu | Semantic mapping: a semantics-based approach to virtual content placement for immersive environments | |
| Soares et al. | Locoboard: low-cost interactive whiteboard using computer vision algorithms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |