CN103984101A - Display content control method and device - Google Patents
Display content control method and device Download PDFInfo
- Publication number
- CN103984101A CN103984101A CN201410239854.5A CN201410239854A CN103984101A CN 103984101 A CN103984101 A CN 103984101A CN 201410239854 A CN201410239854 A CN 201410239854A CN 103984101 A CN103984101 A CN 103984101A
- Authority
- CN
- China
- Prior art keywords
- operating body
- displaying contents
- shows
- hmd
- action message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The embodiment of the invention provides a display content control method and device. The display content control method and device are applied to a head-mounted intelligent device. The head-mounted intelligent device comprises a head-mounted display (HMD) and an image collector. A user achieves control over the display content in the display area of the HMD through an operation body. The display content control method comprises the steps that the initial position of the operation body is determined through the image collector; the operation position, mapped to the display area of the HMD, of the initial position of the operation body is determined, and a first display object in the display content is determined according to the operation position; action information of the operation body is collected through the image collector; the morphological characteristics, in the display content, of the first display object are adjusted according to the action information. The display content control method and device are convenient to control and operate, and meet the control requirement of the user to some extent.
Description
Technical field
The application relates to electronic technology field, relates in particular to a kind of displaying contents control method and device.
Background technology
Head-wearing type intelligent equipment refers to and utilizes wearable technology to carry out intelligentized design to headset equipment, and the smart machine of developing, such as intelligent glasses etc.
Possesses head mounted display (English: Head-mounted display, be called for short: head-wearing type intelligent equipment HMD), by exporting the contents such as image to HMD, user is in the time wearing head-wearing type intelligent equipment, can watch the displaying contents of HMD, and the visual imaging space of displaying contents of the HMD watching due to user is not identical with the display interface of head mounted display, and the visual imaging space of the displaying contents that user watches is virtual vision space.On user perspective, the displaying contents of the HMD that user watches is virtual content, and virtual content is added on the actual environment that user watches, thereby can realize actual situation combination,, for example, in real scene navigation scene, can observe in pavement image directly show navigator map etc. user.
Realizing in process of the present invention, inventor finds, the existing head-wearing type intelligent equipment that possesses HMD, the control of HMD displaying contents is only limited to the control of the display parameter to displaying contents, as display brightness, and only can the physical button by equipment realize, operate not conveniently, cannot meet user's demand for control.
Summary of the invention
In view of this, the application provides a kind of displaying contents control method and device, and facility simple to operate has increased the control mode to displaying contents, has improved user's experience.
For achieving the above object, the application provides following technical scheme:
First aspect, a kind of displaying contents control method is provided, be applied in head-wearing type intelligent equipment, described head-wearing type intelligent equipment comprises head mounted display HMD and image acquisition device, user realizes the control to the displaying contents in the viewing area of described HMD by operating body, and described method comprises:
Determine the initial position of described operating body by described image acquisition device;
The initial position of determining described operating body is mapped to the operating position in the viewing area of described HMD, and determines that according to described operating position first in described displaying contents shows object;
Gather the action message of described operating body by described image acquisition device;
According to described action message, to adjust described first and show the morphological feature of object in described displaying contents, described morphological feature at least comprises the one in display position, display size and display shape.
In the first of described first aspect may implementation, the described initial position of determining described operating body is mapped to the operating position in the viewing area of described HMD, and determines that according to described operating position first in described displaying contents shows that object comprises:
According to the initial position of described operating body, calculate described operating body and be mapped to the relative position in virtual vision space; Described virtual vision space is the visual imaging space of user while watching described displaying contents;
According to eyes of user to the distance of described HMD the ratio with the distance to described virtual vision space, calculate the operating position in the displaying contents that described relative position is mapped to described HMD output;
Determine that in described displaying contents, being positioned at first of described operating position place shows object.
In conjunction with the first possibility implementation of described first aspect or described first aspect, the second possibility implementation of described first aspect is also provided, described according to described action message, adjust described first and show that the morphological feature of object in described displaying contents comprises:
Determine that described first shows the pending region of being blocked by described operating body in object;
In described displaying contents, transparent processing is carried out in described pending region, first after being adjusted shows object.
In conjunction with the second possibility implementation of described first aspect, the third possibility implementation of described first aspect is also provided, described definite described first shows that the pending region of being blocked by described operating body in object comprises:
The shield portions of the described operating body corresponding with described action message that inquiry is preset;
Determine that described first shows the pending region of being blocked by the shield portions of described operating body in object.
In conjunction with the first possibility implementation of described first aspect or described first aspect, also provide the 4th kind of described first aspect may implementation, the action message of operating body characterizes described operating body and moves to end position from described initial position described in the action message of described operating body;
Described according to described action message, adjust described first and show the morphological feature of object in described displaying contents, comprising:
Obtain primary importance and the second place of described operating body in described collection space, described primary importance and the described second place are that initial position moves to any two positions in the movement locus that end position forms described in described operating body;
Determine that described first shows three position of object in described displaying contents;
According to the variation relation of described primary importance and the described second place, determine the 4th position relevant to described the 3rd position in described displaying contents;
Adjust described first and show that three position of object from described displaying contents moves to described the 4th position.
In conjunction with the first possibility implementation of described first aspect or described first aspect, the 5th kind of possibility implementation of described first aspect is also provided, described according to described action message, adjust described first and show that the morphological feature of object in described displaying contents comprises:
Obtain the first control strategy corresponding to described action message;
According to described the first control strategy, adjust the display shape of described the first demonstration object.
Second aspect, a kind of displaying contents control device is provided, be applied in head-wearing type intelligent equipment, described head-wearing type intelligent equipment comprises head mounted display HMD and image acquisition device, user realizes the control to the displaying contents in the viewing area of described HMD by operating body, and described device comprises:
Initial position determination module, for determining the initial position of described operating body by described image acquisition device;
Position mapping block, for determining that the initial position of described operating body is mapped to the operating position of the viewing area of described HMD;
Object determination module, for determining that according to described operating position first of described displaying contents shows object;
Action acquisition module, for gathering the action message of described operating body by described image acquisition device;
Adjusting module, for according to described action message, adjusts described first and shows the morphological feature of object in described displaying contents, and described morphological feature at least comprises the one in display position, display size and display shape.
In the first possibility implementation of described second aspect, described position mapping block comprises:
The first map unit, for according to the initial position of described operating body, calculates described operating body and is mapped to the relative position in virtual vision space; Described virtual vision space is the visual imaging space of user while watching described displaying contents;
The second map unit, for according to eyes of user to the distance of described HMD the ratio with the distance to described virtual vision space, calculate the operating position in the displaying contents that described relative position is mapped to described HMD output.
In conjunction with the first possibility implementation of described second aspect or described second aspect, the second possibility implementation of described second aspect is also provided, described adjusting module comprises:
Region determining unit, for determining that described first shows the pending region that object is blocked by described operating body;
The first adjustment unit, at described displaying contents, carries out transparent processing by described pending region, and first after being adjusted shows object.
In conjunction with the second possibility implementation of described second aspect, the third possibility implementation of described second aspect is also provided, described region determining unit comprises:
Query unit, for inquiring about the shield portions of the default described operating body corresponding with described action message;
Subelement is determined in region, for determining that described first shows the pending region that object is blocked by the shield portions of described operating body.
In conjunction with the first possibility implementation of described second aspect or described second aspect, the 4th kind of possibility implementation of described second aspect is also provided, the action message of described operating body characterizes described operating body and moves to end position from described initial position; Described adjusting module comprises:
Position acquisition unit, for obtaining primary importance and the second place of described operating body in described collection space, described primary importance and the described second place are that initial position moves to any two positions in the movement locus that end position forms described in described operating body;
The first determining unit, for determining that described first shows three position of object at described displaying contents;
The second determining unit, for according to the variation relation of described primary importance and the described second place, determines the 4th position relevant to described the 3rd position in described displaying contents;
The second adjustment unit, shows that for adjusting described first object moves to described the 4th position from the 3rd position of described displaying contents.
In conjunction with the first possibility implementation of described second aspect or described second aspect, the 5th kind of possibility implementation of described second aspect is also provided, described adjusting module comprises:
Strategy acquiring unit, for obtaining the first control strategy corresponding to described action message;
The 3rd adjustment unit, for according to described the first control strategy, adjusts the display shape of described the first demonstration object.
Known via above-mentioned technical scheme, compared with prior art, the application provides a kind of displaying contents control method and device, user can realize the control to displaying contents by operating body, the demonstration object in displaying contents can be adjusted, without the physical button of touching head-wearing type intelligent equipment, can realize the control to displaying contents, simple to operate, increase the control mode to displaying contents, meet to a certain extent user's request.And increase the interactive of user and displaying contents, the adjustment of displaying contents has been considered to the operating position of operating body in viewing area, made accurately to determine first of operation and show object, can realize virtual combination display effect more accurately.
Brief description of the drawings
In order to be illustrated more clearly in the embodiment of the present application or technical scheme of the prior art, to the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described below, apparently, accompanying drawing in the following describes is only the application's embodiment, for those of ordinary skill in the art, do not paying under the prerequisite of creative work, other accompanying drawing can also be provided according to the accompanying drawing providing.
The process flow diagram of the embodiment of a kind of displaying contents control method that Fig. 1 provides for the embodiment of the present application;
Fig. 2 is a kind of imaging schematic diagram of head-wearing type intelligent equipment in the embodiment of the present application;
The process flow diagram of a kind of another embodiment of displaying contents control method that Fig. 3 provides for the embodiment of the present application;
The process flow diagram of a kind of another embodiment of displaying contents control method that Fig. 4 provides for the embodiment of the present application;
Fig. 4 a is that the one of the first demonstration object in the embodiment of the present application shows schematic diagram;
Fig. 4 b is that the one of the second demonstration object in the embodiment of the present application shows schematic diagram;
Fig. 4 c is that the another kind of the first demonstration object in the embodiment of the present application shows schematic diagram;
Fig. 4 d is that the another kind of the second demonstration object in the embodiment of the present application shows schematic diagram.
The structural representation of the embodiment of a kind of displaying contents control device that Fig. 5 provides for the embodiment of the present application;
The structural representation of a kind of another embodiment of displaying contents control device that Fig. 6 provides for the embodiment of the present application;
A kind of structural representation of adjusting module in a kind of displaying contents control device that Fig. 7 the embodiment of the present application provides;
The another kind of structural representation of adjusting module in a kind of displaying contents control device that Fig. 8 provides for the embodiment of the present application;
Another structural representation of adjusting module in a kind of displaying contents control device that Fig. 9 provides for the embodiment of the present application;
The structural representation of the embodiment of a kind of head-wearing type intelligent equipment that Figure 10 provides for the embodiment of the present application.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present application, the technical scheme in the embodiment of the present application is clearly and completely described, obviously, described embodiment is only some embodiments of the present application, instead of whole embodiment.Based on the embodiment in the application, those of ordinary skill in the art are not making the every other embodiment obtaining under creative work prerequisite, all belong to the scope of the application's protection.
The application's technical scheme, can be applied in head-wearing type intelligent equipment, and head-wearing type intelligent equipment generally includes fixing firmware, can be worn on head by user.
The applied head-wearing type intelligent equipment of present techniques scheme comprises head mounted display (English: Head-mounted display, abbreviation: HMD), HMD is visual display, can realize the combination of virtual content and actual environment.
In the embodiment of the present application, user wears in the situation of head-wearing type intelligent equipment, user can realize the control to displaying contents by operating body, demonstration object in displaying contents can be adjusted, user, without the physical button of touching head-wearing type intelligent equipment, can realize the control to displaying contents, simple to operate, increase the control mode to displaying contents, met to a certain extent user's request.And increase the interactive of user and displaying contents, the adjustment of displaying contents has been considered to the operating position of operating body in viewing area, make accurately to determine first of operation and show object, can realize virtual combination display effect more accurately, improved user's experience.
Below in conjunction with accompanying drawing, present techniques scheme is described in detail.
The process flow diagram of the embodiment of a kind of displaying contents control method that Fig. 1 provides for the embodiment of the present application, described in the embodiment of the present application, control method is mainly used in head-wearing type intelligent equipment, this head-wearing type intelligent equipment comprises HMD and image acquisition device, and user can realize the control to the displaying contents in the viewing area of described HMD by operating body.
The method can comprise following step:
101: the initial position of determining described operating body by described image acquisition device.
Wear described head-wearing type intelligent equipment user, and in situation by described head mounted display output display content, can gather gathering space by described image acquisition device.
The initial position of described operating body is that image acquisition device arrives in collection space acquisition.This initial position is that operating body is arranged in the position that gathers space.
When user wears head-wearing type intelligent equipment, the imaging space of viewed displaying contents is not identical with the display interface of head mounted display.The imaging space of the displaying contents that user observes is user's virtual vision space.
In virtual vision space, the viewed displaying contents of user, the displaying contents in the viewing area of HMD is mapped to the virtual content in virtual vision space, on user perspective, is added on the actual environment that user observes.
As shown in Figure 2, show a kind of imaging schematic diagram of head-wearing type intelligent equipment.In the viewing area of HMD 100 output display contents, when user wears head-wearing type intelligent equipment, viewed virtual content is imaged in virtual vision space 200, and described virtual vision space is the visual imaging space of user while watching described displaying contents.
Gather space 300 and virtual vision space 200 space overlap at least partly, can refer to that gathering space comprises virtual vision space.
Wherein, collection space 300 is to be determined by the angular field of view of the image acquisition device 400 of head-wearing type intelligent equipment, and image acquisition device 400 specifically can refer to that shooting is first-class.
Operating body refers to the object being collected by wear-type smart machine of user's true operation, a kind of possible implementation, this operating body is the user's who collects hand, and the action message of operating body specifically can refer to gesture information, as the operating body 500 in Fig. 2.
Certainly, this operating body can also refer to the object that user's hand is managed, as rod shape thing etc.
102: the initial position of determining described operating body is mapped to the operating position in the viewing area of described HMD, and determines that according to described operating position first in described displaying contents shows object.
In order to realize the control to displaying contents according to the action message of operating body, need the first demonstration object in the accurate displaying contents of determining operating body operation.And in prior art, the operational motion that conventionally can not know operating body is which part in corresponding displaying contents, thereby can not realize the accurate control to displaying contents, poor with user interaction.
In the embodiment of the present application, operating body is mapped in the viewing area of HMD gathering the initial position in space, determines the operating position being mapped in viewing area, thereby can accurately determine the demonstration object in the displaying contents of operating body operation.
Owing to there are differences between human eye visual angle and collection visual angle, if head-wearing type intelligent equipment wants to know in human eye visual angle that operating body is in the time that virtual content is accurately combined, operating body is mapped to the operating position of HMD viewing area, can be according to the proportionate relationship of virtual content and displaying contents, know that the relative position of operating body in virtual vision space is mapped to the operating position in HMD viewing area.
In a kind of possible implementation, this first shows object, can refer to that default position taking operating body is as reference point, and the displaying contents at the operating position place of this reference point mapping is the first demonstration object.For example, when operating body is hand, reference point can be selected forefinger summit.
In the possible implementation of another kind, this first demonstration object, can refer to the content that operating body collected in displaying contents blocks.
In another possible implementation, this the first demonstration object, can be according to the content of the operating position place instruction of the reference point of operating body, or the content of being blocked by operating body, determined wherein to have given shape or meet pre-conditioned information be the first demonstration object.This given shape or meet pre-conditioned can obtaining by analyzing the information of displaying contents.For example, comprising in the displaying contents with the information such as water tumbler, school bag, according to the action message of operating body, the forefinger summit of identifying operating body is positioned at arbitrary position of viewing area, water tumbler place, and first shows that object can be defined as water tumbler.
103: the action message that gathers described operating body by described image acquisition device.
The action message of operating body can comprise that operating body is in the action message of two dimensional surface or in three-dimensional action message, specifically can refer to operating body from initial position move to profile information, the deformation characteristics of operating body and the positional information of operating body of motion track information, operating body that end position forms at least one etc.
The depth information that operating body can gather by degree of depth camera or binocular camera shooting head in a kind of possible implementation of three-dimensional action message obtains.
The action of the execution of operating body can be that user is while wearing head-wearing type intelligent equipment, according to the virtual content arriving in virtual vision space observation, the operation that control operation body is carried out the virtual objects in virtual content, for example click a certain virtual objects, now head-wearing type intelligent equipment can gather the action message of space acquisition to operating body.
The collection identification of the action message to operating body, take operating body as example as staff, can set up in advance hand model, thereby when operating body detected, according to hand model, can identify profile and the key point of staff, and key point is as finger, joint etc.Thereby can accurately identify the action message of operating body.
104: according to described action message, adjust described first and show the morphological feature of object in described displaying contents.
Described morphological feature at least comprises the one in display position, display size and display shape.
The morphological feature of the first demonstration object is adjusted, make described first to show action message described in object matching, when being mapped in virtual vision space, from user perspective, first shows object and accurately combination of operating body, and mates described action message.
In the present embodiment, user wears head-wearing type intelligent equipment, user can realize the control to displaying contents in HMD viewing area by operating body, demonstration object in displaying contents is adjusted, user, without the physical button of touching head-wearing type intelligent equipment, can realize the control to displaying contents, simple to operate, increase the control mode to displaying contents, met to a certain extent user's request.And increase the interactive of user and displaying contents, the adjustment of displaying contents has been considered to the operating position of operating body in viewing area, made accurately to determine the first demonstration object, realized virtual combination display effect more accurately.
The initial position that head-wearing type intelligent equipment how to confirm goes out described operating body is mapped to the operating position in the viewing area of described HMD, can there is multiple implementation, in a kind of possibility implementation, as shown in Figure 3, show the process flow diagram of a kind of another embodiment of displaying contents control method that the embodiment of the present application provides, step 101 in the present embodiment, step 103～104 can embodiment shown in Figure 1, and wherein, step 102 can comprise the following steps:
301: according to the initial position of described operating body, calculate described operating body and be mapped to the relative position in virtual vision space.
Described virtual vision space is the visual imaging space of user while watching described displaying contents.
Head-wearing type intelligent equipment can gather space at it and set up space coordinates, thereby can determine the initial position of operating body in these space coordinates.
This initial position can represent with the coordinate of a pixel in the operating body image collecting, or multiple pixel coordinate represents.
Be shown example with initial position with a pixel coordinates table, initial position and relative position have following mapping relations:
Wherein, (m, n) represents collection position coordinate, and (M, N) represents relative position coordinates, from this formula, W=am+bn+1.
(a, b, c, d, e, f, g, h) is parameter.
When collection position is known, by solving (a, b, c, d, e, f, g, h), can learn relative position (M, N).
Wherein, solving of (a, b, c, d, e, f, g, h) can be carried out according to following formula:
(x
i, y
j) be illustrated in the coordinate of the arbitrary pixel in the image that gathers space arbitrary station acquisition, i=1,2 ... n, j=1,2 ... n, (X
i, Y
j) expression (x
i, y
j) be mapped to coordinate corresponding behind virtual vision space.
Suppose that λ is the vector [a b c d e f g h] solving.
Above formula can be reduced to A* λ=B.
Thereby can obtain λ=(A
ta)
-1a
tb
Obtain (X according to test in advance
i, Y
j) and (x
i, y
j) coordinate figure, can obtain λ in the hope of solution, thereby can obtain the relative position coordinates (M, N) of operating body according to initial position coordinate (m, n).
In the time that initial position is represented by multiple pixel coordinates, can be according to the above-mentioned relative position coordinates that solves respectively each pixel and be mapped to virtual vision space, thus relative position is represented by multiple relative position coordinates of multiple pixels.
302: according to eyes of user to the distance of described HMD the ratio with the distance to described virtual vision space, calculate the operating position in the displaying contents that described relative position is mapped to described HMD output.
In a kind of possibility implementation, suppose in space coordinates, taking human eye position as true origin.Suppose that HMD place plane is parallel with the imaging surface in virtual vision space.Be a to the distance of HMD with the ratio of the distance to virtual vision space according to eyes of user, suppose that the relative position coordinates calculating is (M, N), can calculate (M, N) relative position coordinates that is mapped to virtual vision space is (M/a, N/a).
303: determine that in described displaying contents, being positioned at first of described operating position place shows object.
In order to know that first in the displaying contents of operating body operation shows object, operating body need to be mapped in the viewing area of HMD at the relative position in virtual vision space, show object to obtain in the displaying contents of operating body operation first.
Certainly, wear-type electronic equipment also can pre-determine out the ratio of virtual content and the displaying contents in virtual vision space, thereby can obtain the operating position of described relative position correspondence in displaying contents according to this ratio.
Wherein, according to described action message, adjust described first and show that the morphological feature of object in described displaying contents can have multiple implementation, in a kind of possible implementation, as shown in Figure 4, the process flow diagram of a kind of another embodiment of displaying contents control method providing for the embodiment of the present application, the operation of step 101～step 103 can embodiment shown in Figure 1, the operation of step 102 specifically can be according to execution embodiment illustrated in fig. 3, and in the present embodiment, step 104 can comprise:
401: determine that described first shows the pending region of being blocked by described operating body in object.
According to determined proportionate relationship in the initial position of operating body and step 302, can by operating body proportionally relation etc. zoom in the viewing area of HMD, thereby can determine the pending region that the first demonstration object is blocked by described operating body.
402: in described displaying contents, transparent processing is carried out in described pending region, first after being adjusted shows object.
Now, in the viewed virtual content of user, pending region is transparent image, therefore can observe actual environment in this pending region, i.e. described operating body.
Carry out transparent processing by the pending region that described operating body is blocked, can realize displaying contents mutual of user and HMD output, contribute to improve the display effect of actual situation combination, bring the better user of user and experience.
As Fig. 4 a, show and be mapped to the demonstration schematic diagram that first in virtual vision space shows object.Fig. 4 b shows the demonstration schematic diagram in virtual vision space that is mapped to after the first demonstration object adjustment.Fig. 4 a and Fig. 4 b are from user perspective, viewed view.From Fig. 4 a and Fig. 4 b, operating body 500 is staff, and the action message of its execution is " crawl ".First of operating body operation shows that object 600 is water tumbler, thereby according to the operating body collecting and action message thereof, can determine the pending region of being blocked by described operating body in the first demonstration object, transparent processing is carried out in pending region, be first after the adjustment shown in Fig. 4 b and show object.From Fig. 4 b, after the first demonstration object is adjusted, realize " staff " in actual environment and accurately grabbed " water tumbler " in virtual content, mate action message, from user perspective, staff and water tumbler are realized accurate combination, have improved the display effect of actual situation combination, and having more than is that virtual content is added in actual environment, increase the mutual of user and displaying contents, improved user's experience.
Fig. 4 a and Fig. 4 b are exemplary illustration, can also realize Various Complex mutual by present techniques scheme.
Wherein, first shows that the determining of pending region of object can have multiple possible implementation, in a kind of possible implementation, can determine according to the corresponding relation of the action message setting in advance and operating body shield portions, therefore, can inquire about the shield portions of the default described operating body corresponding with described action message; Thereby determine that described first shows the pending region of being blocked by the shield portions of described operating body in object.For example, take action message as example as gesture information, while carrying out " crawl " action, shield portions is the people's hand portion including thumb; When carrying out " clicks " and operating, it is whole that shield portions is staff etc.
In addition, as the possible implementation of another kind, described according to described action message, adjust described first and show that the morphological feature of object in described displaying contents can be:
Obtain the first control strategy corresponding to described action message.
According to described the first control strategy, adjust the display shape of described the first demonstration object.
The corresponding relation that also can preset different action messages and different control strategies, obtains the first control strategy corresponding to described action message thereby can inquire about, and shows that object adjusts according to described the first control strategy to described first.
As Fig. 4 c and Fig. 4 d, Fig. 4 c shows the first demonstration object map to the demonstration schematic diagram in virtual vision space, and Fig. 4 d shows the second demonstration object map to the demonstration schematic diagram in virtual vision space.
Known in conjunction with Fig. 4 c and Fig. 4 d, operating body is user's both hands, and action message is " lifting " action.First shows that object 600 is unopened flower, by identifying described " lifting " action, the first demonstration object is adjusted, and first after being adjusted shows that object is the flower bursting forth.
Wherein, the first control strategy can also refer to that described the first object changes and meets the natural law according to the display shape of described action message.For example first shows that object is assumed to be " balloon of inflation ", when action message is the action of hand-held information " stamp ", according to the natural law, be somebody's turn to do " balloon of inflation " under the action of " stamp ", " break ",, there is metamorphosis and resemble in " thereby balloon changes state of rupture into by inflated condition ".
Seen from the above description, in the embodiment of the present application, the first demonstration object is not defined as static object.
As another possible implementation, the action message of described operating body characterizes described operating body and moves to end position from described initial position;
Described according to described action message, adjust described first and show that the morphological feature of object in described displaying contents can be:
Obtain primary importance and the second place of described operating body in described collection space, wherein, described primary importance and the described second place are that initial position moves to any two positions in the movement locus that end position forms described in described operating body;
Determine that described first shows three position of object in described displaying contents;
According to the variation relation of described primary importance and the described second place, determine the 4th position relevant to described the 3rd position in described displaying contents;
Adjust described first and show that three position of object from described displaying contents moves to described the 4th position.
Thereby the mobile realization that can follow operating body is moved in the position of having realized demonstration object.
In the embodiment of the present application, user wears in the situation of head-wearing type intelligent equipment, user can realize the control to displaying contents in the viewing area of HMD by operating body, user is without the physical button of touching head-wearing type intelligent equipment, can realize the control to displaying contents, simple to operate, increase the control mode to displaying contents, meet to a certain extent user's request.And increase the interactive of user and displaying contents, the adjustment of displaying contents has been considered to the operating position of operating body in viewing area, made accurately to determine the first demonstration object, realized virtual combination display effect more accurately.
For aforesaid each embodiment of the method, for simple description, therefore it is all expressed as to a series of combination of actions, but those skilled in the art should know, the application is not subject to the restriction of described sequence of movement, because according to the application, some step can adopt other orders or carry out simultaneously.Secondly, those skilled in the art also should know, the embodiment described in instructions all belongs to preferred embodiment, and related action and module might not be that the application is necessary.
The structural representation of the embodiment of a kind of displaying contents control device that Fig. 5 provides for the embodiment of the present application, this displaying contents control device is specifically applied in head-wearing type intelligent equipment, described head-wearing type intelligent equipment comprises HMD and image acquisition device, user realizes the control to the displaying contents in the viewing area of described HMD by operating body, wherein, this displaying contents control device can be integrated in the processor of head-wearing type intelligent equipment, a function that can realize as processor, or be connected with processor as independent device, described device can comprise:
Initial position determination module 501, for determining the initial position of described operating body by described image acquisition device.
The initial position of described operating body be image acquisition device gather space acquisition to this initial position be that operating body is arranged in the position that gathers space.
Position mapping block 502, for determining that the initial position of described operating body is mapped to the operating position of the viewing area of described HMD.
Operating body is mapped in the viewing area of HMD gathering the initial position in space, determines the operating position being mapped in viewing area, show object thereby can accurately determine in the displaying contents of operating body operation.
Object determination module 503, for determining that according to described operating position first of described displaying contents shows object.
Action acquisition module 504, for gathering the action message of described operating body by described image acquisition device.
The action message of operating body can comprise that operating body is in the action message of two dimensional surface or in three-dimensional action message, specifically can refer to operating body from initial position move to profile information, the deformation characteristics of operating body and the positional information of operating body of motion track information, operating body that end position forms at least one etc.
The action of the execution of operating body can be that user is while wearing head-wearing type intelligent equipment, according to the virtual content arriving in virtual vision space observation, the operation that control operation body is carried out the virtual objects in virtual content, for example click a certain virtual objects, now head-wearing type intelligent equipment can gather the action message of space acquisition to operating body.
Adjusting module 505, for according to described action message, adjusts described first and shows the morphological feature of object in described displaying contents.
Described morphological feature at least comprises the one in display position, display size and display shape.
The morphological feature of the first demonstration object is adjusted, make described first to show action message described in object matching, when being mapped in virtual vision space, from user perspective, first shows object and accurately combination of operating body, and mates described action message.
In the present embodiment, when user wears head-wearing type intelligent equipment, user can realize the control to displaying contents in HMD viewing area by operating body, demonstration object in displaying contents can be adjusted by displaying contents control device, user, without the physical button of touching head-wearing type intelligent equipment, can realize the control to displaying contents, simple to operate, increase the control mode to displaying contents, met to a certain extent user's request.And increase the interactive of user and displaying contents, the adjustment of displaying contents has been considered to the operating position of operating body in viewing area, made accurately to determine the first demonstration object, realized virtual combination display effect more accurately.
Wherein, position mapping block 502 determines that the initial position of described operating bodies is mapped to the operating position in the viewing area of described HMD, can there is multiple implementation, in a kind of possibility implementation, as shown in Figure 6, show the structural representation of a kind of another embodiment of displaying contents control device that the embodiment of the present application provides, this device can comprise initial position determination module 501, position mapping block 502, object determination module 503, action acquisition module 504 and adjusting module 505, each functions of modules can embodiment shown in Figure 5, be with difference embodiment illustrated in fig. 5, described position mapping block 502 can comprise:
The first map unit 601, for according to the initial position of described operating body, calculates described operating body and is mapped to the relative position in virtual vision space;
Wherein, described virtual vision space is the visual imaging space of user while watching described displaying contents.
The second map unit 602, for according to eyes of user to the distance of described HMD the ratio with the distance to described virtual vision space, calculate the operating position in the displaying contents that described relative position is mapped to described HMD output.
Wherein, according to described action message, adjust described first and show that the morphological feature of object in described displaying contents can have multiple implementation, in a kind of possible implementation, as shown in Figure 7, adjusting module 505 can comprise:
Region determining unit 701, for determining that described first shows the pending region that object is blocked by described operating body.
The first adjustment unit 702, at described displaying contents, carries out transparent processing by described pending region, and first after being adjusted shows object.
Wherein, wherein, first shows that the determining of pending region of object can have multiple possible implementation, in a kind of possible implementation, can determine according to the corresponding relation of the action message setting in advance and operating body shield portions, therefore, this region determining unit can comprise:
Query unit, for inquiring about the shield portions of the default described operating body corresponding with described action message;
Subelement is determined in region, for determining that described first shows the pending region that object is blocked by the shield portions of described operating body.
In addition,, in another kind of possibility implementation, the action message of described operating body characterizes described operating body and moves to end position from described initial position; As shown in Figure 8, described adjusting module 505 can comprise:
Position acquisition unit 801, for obtaining primary importance and the second place of described operating body in described collection space, described primary importance and the described second place are that initial position moves to any two positions in the movement locus that end position forms described in described operating body;
The first determining unit 802, for determining that described first shows three position of object at described displaying contents;
The second determining unit 803, for according to the variation relation of described primary importance and the described second place, determines the 4th position relevant to described the 3rd position in described displaying contents;
The second adjustment unit 804, shows that for adjusting described first object moves to described the 4th position from the 3rd position of described displaying contents.
Thereby the mobile realization that can follow operating body is moved in the position of having realized demonstration object.
In the embodiment of the present application, user, without the physical button of touching head-wearing type intelligent equipment, can realize the control to displaying contents, and operation is convenient, has increased the control mode to displaying contents, has met to a certain extent user's request.And increase the interactive of user and displaying contents, the adjustment of displaying contents has been considered to the operating position of operating body in viewing area, made accurately to determine the first demonstration object, realized virtual combination display effect more accurately.
In another possibility implementation, as shown in Figure 9, adjusting module 505 can comprise
Strategy acquiring unit 901, for obtaining the first control strategy corresponding to described action message;
The 3rd adjustment unit 902, for according to described the first control strategy, adjusts the display shape of described the first demonstration object.
The corresponding relation that also can preset different action messages and different control strategies, obtains the first control strategy corresponding to described action message thereby can inquire about, and shows that object adjusts according to described the first control strategy to described first.
This first control strategy can also refer to that described the first object changes and meets the natural law according to the display shape of described action message.
Adjusting module in Fig. 5 or Fig. 6 specifically can adopt the implementation of any adjusting module of Fig. 7～Fig. 9.
Displaying contents control device described in above-described embodiment, in actual applications, can be integrated in head-wearing type intelligent equipment.Dispose the electronic equipment of the embodiment of the present application displaying contents control device, operation is convenient, has increased the control mode to displaying contents, has met to a certain extent user's request.
Known by above description, those skilled in the art can be well understood to the mode that the application can add essential general hardware platform by software and realize.Therefore, as shown in figure 10, the embodiment of the present application also provides a kind of head-wearing type intelligent equipment, and this head-wearing type intelligent equipment can be worn at user's head by stationary installation.
This head-wearing type intelligent equipment at least comprises and connects storer 1001, processor 1002, HMD1003 and image acquisition device 1004.
This storer 1001 is stored batch processing instruction, and this storer can be high-speed RAM storer, may be also nonvolatile memory (non-volatile memory), for example at least one magnetic disk memory etc.
This processor 1002, the programmed instruction of storing for calling this storer 1001, carry out operation as follows:
By described HMD1003, in the viewing area of HMD1003 output display content;
By described image acquisition device 1004 acquisition operations bodies, determine the initial position of described operating body;
The initial position of determining described operating body is mapped to the operating position in the viewing area of described HMD1003, and determines that according to described operating position first in described displaying contents shows object;
Gather the action message of described operating body by described image acquisition device 1004;
According to described action message, to adjust described first and show the morphological feature of object in described displaying contents, described morphological feature at least comprises the one in display position, display size and display shape.
In this instructions, each embodiment adopts the mode of going forward one by one to describe, and what each embodiment stressed is and the difference of other embodiment, between each embodiment identical similar part mutually referring to.For the disclosed device of embodiment, because it corresponds to the method disclosed in Example, so description is fairly simple, relevant part illustrates referring to method part.
Finally, also it should be noted that, in this article, relational terms such as the first and second grades is only used for an entity or operation to separate with another entity or operational zone, and not necessarily requires or imply and between these entities or operation, have the relation of any this reality or sequentially.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thereby the process, method, article or the equipment that make to comprise a series of key elements not only comprise those key elements, but also comprise other key elements of clearly not listing, or be also included as the intrinsic key element of this process, method, article or equipment.The in the situation that of more restrictions not, the key element being limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment that comprises described key element and also have other identical element.
To the above-mentioned explanation of the disclosed embodiments, make professional and technical personnel in the field can realize or use the application.To be apparent for those skilled in the art to the multiple amendment of these embodiment, General Principle as defined herein can, in the situation that not departing from the application's protection domain, realize in other embodiments.Therefore, the application will can not be restricted to these embodiment shown in this article, but will meet the widest scope consistent with principle disclosed herein and features of novelty.
Claims (12)
1. a displaying contents control method, it is characterized in that, be applied in head-wearing type intelligent equipment, described head-wearing type intelligent equipment comprises head mounted display HMD and image acquisition device, user realizes the control to the displaying contents in the viewing area of described HMD by operating body, and described method comprises:
Determine the initial position of described operating body by described image acquisition device;
The initial position of determining described operating body is mapped to the operating position in the viewing area of described HMD, and determines that according to described operating position first in described displaying contents shows object;
Gather the action message of described operating body by described image acquisition device;
According to described action message, to adjust described first and show the morphological feature of object in described displaying contents, described morphological feature at least comprises the one in display position, display size and display shape.
2. method according to claim 1, is characterized in that, describedly determines that the initial position of described operating body is mapped to the operating position in the viewing area of described HMD, and determines that according to described operating position first in described displaying contents shows that object comprises:
According to the initial position of described operating body, calculate described operating body and be mapped to the relative position in virtual vision space; Described virtual vision space is the visual imaging space of user while watching described displaying contents;
According to eyes of user to the distance of described HMD the ratio with the distance to described virtual vision space, calculate the operating position in the displaying contents that described relative position is mapped to described HMD output;
Determine that in described displaying contents, being positioned at first of described operating position place shows object.
3. method according to claim 1 and 2, is characterized in that, described according to described action message, adjusts described first and shows that the morphological feature of object in described displaying contents comprises:
Determine that described first shows the pending region of being blocked by described operating body in object;
In described displaying contents, transparent processing is carried out in described pending region, first after being adjusted shows object.
4. method according to claim 3, is characterized in that, described definite described first shows that the pending region of being blocked by described operating body in object comprises:
The shield portions of the described operating body corresponding with described action message that inquiry is preset;
Determine that described first shows the pending region of being blocked by the shield portions of described operating body in object.
5. method according to claim 1 and 2, is characterized in that, the action message of described operating body characterizes described operating body and moves to end position from described initial position;
Described according to described action message, adjust described first and show that the morphological feature of object in described displaying contents comprises:
Obtain primary importance and the second place of described operating body, described primary importance and the described second place are that initial position moves to any two positions in the movement locus that end position forms described in described operating body;
Determine that described first shows three position of object in described displaying contents;
According to the variation relation of described primary importance and the described second place, determine the 4th position relevant to described the 3rd position in described displaying contents;
Adjust described first and show that three position of object from described displaying contents moves to described the 4th position.
6. method according to claim 1 and 2, is characterized in that, described according to described action message, adjusts described first and shows that the morphological feature of object in described displaying contents comprises:
Obtain the first control strategy corresponding to described action message;
According to described the first control strategy, adjust the display shape of described the first demonstration object.
7. a displaying contents control device, it is characterized in that, be applied in head-wearing type intelligent equipment, described head-wearing type intelligent equipment comprises head mounted display HMD and image acquisition device, user realizes the control to the displaying contents in the viewing area of described HMD by operating body, and described device comprises:
Initial position determination module, for determining the initial position of described operating body by described image acquisition device;
Position mapping block, for determining that the initial position of described operating body is mapped to the operating position of the viewing area of described HMD;
Object determination module, for determining that according to described operating position first of described displaying contents shows object;
Action acquisition module, for gathering the action message of described operating body by described image acquisition device;
Adjusting module, for according to described action message, adjusts described first and shows the morphological feature of object in described displaying contents, and described morphological feature at least comprises the one in display position, display size and display shape.
8. device according to claim 7, is characterized in that, described position mapping block comprises:
The first map unit, for according to the initial position of described operating body, calculates described operating body and is mapped to the relative position in virtual vision space; Described virtual vision space is the visual imaging space of user while watching described displaying contents;
The second map unit, for according to eyes of user to the distance of described HMD the ratio with the distance to described virtual vision space, calculate the operating position in the displaying contents that described relative position is mapped to described HMD output.
9. according to the device described in claim 7 or 8, it is characterized in that, described adjusting module comprises:
Region determining unit, for determining that described first shows the pending region that object is blocked by described operating body;
The first adjustment unit, at described displaying contents, carries out transparent processing by described pending region, and first after being adjusted shows object.
10. device according to claim 9, is characterized in that, described region determining unit comprises:
Query unit, for inquiring about the shield portions of the default described operating body corresponding with described action message;
Subelement is determined in region, for determining that described first shows the pending region that object is blocked by the shield portions of described operating body.
11. according to the device described in claim 7 or 8, it is characterized in that, the action message of described operating body characterizes described operating body and moves to end position from described initial position;
Described adjusting module comprises:
Position acquisition unit, for obtaining primary importance and the second place of described operating body in described collection space, described primary importance and the described second place are that initial position moves to any two positions in the movement locus that end position forms described in described operating body;
The first determining unit, for determining that described first shows three position of object at described displaying contents;
The second determining unit, for according to the variation relation of described primary importance and the described second place, determines the 4th position relevant to described the 3rd position in described displaying contents;
The second adjustment unit, shows that for adjusting described first object moves to described the 4th position from the 3rd position of described displaying contents.
12. according to the device described in claim 7 or 8, it is characterized in that, described adjusting module comprises:
Strategy acquiring unit, for obtaining the first control strategy corresponding to described action message;
The 3rd adjustment unit, for according to described the first control strategy, adjusts the display shape of described the first demonstration object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410239854.5A CN103984101B (en) | 2014-05-30 | 2014-05-30 | Display contents controlling method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410239854.5A CN103984101B (en) | 2014-05-30 | 2014-05-30 | Display contents controlling method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103984101A true CN103984101A (en) | 2014-08-13 |
| CN103984101B CN103984101B (en) | 2016-08-24 |
Family
ID=51276136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410239854.5A Active CN103984101B (en) | 2014-05-30 | 2014-05-30 | Display contents controlling method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103984101B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104598036A (en) * | 2015-03-02 | 2015-05-06 | 联想(北京)有限公司 | Information processing method and device |
| CN105759422A (en) * | 2015-01-06 | 2016-07-13 | 精工爱普生株式会社 | Display System And Control Method For Display Device |
| WO2018103072A1 (en) * | 2016-12-09 | 2018-06-14 | 深圳市柔宇科技有限公司 | Adjustment method and adjustment system for user interface, and head-mounted display device |
| CN110837764A (en) * | 2018-08-17 | 2020-02-25 | 广东虚拟现实科技有限公司 | Image processing method and device, electronic equipment and visual interaction system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6346929B1 (en) * | 1994-04-22 | 2002-02-12 | Canon Kabushiki Kaisha | Display apparatus which detects an observer body part motion in correspondence to a displayed element used to input operation instructions to start a process |
| US20090243968A1 (en) * | 2008-03-31 | 2009-10-01 | Brother Kogyo Kabushiki Kaisha | Head mount display and head mount display system |
| CN101750738A (en) * | 2008-12-19 | 2010-06-23 | 索尼株式会社 | Head mounted display |
| CN102346544A (en) * | 2010-07-30 | 2012-02-08 | 鸿富锦精密工业（深圳）有限公司 | Head-worn display system with interactive function and display method thereof |
| CN103534665A (en) * | 2011-04-04 | 2014-01-22 | 英特尔公司 | Keyboard avatar for heads up display (hud) |
| CN103698884A (en) * | 2013-12-12 | 2014-04-02 | 京东方科技集团股份有限公司 | Opening type head-mounted display device and display method thereof |
-
2014
- 2014-05-30 CN CN201410239854.5A patent/CN103984101B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6346929B1 (en) * | 1994-04-22 | 2002-02-12 | Canon Kabushiki Kaisha | Display apparatus which detects an observer body part motion in correspondence to a displayed element used to input operation instructions to start a process |
| US20090243968A1 (en) * | 2008-03-31 | 2009-10-01 | Brother Kogyo Kabushiki Kaisha | Head mount display and head mount display system |
| CN101750738A (en) * | 2008-12-19 | 2010-06-23 | 索尼株式会社 | Head mounted display |
| CN102346544A (en) * | 2010-07-30 | 2012-02-08 | 鸿富锦精密工业（深圳）有限公司 | Head-worn display system with interactive function and display method thereof |
| CN103534665A (en) * | 2011-04-04 | 2014-01-22 | 英特尔公司 | Keyboard avatar for heads up display (hud) |
| CN103698884A (en) * | 2013-12-12 | 2014-04-02 | 京东方科技集团股份有限公司 | Opening type head-mounted display device and display method thereof |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105759422A (en) * | 2015-01-06 | 2016-07-13 | 精工爱普生株式会社 | Display System And Control Method For Display Device |
| CN104598036A (en) * | 2015-03-02 | 2015-05-06 | 联想(北京)有限公司 | Information processing method and device |
| CN104598036B (en) * | 2015-03-02 | 2019-03-08 | 联想(北京)有限公司 | A kind of information processing method and device |
| WO2018103072A1 (en) * | 2016-12-09 | 2018-06-14 | 深圳市柔宇科技有限公司 | Adjustment method and adjustment system for user interface, and head-mounted display device |
| CN110837764A (en) * | 2018-08-17 | 2020-02-25 | 广东虚拟现实科技有限公司 | Image processing method and device, electronic equipment and visual interaction system |
| CN110837764B (en) * | 2018-08-17 | 2022-11-15 | 广东虚拟现实科技有限公司 | Image processing method and device, electronic equipment and visual interaction system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103984101B (en) | 2016-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6393367B2 (en) | Tracking display system, tracking display program, tracking display method, wearable device using them, tracking display program for wearable device, and operation method of wearable device | |
| JP6177872B2 (en) | I / O device, I / O program, and I / O method | |
| JP6333801B2 (en) | Display control device, display control program, and display control method | |
| WO2013035758A1 (en) | Information display system, information display method, and storage medium | |
| CN116027894A (en) | Passive optical and inertial tracking for slim form factors | |
| US8847850B1 (en) | Head mounted display device for displaying augmented reality image capture guide and control method for the same | |
| JP6250024B2 (en) | Calibration apparatus, calibration program, and calibration method | |
| JP2017027206A (en) | Information processing apparatus, virtual object operation method, computer program, and storage medium | |
| CN103984101A (en) | Display content control method and device | |
| CN104598037A (en) | Information processing method and device | |
| US10171800B2 (en) | Input/output device, input/output program, and input/output method that provide visual recognition of object to add a sense of distance | |
| JP6446465B2 (en) | I / O device, I / O program, and I / O method | |
| JP2005339266A (en) | Information processing method, information processor and imaging device | |
| JP2005339267A (en) | Information processing method, information processor and imaging device | |
| CN117043722A (en) | Apparatus, method and graphical user interface for map | |
| JP6479835B2 (en) | I / O device, I / O program, and I / O method | |
| JP2017111537A (en) | Head-mounted display and program for head-mounted display | |
| CN106295570A (en) | Block filtration system and method alternately | |
| JP2017111724A (en) | Head-mounted display for piping | |
| WO2016051430A1 (en) | Input/output device, input/output program, and input/output method | |
| JP2017111721A (en) | Head-mounted display for clean room, control method of head-mounted display for clean room, and control program for head-mounted display for clean room |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |