WO2013104347A1

WO2013104347A1 - Method and device for the recording and reproduction of panoramic representations

Info

Publication number: WO2013104347A1
Application number: PCT/DE2012/000015
Authority: WO
Inventors: Sascha WEISSBACH
Original assignee: Kanna, Michael
Priority date: 2012-01-11
Filing date: 2012-01-11
Publication date: 2013-07-18
Also published as: DE112012005632A5

Abstract

The invention relates to a method and a device - which serves for carrying out the method - for the recording and reproduction of panoramic images, in particular for the pictorial reproduction of premises and landscapes. The problem addressed by the invention is that of providing a method and a device serving for carrying out the method with which less complex recording of panoramic images is made possible, transmission to the client can take place almost in real time and reproduction independently of the operating systems and computer structures of the client is made possible. This problem is solved by virtue of the fact that the rays which are captured by the camera systems (2, 3, 6) and reflected by the mirror-optical assemblies (1, 7) are converted into image signals and projected onto virtual surfaces (5), the forms of which correspond to inverted surfaces of the mirror-optical assemblies, the excerpt to be viewed from the 360° panoramic image is selected by means of a navigable virtual camera positioned in the interior of the projected virtual hyperboloid model and the signals of the images (4) captured by the virtual camera are rendered and, independently of the respective operating system and computer architecture present, are stored in a retrievable manner and/or transmitted live.

Description

Method and device for recording and reproducing panorama displays

The invention relates to a method and an apparatus for carrying out the method for recording and reproducing panoramic images, in particular for the visual reproduction of premises and landscapes. The application is used, for example, for plant and property monitoring, virtual tour of venues, resort areas, videoconferencing and e-learning. Spherical arrangements of cameras are known for the production of 360 ° video recordings in which the recordings are synchronized in a base unit and combined to form a video signal. By means of a software created for this signal is then played back on a monitor. Another known solution consists of a camera and an integrated mirror optics for a 360 ° recording. Disadvantages of these systems, however, are the complex technology and required software for processing the signals. Also known is a system according to US 2002/0126395 Al in which mirror optics are used, which aim at each other by an arrangement of multiple reflectors as realistic as possible reproduction of the panoramic images. Another disadvantage of this solution is the high level of software required for the equalization of the captured images, the lack of real-time operation in the reproduction of the panorama images and the dependence on the existing operating systems of the computer systems of the clients. The object of the invention is therefore to provide a method and a device for performing the method, with which a less complex recording of panoramic images is made possible, the transmission to the client can be done in near real time and playback regardless of the operating systems and computer structures of the Clients is enabled. This object is achieved by using the method with the features of claim 1. Advantageously, the method for a higher accuracy in the real playback is designed as described with the features of claim 2. The equalization and the subsequent display on a monitor is carried out exclusively by a software used by the client. The representation of the real scene on the monitor takes place in near real time. An application of the method according to the invention can be carried out platform-independent within the Internet. The cost of required special processing software or hardware is significantly reduced. Since the panoramic images are taken by hyperbolic-shaped mirrors and are recorded on their surfaces by surfaces of different sizes, different areas of sharpness are created. The position of the focus areas depends on the arrangement of the mirror relative to the camera system. In order to compensate for the different focus ranges, by means of a device in a preferred embodiment according to claim 3, two hyperbolic-shaped mirrors in different positions, one up and one down, each associated with a sensor system of the same camera. The

CONFIRMATION COPY generated image data are projected into a respective virtual projection body, which corresponds to the inverted area of the associated mirror. From these projected virtual projection surfaces, a resulting projection surface is then generated by predetermining the pixels to be compensated for by corresponding pixels captured in the larger focus range. With this device, a compensation of the blurred area is achieved, thus increasing the accuracy. Advantageously, the transmission of the captured image signals to the client via optical fibers proves. Taking advantage of the possible high transmission rates and the large ranges of several 100 km, without a required amplification technology, very accurate images can be transmitted in real time, especially when used in local networks. Furthermore, the transmission causes a reduced effort and a high data security. Below, the method and the device will be explained in more detail using an exemplary embodiment. In the drawing shows

1: the arrangement of the hyperbolic mirror optics,

2 shows the schematic beam path at the mirror optics,

3 shows the schematic representation of the reflection radiation and

4: the virtual projection surface.

On the camera lens 2 shown in Figure 1, a hyperbolic mirror is mounted, which has a predetermined distance from the camera lens 2. By specifying the specific distance for the respective camera system, the rays detected by the panorama are cut after reflection on the mirror in a predetermined focal point F. As the schematic beam path shown in FIG. 2 on the mirror-optical module 1 shows, its focal point F ^' is selected such that the beams impinge on the surface of the camera sensor 3. The relatively "pointed" design of the mirror allows in the vertical direction an angle of view of 1 15 °. The illustration of Figure 3 shows the schematic representation of the reflection radiation. The reflected beams generate on the camera sensor 3 a "donatfÖrmiges" circular image, which still needs to be equalized for a realistic reproduction. According to the invention, the equalizations are carried out by means of the following optical measures. The "donatförmige" camera image is projected onto a surface whose shape corresponds to the inverted surface of the mirror. A schematic representation of the generated virtual projection surface 5 is shown in FIG. The projected area corresponds to the inner surface of the mirror itself, so to speak. Since the real panorama pictures are taken over a mirror, these pictures are mirror-inverted. Before the projection will be preferably corrected. Through this procedure, a panorama can also be displayed without computationally intensive image transformations. For the creation of the projection model known 3D modeling programs are used. The image or video is assigned as surface material to the model body (hyperboloids). The example embodiment of the projection body corresponds to a bivalve hyperboloids. Inside the hyperboloid model, a virtual camera is positioned instead of the viewer. The location of the virtual camera can also be selected for an observer viewing the panoramic images when using the method in a visitor's room. For viewing the panorama, the virtual camera can be navigated. If the scene of the camera view is rendered, you get an equalized image section of the panorama. By means of "optical" image equalization, the representation of the real scene on a monitor in almost real time is possible. FIGS. 5 and 6 show two possibilities for positional positioning of the hyperbolic mirror. The illustration of Figure 5 shows the downward position of the mirror, in which the tip is directed downward and through which the lower portion of the captured panorama blurred image reproduction and the upper area allows a sharp image reproduction. These different areas of sharpness are created by the fact that at the lower end of the hyperbolic mirror, the panorama is displayed on a much smaller area than on the upper areas. If the reflected beams hit the camera sensor 3, more image information per area must be accommodated in the middle of the sensor al at the edge of the sensor. As a result, several pixels are combined in the center of the image, as a result of which detail image information is lost. When projecting the polar coordinate image onto the hyperboloid inner surface, the pixels are enlarged, but the lost image information can not be restored. This results in large areas with the same color information, and thus the image is blurred at these locations. Figure 6 shows the reverse arrangement of the mirror with its tip directed upwards. This gives the bottom of a sharp reproduction of the image information, which are increasingly blurred towards the top. In order to equalize the level of sharpness in the areas and thereby increase the accuracy of the method or adapt to the requirements, the apparatus for performing the method can be designed so that it has two mirror-optical assemblies 1, 7. Each of these mirror-optical assemblies 1, 7 is assigned to a separate sensor of the camera system 6. The image signals of each sensor are projected into a virtual projection body and a resulting projection surface 5 is generated from the two projection bodies. The selection of the image signals to be used for the resulting projection surface 5 takes place on the basis of predetermined coordinates.

The application of the method and the device for carrying out the method can be carried out for the monitoring of plants and land. In this application will be respectively different parts of the panorama are divided into several monitors, although only one video signal is transmitted. An option is to track specific people or objects. Another application is the presentation of venues, resorts and events through virtual tours and live broadcasts. In this case, additional information could be called up at defined points on the panorama. Furthermore, the method according to the invention can be used within video conferencing systems. The required number of video streams would not have to correspond to the number of participants, but only one stream per group would be required. In the process, certain image sections would be assigned to the corresponding person. It would also be possible to implement a guided conference in which the displayed image sections on the clients are controlled from a central location. Furthermore, there is an application in the field of e-learning. In the case of a live transmission of a lecture, with additional equipment of the speaker with a tracking system, its coordinates could be transmitted to all clients in order to be able to display the image section on which it can be seen. The speaker could, as usual, move freely and would not have to take a fixed camera consideration.

The process created, in contrast to the prior art, involves a purely technical process. The generation of the second (virtual) image generated on the basis of the recorded panoramic image takes place by the projection of the first image consisting of image signals after their transfer to the area virtually created by software. The image signals are assigned by application of the software known per se to the projection points on the virtual surface (surface), which correspond to the inverted points on the physical surface of the mirror-optical assemblies. This software also includes the function of a virtual navigable camera whose position the observer (client) occupies. With the solution created, a "shift", ie no transformation, of the optical image of the real world into a virtual world is made. As a result, for example, the representation of the real image on a monitor in almost real time. The optical image of the real world is imported after the transmission of the recorded image signals, so to speak as a "texture" on the programmatically generated surface of the client. The recorded "donat-shaped" camera image and its mirrored inversion serves as an optical image of the real world. Due to the true to original transmission of the images, no image transformation by means of mathematical methods and the required expenditure of computer technology is required for the reproduction or projection. The projection surface at the client is virtually "rebuilt" by software known per se according to the present method, and the transmitted image signals representative of the image of the real world are imported onto this surface as "texture". Accordingly, a generation of a second image by the virtual camera takes place. The required amount of special processing software and hardware is thereby considerably reduced. Reference number mirror-optical assembly

camera lens

camera sensor

Donat image

projection

camera system

mirror-optical assembly

Claims

claims

1. A method for recording and reproducing panoramic images by detecting the optical radiation coming from the panoramic objects by means of hyperbolic mirror-optical assemblies and by reflection of the optical beams to sensor surfaces of associated camera systems and a virtual projection of the captured image by means of software-based room models and virtually navigable cameras, characterized marked that

a) reflected by the mirror optical assemblies (1, 7) and of the

Camera systems detected rays converted into electrical and / or optical image signals that correspond to the shape of Donatbildem, and

b) then these image signals to the client by electrical and / or optical means

and projected onto software-generated virtual surfaces whose virtual shape corresponds to the inverted surfaces of the mirror-optical assemblies (1; 7).

2, Method for recording and reproducing panoramic images according to claim 1, characterized in that

a) by means of at least two of the panoramic areas to be recorded

hyperbolic mirror optical assemblies (1; 7) of the

Panoramic surfaces transmitted optical beams are reflected to a respective sensor surface of the associated camera system (6),

b) that the beams detected by the sensor surfaces are projected onto a respective virtual surface associated with the sensor surface, which corresponds to the shape of the inverted surface of the mirror-optical assembly, and c) from the projection surfaces a resulting projection surface (5) by assignment of signal values of the projection surfaces is determined under specification of certain projection points.

3. The method according to claim 1, characterized in that the image signals converted into electrical and / or optical signals are transmitted to the client by means of optical waveguides (LWL).

4. Device for carrying out the method according to claim 1, characterized in that the camera system (6) has at least two sensor surfaces, each of which a mirror-optical assembly (1, 7) is assigned and these assemblies are mutually adjustable with the camera system (6) ,

For this 3 sheets of drawings.