WO1999055122A1

WO1999055122A1 - 3d ready lamp

Info

Publication number: WO1999055122A1
Application number: PCT/CA1999/000317
Authority: WO
Inventors: Will N. Bauer
Original assignee: Bauer Will N
Priority date: 1998-04-16
Filing date: 1999-04-13
Publication date: 1999-10-28
Also published as: AU3401599A; CA2234486A1

Abstract

A 3D ready lighting system is controlled through a communication protocol such as DMX-512 or TCP/AI. Data concerning various lamps (12) of the lighting system parameters such as ranging and orientation may be gathered independently in real time and used to modify control of the lamps. For this purpose, certain channels (14A) of the communication protocol (18) are used for bi-directional communication. Data retrieved via the communication protocol concerning the parameters of interest is discarded and replaced by data gathered independently.

Description

- 1 - 3D READY LAMP

TECHNICAL FIELD

This invention relates to the provision of lighting systems equipped and ready to respond to a flow of 3D control information (3D-ready intelligent lamps) .

BACKGROUND ART

Robotic lamps are used primarily in the entertainment industry such as night clubs, theatres and concert venues. They also have additional application in areas such as promotion/advertising, architectural lighting, and so- called "immersive reality" among others. These robotic or "intelligent" lamps can be remotely controlled by an industry communications standard called "DMX-512". This is a high speed serial data protocol which allows remote computer control of many different features of the lamp fixture (s) such as the pan and tilt angle at which the light beam is projected, beam intensity, colour selection, beam width (iris), focus, and light pattern ("gobo" selection) among others.

Increasingly these lamps are used in conjunction with computer software running on external PC computers and/or lighting desks to enhance their capabilities. Some of this external software simulates the three dimensional (3D) environment in which the lamps function, allowing programming of lighting effects to occur "off-line" (i.e. without the need for a theatre and lighting system) . This is possible since the software provides a 3D virtual environment visualizing how the lamps will look when they are used in real life. Other software/hardware systems such as 3D tracking systems which follow the movements of a performer and allow lamps to automatically track and respond to performers' movements in various ways are described and claimed in U.S. Patent No. 5,214,615 issued May 25, 1993, to Bauer and 5,412,619 issued May 2, 1995, to Bauer and in PCT application No. PCT/CA97/00724.

As interest in using intelligent lamps to respond to 3D cues and movements grows there is an increasing need for the lamp itself to be "3D-ready" . A 3D-ready lamp is specially equipped and is able to respond in an optimal fashion to streams of 3D control information.

An intelligent lamp can respond to 3D information in a variety of ways. For many of the responses, it is necessary to know the 3D position of the lamp and its orientation. Coordinates for the X, Y and Z axes of three dimensional space plus pitch, yaw, and roll angles for the orientation in space give a six degree of freedom ("6DOF") description of the status of the lamp. This information establishes a coordinate system which completely describes the lamp and is necessary to calculate the way in which the lamp responds to incoming 3D information. For example, for a lamp to follow an object moving in three dimensions, one must know the 3D coordinates of the object and the 6D0F coordinates of the lamp in order to correctly calculate the pan/tilt angles necessary to point the lamp at the moving object .

Clearly then, establishing 6D0F coordinates of the - 3 - lamp is something that a 3D ready light should be able to do with ease. Currently with normal intelligent lighting fixtures, it is possible to calculate the position of the lamp and orientation by pointing the lamp at four reference points located in a common plane (usually the stage floor) and measuring the pan/tilt angles required to point the lamp at each of the four points . This gives enough information that one skilled in the art can calculate the lamp's position and orientation. A problem with this process is that it is quite time consuming to point each lamp to each of the four points. Often there are many lamps for which this must be done and it can take an inordinate amount of time. A better solution would be to point the lamp at only one reference point (say the centre of the stage) and have that be enough to establish the lamp's 6DOF coordinates relative to that point. To do this, the following capabilities of lamp are necessary:

a) a method of sensing two of the lamp's three orientation angles (pitch and roll) ; b) a method of measuring the distance from the lamp to the reference point; and c) a method of sensing the rotation angle (yaw) of the lamp.

Requirement a) can be accomplished in a number of ways but perhaps the easiest is to have a two-axis gravity sensor which can measure the pitch and roll angles of the lamp's orientation relative to the earth's gravity field. A variety of sensors are available which do this and which are easily interfaced to an inexpensive microcontroller - 4 - chip. Some are two axis accelerometers which consist of two DC frequency response accelerometers mounted orthogonally to each other. Others are capacitively based and sense the orientation/position of liquid in a tube much like a traditional carpentry level.

Requirement b) can be met with a ranging system that can sense the distance between the lamp and the reference point. There are also a variety of technologies capable of this. For example, laser ranging systems which use light beams that are either pulsed or continuously present. When pulsed, the pulse is emitted from a laser transmitter and the time delay between its emission and the sensing of its reflection by receiver circuitry is measured. This can yield accuracies anywhere from one or two metres to centimetres depending on the accuracy of the timing and the number of measurements averaged. When used continuously, a phase measurement is performed comparing the phase of the transmitted wave with the phase of the reflected wave. This method allows greater accuracy but usually fewer measurements per second are possible. Another technology is the use of ultrasonic waves. The time delay between the emission and reception of ultrasonic pulses can be measured and this varies linearly with distance.

Requirement c) can be met with an appropriate rotational sensing device such as a gyro, compass or the like. Alternatively, it could be calculated by pointing the light at two points of known (X, Y, Z) position coordinates rather than one. An additional desirable, but not absolutely necessary, characteristic of a 3D-ready lamp is the ability to be superior at responding to streams of 3D positional data. A problem that normal intelligent lamps have is that their control systems are open-loop which is to say that there is no feedback regarding the state of the lamp vis-a-vis the state that it was commanded to assume. Particularly with pan and tilt motor control, this can be problematic since open-loop control greatly restricts the speed and precision at which these motors can operate.

Closed-loop control of pan and tilt is much more desirable for 3D positioning tasks such as follow-spot operation since it allows far greater speed, precision, and smoothness of movement. To this end, another desirable (but not strictly necessary) requirement for a 3D ready lamp is that its pan/tilt motors be equipped with shaft encoders which yield digital outputs of the actual pan/tilt angles at which the light beam is being pointed. This information should have a precision of at least 12 bits per revolution and should be in a form where either the internal lamp microcontroller or an external controller (or both) can access this data.

Thus it would be desirable to provide a "3D-ready intelligent lamp" equipped with a microcontroller or DSP controller which integrates a pitch/roll/yaw angle measurement, pan/tilt encoder angle measurements, and a ranging measurement. This information would be utilized to control lighting parameters in real-time. In its

"calibration mode" this lamp would be capable of detecting its 6DOF position/orientation on being pointed at one reference point. In its normal "operational mode" it is capable of extremely fast and precise pan/tilt positioning using the pan/tilt encoder closed-loop feedback. Further, it is capable of responding to 3D positioning information sent to it, i.e. instead of responding to just pan and tilt angle positioning commands sent to it by an external controller, it is capable of being sent a 3D (x, y, z) coordinate and, based on its own position/orientation coordinates, calculating the pan/tilt angles necessary to point at this position. Additionally, it is capable of continuously measuring the distance between the lamp and the surface onto which the light is shining and modulating any of its lighting parameters in response to that information. This 3D-ready functionality can be implemented either as a standalone package that can be retrofitted to existing lamps or it can be built in as an integral part of a new design of lighting fixture.

As an example of the utility of such a lamp, consider the problem of focussing/irising the lamp depending on where it is pointing. This is a normal problem for intelligent lamps: as the light beam is moved via its pan/tilt controls of the lamp, the width, focus, and intensity of the beam change depending on how far away the lamp is from the wall floor or other surface onto which the beam is being projected. Unless an adjustment is made, the width of the beam will be twice as wide when the projection surface is at twice the distance from the lamp. To make the beam width be the same, the lamp's iris or focal zoom control must be adjusted. Clearly, if there were some way of measuring the distance from the lamp to the projection surface, it would be possible to modulate the lamp's controls to maintain a constant width. In a similar manner, knowledge of the distance between the lamp and the projection surface would allow the light beam to be kept properly focussed continuously despite variations in this distance caused by changing pan/tilt angles of the lamp (or, for that matter, moving projection surfaces such as mobile scenery) in real-time during a show.

Other possibilities include (but are not limited to) the following:

• being able to tell the lamp to point to a specific 3D location with focus and beam width (controlled by iris and zoom) setting determined automatically based on parameters indicating the desired shape/pattern of the light beam rather than just sending pan/tilt coordinates .

• being able to have exceptionally fast, smooth, and accurate pan/tilt positioning for pointing the lamp during follow-spot operation.

• being able to calculate predicted values of pan/tilt positioning algorithms such as a Kalman filter which allow the light's pan/tilt positioning to look ahead or "lead" a moving 3D point based on knowledge of its previous recent history of movements. This would allow even better following of moving objects, particularly those being tracked by a 3D tracking system whose coordinates were being fed to the light in real time.

• being able to tell the lamp to automatically change the focus and iris setting in real time (i.e. as the lamp is moved about) bases on the 3D position where the lamp is pointing so that the light is always in focus and maintains a constant beam width.

• being able to send to the lamp either 3D coordinates (X,Y,Z) of where to point, or normal pan/tilt coordinates with an added range parameter.

• being able to feed back real time information such as the current distance to where the lamp is pointing or the 3D coordinates of where it is pointing plus calibration information such as its 6DOF coordinates to a master controller such as a lighting console.

• being able to tell the lamp to move in specific geometric patterns . since the lamp would have knowledge of the geometric distortions involved based on its 6DOF coordinates and the desired centre of the patterns, it could correct for these distortions and generate (for example) true circular movements. This functionality could also be implemented on a lighting console that had been sent the lamp's 6D0F coordinates .

• being capable of generating a 3D map of a particular theatre stage. The light would be scanned over the - 9 - entire stage area, transmitting the 3D coordinates of where the lamp was pointing with a precision appropriate to the situation. This 3D map could then be stored by a lighting console and used to coordinate or match up a previously generated 3D model of the stage (which can currently be generated using a variety of lighting design software packages) with the actual one .

DISCLOSURE THE INVENTION

Accordingly the invention provides a lighting system equipped to respond to a flow of three dimensional control information comprising at least one variably responsive intelligent lamp controlled by a computer communication means to provide input data for the lamp characterized in that the system includes feed back means from the lamp to the computer to provide data to the computer in real-time as to the six degrees of freedom of the lamp, the actual pan/tilt coordinates of the lamp and to provide ranging means between the lamp and a surface on which it is projected. For example, the communication means may be a serial data protocol such as a TCP/IP or can use some other serial protocol such as a DMX-512 data source connected in a loop to provide feedback.

DMX-512 is not normally a bidirectional protocol; normally one may send data from one source to many destinations but one cannot send from the many destinations to the one source. To provide the possibility for 3D lights to provide feedback about their 6DOF coordinates or about the distance between the lamp and the surface onto - 10 - which it was shining, it is necessary to reallocate some of the lamp's channels (for use in calibration mode) and add additional DMX channels for distance data, pan data and tilt data respectively. Normally, lamps are set to receive data from a contiguous range of DMX channels, the lowest numbered of which is called the "base channel" . Thus a lamp might be set to receive data covering DMX channels 98 to 111 i.e. a base channel of 98. It is also normal practice that all other lamps would be set to avoid this range of channels (unless it was somehow advantageous to send the same data to several lamps at the same time) so this range is normally reserved for one lamp. A sub-range of these lighting channels may be reserved for the transmission of data specific to the lamp. Thus if, for example, if channels 98 to 101 inclusive are reserved, the lamp will receive data on DMX channels 1 to 512 inclusive but it will not retransmit data that was incoming on channels 98 - 101 but rather will replace that data with its own internally generated values. In this way, by connecting the DMX serial connection in a loop one can transmit data from a controller such as a lighting console and receive back the data generated by each lamp. This may be possible using a normal serial data UART chip or it may necessitate a special DMX control FPGA chip programmed to replace the specific channels of data but such things are well within the scope of one versed in the art.

BRIEF DESCRIPTION OF THE DRAWINGS

A preferred embodiment will now be described by way of example with reference to the drawings in which: - 11 -

Figures 1A and IB depict the problem of maintaining constant beam width on a moving projection surface;

Figure 2 is a block diagram of one embodiment of the invention; and

Figure 3 is a block diagram of the internal functioning of the 3D light controller of Figure 2.

MODES OF CARRYING OUT THE INVENTION

One problem is easily understood from Figures 1A and IB. Figure 1A shows a focussed light source 10 projected on a projection surface 11 which is located a distance R from the light source. The image on the projection surface 11 covers an area of diameter D. Figure IB, on the other hand, shows the projection surface 11 having moved to a distance 2R from the light source 10. The diameter of the image has correspondingly grown to 2D although it would be desirable for it to remain at the original size.

Figure 2 shows a normal intelligent light 12 connected normally by a unidirectional DMX path 14 to a lighting control console or other DMX source such as a 3D tracking light control system 16 through a 3D light controller module 18. For simplicity only one 3D light controller module is shown but there is no reason why more than one of these modules should not form part of the DMX chain. While the 3D light controller module 18 is shown separately from the "normal lamp" 12 it could be built as an integral part of the lamp itself thus forming a sub-system of the lamp.

In this case DMX might well not be used to communicate with - 12 - the normal portion of the lamp 12 since connections could be made by more direct means within the light circuitry itself .

Figure 2 also shows the DMX signal path 20 for the light controller module 18 whose input is connected to the lighting console 16 and whose output is connected to a series of other DMX controllable lamps 22. DMX data leaves the lighting control console 16 and is sent via DMX to the 3D light controller module 18. The incoming DMX data is echoed to the DMX output of the module 18 with channels reserved for the transmission of internal data germane to the 3D light controller module 18 and the normal intelligent lamp 12 to which it is attached.

The data sent from the DMX output of the 3D light controller module 18 continues along the DMX chain until it leaves the DMX output of the last lamp 22 in the DMX chain and is returned to the DMX input of the lighting control console by path 42. DMX data entering at this input contains reserved channel information from all 3D light controller modules 18 connected in the DMX chain which can then be received, examined, and acted upon by software of the lighting controller 18.

Figure 3 is a block diagram of the internal working of the 3D light controller module 18. The base DMX address of the module 18 is set via the base address dip switches 24 and is read at power-up of microcontroller/DSP 30 or when the switches are changed during operation. Incoming DMX data is sent both to the DMX I/O controller 26 and to the - 13 - module's light DMX out connector 28 where it is sent to the normal lamp's DMX input.

The DMX I/O controller 26 functions as a device which echoes incoming DMX data to the DMX output 28 with minimum latency. The only DMX data not echoed is incoming data sent on the reserved channels (14A) for this module. These data are discarded and replaced by data generated by the module 18 itself. What is actually sent on these channels will vary with functional mode and with the type of operation desired but it will generally be information germane to either the 6DOF coordinates of the lamp its real-time pan/tilt encoder coordinates from the pan/tilt encoders 40, or to the results of calculations involving these coordinates done by the microcontroller/DSP 30.

The microcontroller/DSP 30 can be either a relatively inexpensive microcontroller such as the 89C52, a more complex DSP chip such as the TMS320C50, or a PC ^v 486 chip or chip set depending on the complexity of calculation it is desired to perform. In some situations it may be desirable to have all complex calculations performed externally by the lighting control console 16 and/or 3D tracking light control system (see Figure 2) . In other situations, it may be desirable to have the 3D light controller module 18 perform a variety of complex calculations itself. In any event, the microcontroller/DSP 30 monitors the incoming DMX stream for commands. At the same time, it monitors a laser ranging sub-system 32 via a parallel or serial port connection. The laser ranging subsystem 32 provides information about the distance between - 14 - the lamp and the surface onto which the light is shining. When the laser ranging sub-system 32 is active (something that is controlled by the microcontroller/DSP) this information is updated up to 10 times per second. The microcontroller/DSP is also connected to an analog to digital ( "A/D" ) converter 34 and a two position SPDT electronic switch 36. This switch is connected to a two axis orientation sensor 37 which continuously measures the pitch and roll angles of the 3D light controller module (which is mounted on the normal lamp 12 so that its orientation is the same as that of the normal lamp) . By this arrangement, the microcontroller/DSP can measure these angles during calibration. The microcontroller/DSP 30 is also provided with some RAM and/or ROM memory 38 for storing data plus configuration information such as light patterns etc. and it accesses/uses the RAM/ROM as needed.

Based on instructions from the lighting control console 16, the module containing the laser ranging subsystem 32 can be scanned through a series of pan/tilt angle settings with the distance from the light to the point upon the stage collinear with a line through said pan/tilt angle setting being measured by the laser ranging subsystem 32. Given this information (pan/tilt angle of the light plus range of the light to the stage point plus knowledge of the light's X, Y, Z position and yaw, pitch, roll orientation) it is possible to calculate the 3D positional X, Y, Z coordinates of said stage point. This calculation may either be carried out by the light itself or the relevant information (mentioned above) may be transmitted from the light to the lighting control console - 15 - 16 where it can also be calculated.

Given a series of such points, a 3D model of the stage area plus any props, walls, etc. present upon the stage may be generated and stored for use by software such as a virtual light CAD modelling program or other applications as deemed appropriate. Such virtual lighting CAD programs currently exist and it is always problematic to match up the virtual world created as a model of the performance space with the performance space as it was actually built. The 3D digitization process would allow for rationalization of the virtual model with the constructed reality. In the scanning process use may be made of the pan/tilt encoders 40 since these allow near instantaneous knowledge of the pan/tilt angles of the light which, in turn, allow the digitization process to proceed at a speed limited by the number of range measurements possible per second. Without this, the light would have to moved to one pan/tilt angle setting and stopped there for a "settling period" to be sure that the light was no longer moving and was stably pointed. Only then may the laser ranging subsystem 32 be used to determine the distance to this point, greatly slowing the digitization process.

As an addition, the microcontroller/DSP 30 may also monitor the outputs of pan/tilt encoders 40 which provide real-time information about where the lamp is pointing. This information may be used to provide closed-loop control of the lamp's positioning with all of the advantages this entails. - 16 -

Finally, the microcontroller/DSP may also perform predictive calculations such as Kalman filtering of pan/tilt data to allow the light to lead a performer or object in their trajectory rather than lagging behind and following them.

Claims

- 17 - CLAIMS :

1. A robotic lighting system equipped to respond to a flow of three dimensional control information comprising at least one variably responsive intelligent lamp (12, 22) controlled by a computer communication means (14) to provide input data for the lamp characterized in that the system includes feed back means (14) from the lamp to the computer to provide data to the computer in real-time as to the six degrees of freedom of the lamp, the actual pan/tilt coordinates of the lamp and to provide ranging means between the lamp and a surface on which it is projected.

2. A robotic lighting system equipped and ready to respond to a flow of three dimensional control information comprising at least one variably responsive lamp (12, 22) having six degrees of freedom and being controlled by a computer (30) through computer communication means (16) through a unidirectional communication path (14) and light control means (18) ; characterized in that there is also provided: a number of reserved computer communication channels (14A) from said computer communication means (16) through said light control means (18) to said lamp (12, 22) for bidirectional flow of data; data feed back means (42) from the lamp to the computer communication means (16) for output data in real time from said reserved channels (14A) to the computer (30) ; monitor means (32, 37) in the light control means (18) to monitor operation of the lamp (12) and provide modification data; and modification means (30) to modify input data for said reserved channels (14A) with said modification data.

3. A robotic lighting system as claimed in claim 2 characterized in that said reserved computer communication channels (14A) are communication channels of DMX- 512 protocol .

4. A robotic lighting system as claimed in claim 2 characterized in that said reserved computer communication channels (14A) are bi-directional communication channels of TCP/IP protocol.

5. A robotic lighting system as claimed in claim 2 characterized in that the monitor means includes ranging means (32) to monitor the distance of a projection surface (15) from the lamp (12, 22) .

6. A robotic lighting system as claimed in claim 2 characterized in that the monitor means includes a three axis orientation sensor.

7. A robotic lighting system as claimed in claim 1 characterized in that means are provided to calculate input data as three dimensional positional X, Y, and Z coordinates from pan/tilt coordinates and ranging data; and in that the lamp is responsive to said three dimensional X, Y, and Z coordinates and said ranging data.