WO2011067456A1 - A method and a system for a work machine with a boom - Google Patents

A method and a system for a work machine with a boom Download PDF

Info

Publication number
WO2011067456A1
WO2011067456A1 PCT/FI2009/050975 FI2009050975W WO2011067456A1 WO 2011067456 A1 WO2011067456 A1 WO 2011067456A1 FI 2009050975 W FI2009050975 W FI 2009050975W WO 2011067456 A1 WO2011067456 A1 WO 2011067456A1
Authority
WO
WIPO (PCT)
Prior art keywords
working area
machine
work machine
indicator
boom
Prior art date
Application number
PCT/FI2009/050975
Other languages
French (fr)
Inventor
Osmo Luoto
Original Assignee
John Deere Forestry Oy
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by John Deere Forestry Oy filed Critical John Deere Forestry Oy
Priority to SE1250574A priority Critical patent/SE1250574A1/en
Priority to BR112012013204A priority patent/BR112012013204A2/en
Priority to PCT/FI2009/050975 priority patent/WO2011067456A1/en
Priority to RU2012127569/11A priority patent/RU2524737C2/en
Priority to FI20125702A priority patent/FI129431B/en
Publication of WO2011067456A1 publication Critical patent/WO2011067456A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01GHORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
    • A01G23/00Forestry
    • GPHYSICS
    • G07CHECKING-DEVICES
    • G07CTIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C5/00Registering or indicating the working of vehicles
    • G07C5/08Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
    • G07C5/12Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time in graphical form
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00Teaching not covered by other main groups of this subclass
    • G09B19/003Repetitive work cycles; Sequence of movements
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00Teaching not covered by other main groups of this subclass
    • G09B19/24Use of tools
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B9/00Simulators for teaching or training purposes
    • G09B9/02Simulators for teaching or training purposes for teaching control of vehicles or other craft
    • G09B9/04Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of land vehicles
    • G09B9/042Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of land vehicles providing simulation in a real vehicle
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B9/00Simulators for teaching or training purposes
    • G09B9/02Simulators for teaching or training purposes for teaching control of vehicles or other craft
    • G09B9/04Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of land vehicles
    • G09B9/05Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of land vehicles the view from a vehicle being simulated
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/80Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
    • A63F2300/8017Driving on land or water; Flying

Definitions

  • the simulator of a forest machine aims to help the user of the simulator by adding visual cues to the simulated environment.
  • the simulator may, for example, add a grid indicating a working area of the simulated forest machine so that the user can see the proper working area on the landscape.
  • the grid or other indicator may have a shape that is descriptive of the reach of the forest machine, for example a cone or a rectangle, and it may be attached to the forest machine so that it is able to move with the machine in the simulated environment.
  • the grid may be arranged to be shown on top of the terrain or ground objects of the environment through arrangements in computer graphics.
  • the arrangement according to the invention may help the user to learn proper working methods, e.g. to operate the forest machine so that the working area is on the side of the machine if such operation provides for a better reach and stability.
  • the apparatus for controlling a forest machine is a work machine or a forest machine simulator
  • the method further comprises modeling the work machine or forest machine by a computer so that at least one part of the forest machine is a computer-modeled part, and displaying movement of the computer-modeled part of the work machine or forest machine to a user of the apparatus.
  • the method further com- prises electronically modeling the terrain around the machine with a terrain form, and electronically indicating the working area of the forest machine on top of the terrain form, and electronically indicating the working area with the help of a texture.
  • the indication of the working area comprises at least one of the group of indicator for movement track of the machine, indicator for closest reach of working means or the boom, indicator for furthest reach of working means or the boom, positive working area indicator, negative working area indicator, a working cone indicator and a working square indicator, and an asymmetric working area indicator based on attitude and position of the work machine.
  • the control means are configured to be used with a simulated machine or a real-world machine.
  • the work machine is a real-world work machine such as a forestry machine or a loader having a mechanical boom.
  • the method further comprises indicating the working area using a head-up display.
  • an apparatus com- prising a processor, memory including computer program code, the memory and the computer program code configured to, with the processor, cause the apparatus to electronically determine a dimension of at least part of the working area of a boom pivotally mounted on a work machine or a forest machine, and electronically indicate the working area of the machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the machine.
  • the apparatus further comprises computer program code configured to, with the processor, cause the appa- ratus to model the forest machine by a computer so that at least one part of the forest machine is a computer-modeled part, and to display movement of the computer-modeled part of the forest machine to a user of the apparatus.
  • the apparatus further comprises computer program code configured to, with the pro- cessor, cause the apparatus to electronically model the terrain around the machine with a terrain form, and electronically indicate the working area of the forest or work machine on top of the terrain form, and electronically indicate the working area with the help of a texture.
  • the apparatus further comprises com- puter program code configured to, with the processor, cause the apparatus to align the orientation of the indication of the working area to one of the orientation of the whole machine, the orientation of a part of the machine, the orientation of the cabin or to the orientation of the working means such as the boom pivot.
  • the apparatus further comprises computer program code configured to, with the processor, cause the apparatus to determine information on the forestry task being performed by the user, and electronically indicate the working area of the forest machine based on said information on the forestry task.
  • the indication of the working area comprises at least one of the group of indicator for movement track of the machine, indicator for closest reach of working means, indicator for furthest reach of working means, positive working area indicator, negative working area indicator, a working cone indicator and a working square indicator, and an asymmetric working area indicator based on attitude and position of the work machine.
  • the control means are configured to be used with a simulated machine or a real-world machine.
  • the work machine is a real-world work machine such as a forestry machine or a loader having a mechanical boom.
  • the apparatus further comprises a head-up display for indicating the working area.
  • an apparatus comprising a processor, memory including computer program code, the memory and the computer program code configured to, with the processor, cause the apparatus to perform the method as described in the first aspect.
  • the work machine is a real-world work machine such as a forestry machine or a loader having a mechanical boom.
  • the apparatus further comprises a head-up display for indicating the working area.
  • a computer program product stored on a computer readable medium and executable in a data processing device, wherein the computer program product comprises a computer program code section for determining a dimension of at least part of the working area of a boom pivotally mounted on a work or a forest machine, and a computer program code section for indicating the working area of the machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the machine.
  • a computer program product stored on a computer readable medium and executable in a data processing device, wherein the computer program product comprises computer program code sections for carrying out the method as described in the first aspect.
  • an apparatus comprising control means of a forest machine or a boom pivotally mounted on a work machine or a forest machine for receiving control input from a user and producing control signals for controlling the apparatus, means for processing said control signals from the control unit, means for determining a dimension of at least part of the working area of the forest or work machine, and means for indicating the working area of the machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the forest machine.
  • the apparatus further comprises means for modeling the machine by a computer so that at least one part of the machine is a computer-modeled part, and means for displaying movement of the computer-modeled part of the forest machine to a user of the apparatus.
  • a system comprising at least a first apparatus and a second apparatus according to the first, second and/or fifth aspect, wherein the first apparatus and the second apparatus are connected with a data connection, and the first apparatus and the second apparatus are configured to show information indicative of the operation of the second apparatus at the first apparatus.
  • the first apparatus is configured to display the graphical form of the second apparatus in the virtual scenery of the first apparatus.
  • a system comprising at least a first apparatus and a second apparatus according to the second aspect, wherein the first apparatus and the second apparatus are connected with a data connection, and the first apparatus and the second apparatus are configured to show information indicative of the operation of the second apparatus at the first apparatus.
  • the work machine is configured to indicate the working area in response to operating the boom, and to remove the working area indicator when the boom is not being operated.
  • the work machine further comprises a head-up display for indicating the working area.
  • a work machine such as a forestry machine comprising a movable boom, a display, a processor, memory including computer program code, the memory and the computer program code configured to, with the processor, cause the work machine to electronically indicate a working area of the boom on the display.
  • the work machine is configured to indicate the working area in response to operating the boom, and to remove the working area indicator when the boom is not being operated.
  • the work machine comprises a head-up display for indicating the working area.
  • Fig. 1 shows a flow chart of the operation of a forest machine according to an embodiment of the invention
  • Fig. 2 shows a flow chart of the operation of a forest machine according to an embodiment of the invention
  • Fig. 4 shows a block diagram of a forest machine simulator according to an embodiment of the invention
  • Fig. 5 shows a simulated view of the forest machine simulator according to an embodiment of the invention
  • FIG. 7 shows a view of the forest machine in training mode according to an embodiment of the invention
  • Fig. 8 shows a view of editing a simulated terrain in the forest machine for the training mode according to an embodiment of the invention.
  • Fig. 1 0 shows an example of displaying a working area of a work machine on a display inside the working machine.
  • Fig. 1 1 shows an example of using a head-up display for indicating the working area of a work machine in the real world.
  • Different methods of working may also define different optimal ways of working with the boom and the saw head. For example, it may be advantageous and efficient to cut trees from the side of the machine, and not cutting trees from the front. The trees in the front will come to the optimal range of operation and the optimal direction of working (to the side of the machine) when the machine moves forward. Therefore, a working method may also define an optimal area of working. The working area indicators may help to learn how to position the machine correctly in relation to the trees or cut trunks, as explained later in more detail. For example, a loading machine may be positioned appropriately to the side of a pile of cut trunks.
  • the dimensions (or at least one dimension) of the working area are first determined in step 1 1 0. This may happen so that a computer has predefined information about the working area in numeric format, or has a predefined working area pattern stored in its memory. When the dimensions of the working area are known, the working area can be indicated on top of the terrain surrounding the forest machine in step 1 20. This may happen e.g. by projecting a graphical pattern comprising lines and shapes onto the terrain by using graphical abilities of the computer. The projection may happen inside the virtual world of the computer, or projection means such as a head- up display may be used in a real world.
  • Fig. 2 shows a flow chart of the operation of a forest machine according to an embodiment of the invention.
  • the working area is aligned with the forest machine so that the working area corresponds to the situation in the real world.
  • the working area may also depend on the position, attitude and/or tilt of the work machine so that work areas on different sides of the machine are of different size and/or shape.
  • the working area below the machine on a side of a hill may be smaller than the working area above the machine. This may be done e.g. to increase the stability of the machine.
  • Modeling the terrain with a terrain form in step 270 is one example of how the working area can be displayed on the terrain.
  • the terrain form may e.g. be a triangle mesh or another shape that is able to lay on top of the terrain in the model.
  • the terrain form may be adjusted to be slightly on top of the terrain so that an opaque terrain form would mask the underlying terrain.
  • a texture or other graphical element may be determined, e.g. loaded from a file or generated by an algorithm.
  • the texture or graphical element indicates the working area and may contain e.g. a shape of a cone or a rectangle, as well as color or hash fill.
  • the texture may then be mapped to the terrain form for showing the working area.
  • Figs. 3a and 3b show a forest or work machine simulator according to an embodiment of the invention.
  • the simulator is used to teach a person the operation and work methods of a real forest machine.
  • Real forest machines are expensive, and operating them needs to be done in a proper environment, i.e. forest.
  • the teaching use of these expensive machines takes them away from productive work of forest harvesting.
  • the conditions in a forest may often be harsh and impede learning. Due to safety requirements, only one learner can be instructed at a time in real world forest machine.
  • simulators like the one in Fig. 3 have been built for teaching purposes.
  • a forest machine simulator comprises controls 31 0 and 31 2 that can be operated by the user.
  • Fig. 4 shows a block diagram of a forest machine simulator according to an embodiment of the invention.
  • the controller 41 0 of the simulator may comprise e.g. the controls 41 5 and some control logic 41 8 for creating control signals to the machine.
  • the controls 41 5 may be similar to the controls of a real forest machine, they may be a simplified version of the real controls, or e.g. regular computer controls may be used.
  • the control logic may include circuitry and/or software on a processor like a microcontroller for generating signals according to the controls operated by the user.
  • the interface to the simulator 440 from the controller 41 0 may be the same or highly similar to the interface between a real forest machine and its controller.
  • the simulator 440 is connected to the controller 41 0 via signal lines.
  • the simulator comprises I/O circuitry 441 .
  • the simulator may also comprise an input block 442 for accepting input from the user e.g. through a keyboard and a mouse.
  • the simulator comprises a processor 445 and memory 446 for running and storing computer program code for the simulator operations.
  • There may be multiple processors e.g. a general purpose processor and a graphics processor and/or multiple different memories e.g. volatile memory for storing data and programs at run-time and nonvolatile memory like a hard disk for permanently storing data and programs.
  • the simulator may also include a video controller 448 and an audio controller 449 for generating signals that can be produced to the user with computer accessories.
  • a first simulator and a second simulator may be connected with a data connection so that the second apparatus can send information to the first apparatus, and the first apparatus may show this information to the user or use it in the modeling of the scenery.
  • performance data of the second apparatus may be shown at the first apparatus.
  • the second apparatus may appear as a graphical form in the virtual scenery of the first apparatus. This may give a view to the user that there is another machine working in the same area, and the user may see this other machine moving and working.
  • Such a system may also be used in a teaching purpose where a teacher or a supervisor is able to observe any single machine or multiple machines working in the environment. Practically, the first machine may be a harvester machine and the second machine may be a loader machine.
  • the simulator produces output to the user through output means 470.
  • the video controller 448 may be connected to a display 475.
  • the display may be e.g. a flat panel display or a projector for producing a larger image.
  • the display may also be adapted to wearable video goggles.
  • the audio controller 449 may be connected to a sound source 478 such as loudspeakers or earphones.
  • Fig. 5 shows a simulated view of the forest machine simulator according to an embodiment of the invention.
  • the terrain may have shapes e.g. ditches like those next to the road 51 6 and elevation variations like hills.
  • the forest machine itself may comprise two main parts 520 (rear body) and 522 (front body), with the boom 524 attached e.g. to the front part and having a saw head 526.
  • the forest machine may run on wheels 528.
  • the working area indicators 530, 532 and 536 are projected around the forest machine.
  • the working area indicators 530 and 532 show optimal working areas on the side of the machine, and the working area indicator 536 shows a sub-optimal working area in the front.
  • the working area indicators may comprise elements 540 and 542 to indicate the furthest and closest reach of the boom, respectively.
  • the movement track of the forest machine may also be indicated with the working area indicators with track signs 544 for showing the width of the needed track when the forest machine moves forward.
  • the working area indicators are assigned or otherwise moving with the machine, i.e. at least somewhat attached to the machine coordinate system.
  • the working area indicator center point may be attached to the boom pole center point i.e. the axis of the pole bearing of the boom 524.
  • the orientation of the working area indicators may be aligned with the front body 522 of the harvester machine i.e. the same body where the boom 524 is attached or to the boom pivot point (the bearing housing).
  • the working area indicators may be aligned with the back body of the machine, since the pole is mounted to the back body in a loader machine.
  • the center point of the working area indicators may be arranged to shift e.g. when the machine is tilted or some other conditions are present.
  • the alignment of the working area indicators may also be done in a flexible manner with reference to multiple elements, like the front body 522 and the rear body 520 together, or the alignment may change with the tilting of the machine, ground shapes and so on.
  • the working area indicators may comprise different radiuses, different shapes, circular cones, elliptic cones, squares, and zones of any kind. Colors can be used to indicate a positive or optimal working area and a negative or sub-optimal working area.
  • the working area indicators or parts of the indicators may be labeled with letters, numbers, words, icons or other labels. For example, squares of the working area indicator may receive labels like "AT, "A2", ... , "A1 2", “ ⁇ , "B2" "L1 1 " and "L1 2".
  • the tasks at hand may then refer to these labels to guide the student.
  • the boom has a better reach from the side of the machine, and with the help of the working area indicators, this can be now taught at the simulator.
  • the right working methods and work order can be taught with the help of the working area indicators, as well.
  • the working area indicator may helping to operate the loader in a stable way so that the danger of tipping the machine is smaller.
  • the harvester may guide to place the left-over branches to a certain area.
  • the guide may also help to position the harvester or the loading machine correctly, or the guide may indicate an area for loading.
  • the working area indicator may turn to green or may blink when the machine has been positioned correctly according to a given task.
  • textures may be used in showing the working area.
  • the texture may be attached on a terrain form, or it may be attached to the ground object directly.
  • the terrain form is a floating intermediary object following the ground on top of which the texture is attached.
  • the terrain form is placed upwards in z-direction from the ground or towards the viewer in line of sight.
  • the terrain form may "dive" inside objects on terrain, or it may be shown on top of the objects.
  • the terrain object may follow the ground carefully, as seen in context of the ditch next to the road in Fig. 5.
  • Figs. 6a and 6b show schematic representations of different working area indicators for the forest machines according to an embodiment of the invention.
  • the reach of the boom 603 may be shown with a cone-shaped indicator 610, 612 (side working areas) and 630 (front working area).
  • the furthest reach of the boom 603 may be shown with arcs 614 and 626, as well as the closest reach with 616 and 622.
  • There may be other indicators for showing an optimal range of the boom such as the area between arcs 618 and the arc 624.
  • the track of the machine for moving forward may be shown with the front working area.
  • There may also be a negative working area shown such as the back working area 640 where the boom cannot or should not reach.
  • the different working area indicators may have sub-areas, and these sub areas may be labeled.
  • the left working area may be divided to sections L1 , L2, L3, L4 and L5
  • the right working area may be divided to sections R1 , R2, R3, R4 and R5
  • the front working area may be divided to sections F1 , F2 and F3
  • the back working area may be divided to sections B1 , B2 and B3.
  • These labels may then be used in e.g. guiding the task at hand by referring in the instructions or scoring to these labels.
  • An asymmetric working area indicator based on attitude and position of the work machine may also be used.
  • Fig. 7 shows a view of the forest machine in training mode according to an embodiment of the invention.
  • the training mode provides a clear scoring system that can be defined to follow the student's progress.
  • working area indicators may provide advantages for learning. For example, a good working method can be trained by the simulator. The trees are removed from the side, not from the front. The trees in the front will come to optimal working area (side) when the machine moves forward. As another example, the needed driving track (width) can be shown, and minimum and maximum reach of the boom, as well as an optimal operating range are shown.
  • the use of working area indicator makes it possible to continue the learning in a simulator for a longer time without the need to move to a real machine, since less "false learning" happens. This may provide cost savings and accelerate learning.
  • the working area indicator may also help to learn the reach of the boom so that this knowledge is then usable in real world.
  • the working area indicator helps to understand the effect of the boom to machine stability - in a simulator, this is otherwise challenging to understand for a student. Visible information on widths and radiuses helps to learn proper positioning of the harvester machine and proper use of boom in a loader machine.
  • Fig. 8 shows a view of editing a simulated terrain in the forest machine for the training mode according to an embodiment of the invention.
  • the simulator software may comprise, for example, the following three modes that can be implemented as separate, interconnected programs or different views in a single program, or any combination of the same.
  • the first mode may include editors for the surrounding terrain, trees, roads and other objects in the scenery - it may be called a scenery editor.
  • the scenery editor allows to shape the terrain in terms of surface elevation, and it allows to position different objects onto the terrain (or possibly into the terrain).
  • the different objects may be positioned either automatically or manually, or semi-automatically as a group.
  • the scenery editor may allow loading a pre-created scenery from a file, and it may allow storing a part of the scenery or the whole scenery to a file for later use.
  • the second mode may be a scoring editor, where points for different operations can be defined.
  • the third mode may be the simulation mode, as shown e.g. in Fig. 5.
  • Figs 9a and 9b show two types of forest machines whose operation can be modeled in the simulator according to an embodiment of the invention.
  • a harvester is shown, comprising two main parts, the engine part 910 and the cabin part, connected with a swivel 925.
  • the harvester may have wheels 930 to run on the terrain, or other means of supporting it on the ground.
  • the harvester has a boom with a saw head for cutting the trees and delimbing them.
  • a loader machine for transporting the trunks from the forest is shown.
  • the loader machine has two parts, the cabin part 950 and the cage part 960.
  • the loader machine has a boom and a grabber head attached to the boom for collecting the trunks to the cage 980.
  • the loader machine may also run on wheels 990.
  • Other types of machines are also possible in the simulator.
  • Fig. 10 shows an example of displaying a working area of a work machine on a display inside the working machine.
  • the display is arranged to show to the operator of the work machine information on the work machine status 1030 as well as information on the task at hand 1040, 1 050 and 1055.
  • the display may show the engine and battery status of a forest machine or a harvester, and it may indicate to the operator the dimensions of the tree being cut and the kind of tree being cut (e.g. here "KUUSI" or "SPRUCE”).
  • the display may be adapted to show the working area indicators on the screen.
  • There may also be information like the operator's name (here "Pekka"), and the machine may record the produced output related to the person operating the machine, allowing multiple persons to successively operate the machine.
  • the display may show the position and attitude of the machine 1010, as well as the direction, length and movement of the boom 1 095.
  • the left working area 1025, the right working area 1020 as well as the front working area 1022 may be indicated, with the respective sub-areas and the sub-area labels. Showing the working area indicators on the screen may help the operator to operate the machine optimally.
  • the terrain may be shown in the work area indicator on the screen with the help of camera or scanner means, e.g. attached to the boom.
  • the working area indicator may also be shown on the real controller screen in a simulator, in addition to showing the working area indicators in the simulated world.
  • the working area indicator may also be brought to the screen at times when the boom is being moved by the operator, and removed from screen at other times. This helps to use the display space optimally and allows the operator to observe only the most important matters on the screen. For example, when the tree is being cut, and the boom is not moving, the operator may appreciate to see the data related to the cutting. Therefore, at times when the tree is being cut, indicators like the length of the trunk and volume of prepared logs may be shown. At times of other operations, other data may be shown, and the working area indicator may be removed from the screen or minimized to occupu less screen area.
  • the working area indicator When the operator is moving the boom, the working area indicator may be brought onto the screen, or it may be enlarged from a small indicator to a larger indicator to be better visible, or the working area indicators may fill essentially the entire screen. Showing the boom movement and position may require sensors on the boom for sensing the direction, length (extension) and movement of the boom.
  • Fig. 1 1 shows an example of using a head-up display for indicating the working area of a work machine in the real world.
  • the operator of the work machine may have been trained in a simulator by showing the working area indicators in the simulated world, and possibly at the same time on the real controller screen as explained with reference to Fig. 1 0.
  • the real work machine like a forest machine or a harvester or a loader machine, may be fitted with a head-up display for showing the working area indicators familiar from the training indicator.
  • the real work machine has a cabin with windows 1 1 1 0 (right window), 1 1 20 (left window) and 1 1 30 (front window).
  • the work machine may have the same controls 1 1 40, 1 1 45, and 1 1 50 as in the simulator, and the control screen 1 1 45 may show the same working area indicators as in the simulator. To even further improve the usefulness of the working area indicators, they may be shown projected onto the windows with the help of head-up displays. There may be multiple head-up displays, for example three or four or more, or just one.
  • the right working area indicator 1 1 18 is projected onto the right window 1 1 1 0 with the help of a head-up display 1 1 15.
  • the left working area indicator 1 128 is projected onto the left window 1 120 with the help of a head-up display 1 125.
  • the front working area indicator 1 138 is projected onto the front window 1 130 with the help of a head-up display 1 135.
  • the head-up displays may be adapted to take into account the height of the operator so that the working area indicators are seen at the right place on top of the terrain.
  • the head-up displays may also be able to adapt to the movement of the operator's head, and they may adapt to the tilt of the machine by changing the projection onto the window.
  • a terminal device may comprise circuitry and electronics for handling, receiving and transmitting data, computer program code in a memory, and a processor that, when running the computer program code, causes the terminal device to carry out the features of an embodiment.
  • a network device may comprise circuitry and electronics for handling, receiving and transmitting data, computer program code in a memory, and a processor that, when running the computer program code, causes the network device to carry out the features of an embodiment.

Abstract

The invention relates to a forest machinery or work machinery, especially to a forest machine simulator. The simulator of a forest machine aims to help the user of the simulator by adding visual cues to the simulated environment. The simulator may, for example, add a grid indicating a working area of the simulated forest machine so that the user can see the proper working area on the landscape.

Description

A method and a system for a work machine with a boom
Field of the Invention
The present invention relates to a work machine or a forest machine, especially to a computer-simulated machine in an artificial environment. Background of the Invention
The development of forest machines has enabled significant improvements in productivity in forest care and harvesting. Modern forest machines are able to work accurately and with high speed, and the output of such machines is known by measurements already at the time the work is being done. The machinery in these sophisticated devices has many innovative solutions that increase safety and productivity of such machines. Operating a modern forest machine is a demanding task that requires high professional skills. Learning to operate the machine involves both theoretical and practical training. The theoretical training includes e.g. learning to use the right principles at work and understanding the purpose of each phase in the work. Practical training involves learning to operate the physical controls of a forest machine, typically by operating the machine in real nature.
In recent times, it has become possible to practice operating a forest machine by using a simulator. In a very developed simulator, such as the ones available for John Deere® forest machines, the physical controls are the same or at least very much like those in a real forest machine. The operations of the machine are simulated by a computer, and may resemble the operation of a real machine to a good degree. Unfortunately, however good the simulation of the machine itself, the computer-created environment cannot completely correspond to the real world environment in a forest. For example, the perspective of the environment on the computer screen may be more difficult to grasp, and imagination and spatial abstraction skills may be needed to see the position and operation of the machine in relation to the simulated environment. This challenge is partly created by the limited display means compared to real world sight, but also by other factors.
There is, therefore, a need for a solution that makes it easier for a person to learn how to operate a forest machine in a simulated environment. Summary of the Invention
Now there has been invented an improved method and technical equipment implementing the method, by which the above problems are alleviated. Various aspects of the invention include a method, an appa- ratus, a server, a client and a computer readable medium comprising a computer program stored therein, which are characterized by what is stated in the independent claims. Various embodiments of the invention are disclosed in the dependent claims. The simulator of a forest machine according to various embodiments of the invention aims to help the user of the simulator by adding visual cues to the simulated environment. The simulator may, for example, add a grid indicating a working area of the simulated forest machine so that the user can see the proper working area on the landscape. The grid or other indicator may have a shape that is descriptive of the reach of the forest machine, for example a cone or a rectangle, and it may be attached to the forest machine so that it is able to move with the machine in the simulated environment. The grid may be arranged to be shown on top of the terrain or ground objects of the environment through arrangements in computer graphics. The arrangement according to the invention may help the user to learn proper working methods, e.g. to operate the forest machine so that the working area is on the side of the machine if such operation provides for a better reach and stability.
According to a first aspect, there is provided a method for displaying a working area of a boom pivotally mounted on a work machine at an apparatus for controlling the boom and/or the work machine, the method comprising electronically determining a dimension of at least part of the working area of the work machine, and electronically indicating the working area of the work machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the work machine.
According to an embodiment, the apparatus for controlling a forest machine is a work machine or a forest machine simulator, and the method further comprises modeling the work machine or forest machine by a computer so that at least one part of the forest machine is a computer-modeled part, and displaying movement of the computer-modeled part of the work machine or forest machine to a user of the apparatus. According to an embodiment, the method further com- prises electronically modeling the terrain around the machine with a terrain form, and electronically indicating the working area of the forest machine on top of the terrain form, and electronically indicating the working area with the help of a texture. According to an embodiment, the method further comprises aligning the orientation of the indication of the working area to one of the orientation of the whole machine, the orientation of a part of the machine, the orientation of the cabin or to the orientation of the working means of the machine such as the boom pivot. According to an embodiment, the method further comprises determining information on the forestry task being performed by the user, and electronically indicating the working area of the forest machine based on said information on the forestry task. According to an embodiment, the indication of the working area comprises at least one of the group of indicator for movement track of the machine, indicator for closest reach of working means or the boom, indicator for furthest reach of working means or the boom, positive working area indicator, negative working area indicator, a working cone indicator and a working square indicator, and an asymmetric working area indicator based on attitude and position of the work machine. According to an embodiment, the control means are configured to be used with a simulated machine or a real-world machine. According to an embodiment, the work machine is a real-world work machine such as a forestry machine or a loader having a mechanical boom. According to an embodiment, the method further comprises indicating the working area using a head-up display.
According to a second aspect, there is provided an apparatus com- prising a processor, memory including computer program code, the memory and the computer program code configured to, with the processor, cause the apparatus to electronically determine a dimension of at least part of the working area of a boom pivotally mounted on a work machine or a forest machine, and electronically indicate the working area of the machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the machine.
According to an embodiment, the apparatus further comprises computer program code configured to, with the processor, cause the appa- ratus to model the forest machine by a computer so that at least one part of the forest machine is a computer-modeled part, and to display movement of the computer-modeled part of the forest machine to a user of the apparatus. According to an embodiment, the apparatus further comprises computer program code configured to, with the pro- cessor, cause the apparatus to electronically model the terrain around the machine with a terrain form, and electronically indicate the working area of the forest or work machine on top of the terrain form, and electronically indicate the working area with the help of a texture. According to an embodiment, the apparatus further comprises com- puter program code configured to, with the processor, cause the apparatus to align the orientation of the indication of the working area to one of the orientation of the whole machine, the orientation of a part of the machine, the orientation of the cabin or to the orientation of the working means such as the boom pivot. According to an embodiment, the apparatus further comprises computer program code configured to, with the processor, cause the apparatus to determine information on the forestry task being performed by the user, and electronically indicate the working area of the forest machine based on said information on the forestry task. According to an embodiment, the indication of the working area comprises at least one of the group of indicator for movement track of the machine, indicator for closest reach of working means, indicator for furthest reach of working means, positive working area indicator, negative working area indicator, a working cone indicator and a working square indicator, and an asymmetric working area indicator based on attitude and position of the work machine. According to an embodiment, the control means are configured to be used with a simulated machine or a real-world machine. According to an embodiment, the work machine is a real-world work machine such as a forestry machine or a loader having a mechanical boom. According to an embodiment, the apparatus further comprises a head-up display for indicating the working area.
According to a third aspect, there is provided an apparatus comprising a processor, memory including computer program code, the memory and the computer program code configured to, with the processor, cause the apparatus to perform the method as described in the first aspect. According to an embodiment, the work machine is a real-world work machine such as a forestry machine or a loader having a mechanical boom. According to an embodiment, the apparatus further comprises a head-up display for indicating the working area. According to a fourth aspect, there is provided a computer program product stored on a computer readable medium and executable in a data processing device, wherein the computer program product comprises a computer program code section for determining a dimension of at least part of the working area of a boom pivotally mounted on a work or a forest machine, and a computer program code section for indicating the working area of the machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the machine. According to a fifth aspect, there is provided a computer program product stored on a computer readable medium and executable in a data processing device, wherein the computer program product comprises computer program code sections for carrying out the method as described in the first aspect.
According to a fifth aspect, there is provided an apparatus comprising control means of a forest machine or a boom pivotally mounted on a work machine or a forest machine for receiving control input from a user and producing control signals for controlling the apparatus, means for processing said control signals from the control unit, means for determining a dimension of at least part of the working area of the forest or work machine, and means for indicating the working area of the machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the forest machine. According to an embodiment, the apparatus further comprises means for modeling the machine by a computer so that at least one part of the machine is a computer-modeled part, and means for displaying movement of the computer-modeled part of the forest machine to a user of the apparatus.
According to a sixth aspect, there is provided a system comprising at least a first apparatus and a second apparatus according to the first, second and/or fifth aspect, wherein the first apparatus and the second apparatus are connected with a data connection, and the first apparatus and the second apparatus are configured to show information indicative of the operation of the second apparatus at the first apparatus. According to an embodiment, the first apparatus is configured to display the graphical form of the second apparatus in the virtual scenery of the first apparatus.
According to a seventh aspect, there is provided a system comprising at least a first apparatus and a second apparatus according to the second aspect, wherein the first apparatus and the second apparatus are connected with a data connection, and the first apparatus and the second apparatus are configured to show information indicative of the operation of the second apparatus at the first apparatus.
According to an embodiment, the work machine is configured to indicate the working area in response to operating the boom, and to remove the working area indicator when the boom is not being operated. According to an embodiment, the work machine further comprises a head-up display for indicating the working area. According to an eighth aspect, there is provided a work machine such as a forestry machine comprising a movable boom, a display, a processor, memory including computer program code, the memory and the computer program code configured to, with the processor, cause the work machine to electronically indicate a working area of the boom on the display. According to an embodiment, the work machine is configured to indicate the working area in response to operating the boom, and to remove the working area indicator when the boom is not being operated. According to an embodiment, the work machine comprises a head-up display for indicating the working area.
Description of the Drawings In the following, various embodiments of the invention will be described in more detail with reference to the appended drawings, in which
Fig. 1 shows a flow chart of the operation of a forest machine according to an embodiment of the invention;
Fig. 2 shows a flow chart of the operation of a forest machine according to an embodiment of the invention;
Figs. 3a and 3b
show a forest machine simulator according to an embodiment of the invention;
Fig. 4 shows a block diagram of a forest machine simulator according to an embodiment of the invention;
Fig. 5 shows a simulated view of the forest machine simulator according to an embodiment of the invention;
Figs. 6a and 6b
show schematic representations of different working indicators for the forest machine according to embodiment of the invention; Fig. 7 shows a view of the forest machine in training mode according to an embodiment of the invention;
Fig. 8 shows a view of editing a simulated terrain in the forest machine for the training mode according to an embodiment of the invention; and
Figs 9a and 9b
show two types of forest machines whose operation can be modeled in the simulator according to an embodiment of the invention.
Fig. 1 0 shows an example of displaying a working area of a work machine on a display inside the working machine.
Fig. 1 1 shows an example of using a head-up display for indicating the working area of a work machine in the real world.
Detailed Description of the Embodiments
In the following, several embodiments of the invention will be described in the context of a forest machine or generally work machine. It is to be noted, however, that the invention is not limited to forest machines alone. In fact, the different embodiments have applications widely in any environment where indicating the working area of a machine is required. For example, different kind of machines like cranes, loaders, agriculture machines and such may benefit from indication of the working area according to the invention. The working area especially needs to be indicated when the work machine or forest machine has a boom that may change in length or may otherwise be arranged to be able to reach at different distances from the work machine. The boom may be arranged so that there is a boom pivot around which the boom can rotate. Fig. 1 shows a flow chart of the operation of a forest machine according to an embodiment of the invention. A forest machine typically has a boom or another kind of an arrangement to bring the saw head of the machine to a tree. The boom may be arranged to change length as needed, for example by comprising two or more parts connected by a joint. This allows the saw head to be brought close to the machine by closing the angle in the joint, and far away from the machine by opening the angle. The boom therefore has a certain range of operation.
Different methods of working may also define different optimal ways of working with the boom and the saw head. For example, it may be advantageous and efficient to cut trees from the side of the machine, and not cutting trees from the front. The trees in the front will come to the optimal range of operation and the optimal direction of working (to the side of the machine) when the machine moves forward. Therefore, a working method may also define an optimal area of working. The working area indicators may help to learn how to position the machine correctly in relation to the trees or cut trunks, as explained later in more detail. For example, a loading machine may be positioned appropriately to the side of a pile of cut trunks.
In Fig. 1 , the dimensions (or at least one dimension) of the working area are first determined in step 1 1 0. This may happen so that a computer has predefined information about the working area in numeric format, or has a predefined working area pattern stored in its memory. When the dimensions of the working area are known, the working area can be indicated on top of the terrain surrounding the forest machine in step 1 20. This may happen e.g. by projecting a graphical pattern comprising lines and shapes onto the terrain by using graphical abilities of the computer. The projection may happen inside the virtual world of the computer, or projection means such as a head- up display may be used in a real world. Fig. 2 shows a flow chart of the operation of a forest machine according to an embodiment of the invention. In step 21 0, the forest machine is modeled with a computer. The model may comprise mechanical modeling, operational modeling of the engine, controls, boom and saw head and graphical modeling. This modeling of the forest machine acts as a basis for displaying the movement of the machine in step 220. For example, the movement of the boom may affect stability of the machine, and hitting objects may cause sounds to be produced.
In step 230, the task given to the user of the simulator is determined. More specifically, the parameters for success and the teaching goals of the task may be determined. This determination may happen e.g. by user input, or these data may be loaded from the computer memory. As explained earlier, the work area may be determined based on the task at hand in step 240, for example, the specific task parameters. The work area may also be determined without using information of the specific task, e.g. purely depending on the forest machine being used. When the working area is known, the dimensions of the working area may be determined in step 250. This may comprise e.g. computing the size of the working area or the zoom size of the working area template. In step 260, the working area is aligned with the forest machine so that the working area corresponds to the situation in the real world. The working area may also depend on the position, attitude and/or tilt of the work machine so that work areas on different sides of the machine are of different size and/or shape. For example, the working area below the machine on a side of a hill may be smaller than the working area above the machine. This may be done e.g. to increase the stability of the machine.
Modeling the terrain with a terrain form in step 270 is one example of how the working area can be displayed on the terrain. The terrain form may e.g. be a triangle mesh or another shape that is able to lay on top of the terrain in the model. The terrain form may be adjusted to be slightly on top of the terrain so that an opaque terrain form would mask the underlying terrain. In step 280, a texture or other graphical element may be determined, e.g. loaded from a file or generated by an algorithm. The texture or graphical element indicates the working area and may contain e.g. a shape of a cone or a rectangle, as well as color or hash fill. The texture may then be mapped to the terrain form for showing the working area. This mapping may happen so that points of the texture are assigned to points of the terrain form, and the texture is stretched onto the terrain form. When the texture is displayed in step 290, the resulting view is that the texture indicating the working area seems to be floating or drawn on top of the terrain and the working area is aligned to the position and attitude of the forest machine.
Figs. 3a and 3b show a forest or work machine simulator according to an embodiment of the invention. The simulator is used to teach a person the operation and work methods of a real forest machine. Real forest machines are expensive, and operating them needs to be done in a proper environment, i.e. forest. The teaching use of these expensive machines takes them away from productive work of forest harvesting. Furthermore, the conditions in a forest may often be harsh and impede learning. Due to safety requirements, only one learner can be instructed at a time in real world forest machine. For these and other reasons, simulators like the one in Fig. 3 have been built for teaching purposes. A forest machine simulator comprises controls 31 0 and 31 2 that can be operated by the user. These controls resemble the real controls of a forest machine, or they can be identical to them. Likewise, the seat 320 of the simulator may be a real seat used in a forest machine. The simulator logic 330 is built to be moved with the simulator e.g. under the seat. The display 340 may be a regular computer display in the front of the user. The simulator has been built to be movable, and it may contain handles 350 and wheels 352 for moving it around.
A forest machine like the one in Figs. 3a and 3b may be used to teach work methods of forestry to the user. In forestry, there are many concepts like work location and work direction that may be difficult to understand and visualize in real terrain. Furthermore, the task at hand, when teaching happens on a real machine, can be easily forgotten. On a simulator, the teaching situation and instructions for the task can be shown as a reminder to the student. This is a marked improvement to teaching in real world, where the task is often given by oral and written instructions. Yet, as shown in Figs. 3a and 3b, a simulator may give a very real learning experience to a user. For example, the simulator may comprise an ignition system 360 like an ignition system in a real forest machine, and it may comprise hand controls 31 0 and 31 2 as well as foot controls 370 (brake) that are very much like or exactly the same as in a real forest machine.
Fig. 4 shows a block diagram of a forest machine simulator according to an embodiment of the invention. The controller 41 0 of the simulator may comprise e.g. the controls 41 5 and some control logic 41 8 for creating control signals to the machine. The controls 41 5 may be similar to the controls of a real forest machine, they may be a simplified version of the real controls, or e.g. regular computer controls may be used. The control logic may include circuitry and/or software on a processor like a microcontroller for generating signals according to the controls operated by the user. The interface to the simulator 440 from the controller 41 0 may be the same or highly similar to the interface between a real forest machine and its controller.
The simulator 440 is connected to the controller 41 0 via signal lines. For handling the signals to/from the signal lines, the simulator comprises I/O circuitry 441 . The simulator may also comprise an input block 442 for accepting input from the user e.g. through a keyboard and a mouse. The simulator comprises a processor 445 and memory 446 for running and storing computer program code for the simulator operations. There may be multiple processors e.g. a general purpose processor and a graphics processor and/or multiple different memories e.g. volatile memory for storing data and programs at run-time and nonvolatile memory like a hard disk for permanently storing data and programs. The simulator may also include a video controller 448 and an audio controller 449 for generating signals that can be produced to the user with computer accessories.
There may also be multiple simulators connected to each other over a wireless or wired network (not shown). In such a system a first simulator and a second simulator may be connected with a data connection so that the second apparatus can send information to the first apparatus, and the first apparatus may show this information to the user or use it in the modeling of the scenery. For example, performance data of the second apparatus may be shown at the first apparatus. Also, the second apparatus may appear as a graphical form in the virtual scenery of the first apparatus. This may give a view to the user that there is another machine working in the same area, and the user may see this other machine moving and working. Such a system may also be used in a teaching purpose where a teacher or a supervisor is able to observe any single machine or multiple machines working in the environment. Practically, the first machine may be a harvester machine and the second machine may be a loader machine.
The simulator produces output to the user through output means 470. The video controller 448 may be connected to a display 475. The display may be e.g. a flat panel display or a projector for producing a larger image. The display may also be adapted to wearable video goggles. The audio controller 449 may be connected to a sound source 478 such as loudspeakers or earphones.
Fig. 5 shows a simulated view of the forest machine simulator according to an embodiment of the invention. In the simulated view, there are various modeled objects placed onto the terrain 510, for example trees 512 that are the target of the forest machine, and other objects like rocks 514 and roads 516. The terrain may have shapes e.g. ditches like those next to the road 51 6 and elevation variations like hills. The forest machine itself may comprise two main parts 520 (rear body) and 522 (front body), with the boom 524 attached e.g. to the front part and having a saw head 526. The forest machine may run on wheels 528. The working area indicators 530, 532 and 536 are projected around the forest machine. The working area indicators 530 and 532 show optimal working areas on the side of the machine, and the working area indicator 536 shows a sub-optimal working area in the front. The working area indicators may comprise elements 540 and 542 to indicate the furthest and closest reach of the boom, respectively. The movement track of the forest machine may also be indicated with the working area indicators with track signs 544 for showing the width of the needed track when the forest machine moves forward.
The working area indicators are assigned or otherwise moving with the machine, i.e. at least somewhat attached to the machine coordinate system. For example, the working area indicator center point may be attached to the boom pole center point i.e. the axis of the pole bearing of the boom 524. Further, the orientation of the working area indicators may be aligned with the front body 522 of the harvester machine i.e. the same body where the boom 524 is attached or to the boom pivot point (the bearing housing). In a loader machine, the working area indicators may be aligned with the back body of the machine, since the pole is mounted to the back body in a loader machine. The center point of the working area indicators may be arranged to shift e.g. when the machine is tilted or some other conditions are present. The alignment of the working area indicators may also be done in a flexible manner with reference to multiple elements, like the front body 522 and the rear body 520 together, or the alignment may change with the tilting of the machine, ground shapes and so on. The working area indicators may comprise different radiuses, different shapes, circular cones, elliptic cones, squares, and zones of any kind. Colors can be used to indicate a positive or optimal working area and a negative or sub-optimal working area. The working area indicators or parts of the indicators may be labeled with letters, numbers, words, icons or other labels. For example, squares of the working area indicator may receive labels like "AT, "A2", ... , "A1 2", "ΒΓ, "B2" "L1 1 " and "L1 2". The tasks at hand may then refer to these labels to guide the student. Typically, the boom has a better reach from the side of the machine, and with the help of the working area indicators, this can be now taught at the simulator. The right working methods and work order can be taught with the help of the working area indicators, as well. For a loader machine, the working area indicator may helping to operate the loader in a stable way so that the danger of tipping the machine is smaller.
The working areas may be different for different tasks, e.g. for loading and unloading. There may be a warning displayed if the user tries to reach too far or outside the good working area. The working area indicators may depend on the teaching level, e.g. narrower working areas may be used for advanced students. It is also possible to give points for working on the optimal working area. There may also be different working area indicators, different sizes and shapes to choose from depending on the task. The working area indicator may comprise reminders of the task at hand like text, numbers or icons, e.g. "Pile here" or "Cutting area" or "Loading zone". Such reminders or guides may help e.g. to place the cut trees to the right place by the harvester (to the side or to the front), or they may guide to place the left-over branches to a certain area. The guide may also help to position the harvester or the loading machine correctly, or the guide may indicate an area for loading. For example, the working area indicator may turn to green or may blink when the machine has been positioned correctly according to a given task.
As described earlier, textures may be used in showing the working area. The texture may be attached on a terrain form, or it may be attached to the ground object directly. The terrain form is a floating intermediary object following the ground on top of which the texture is attached. The terrain form is placed upwards in z-direction from the ground or towards the viewer in line of sight. The terrain form may "dive" inside objects on terrain, or it may be shown on top of the objects. The terrain object may follow the ground carefully, as seen in context of the ditch next to the road in Fig. 5.
The working area indicator may help learning in several ways. The working area indicator and instructions regarding the given task may stay on screen for the whole learning session. The working area indicator guides the learning so that the student learns to operate the machine right. The indicator can be switched on and off easily, and it may be applied to many different machines. The indicator helps to visualize the scene where 3-dimensional cues are missing due to the limited display arrangement (2-dimensional display). The indicator improves perspective and gives a better estimation what will be at the reach of the machine. In the simulator, learning to reach to the side is normally slightly difficult since the screen is in the front. The working area indicator helps to guide the student to operate on the side of the machine.
Figs. 6a and 6b show schematic representations of different working area indicators for the forest machines according to an embodiment of the invention. In Fig. 6a, the working areas for the harvester with two main parts 601 and 602 are shown. The reach of the boom 603 may be shown with a cone-shaped indicator 610, 612 (side working areas) and 630 (front working area). The furthest reach of the boom 603 may be shown with arcs 614 and 626, as well as the closest reach with 616 and 622. There may be other indicators for showing an optimal range of the boom such as the area between arcs 618 and the arc 624. The track of the machine for moving forward may be shown with the front working area. There may also be a negative working area shown such as the back working area 640 where the boom cannot or should not reach.
The different working area indicators may have sub-areas, and these sub areas may be labeled. For example, as shown in Fig 6a, the left working area may be divided to sections L1 , L2, L3, L4 and L5, the right working area may be divided to sections R1 , R2, R3, R4 and R5, the front working area may be divided to sections F1 , F2 and F3 and the back working area may be divided to sections B1 , B2 and B3. These labels may then be used in e.g. guiding the task at hand by referring in the instructions or scoring to these labels. An asymmetric working area indicator based on attitude and position of the work machine may also be used.
In Fig. 6b, working areas for a loader comprising two main parts 651 and 652 are shown. These working areas may be cone-shaped as above or reactangular areas 660 and 662 as shown in the figure. The working area indicators may have labeled squares or other tagged locations for guiding the work even more accurately. Tagging can be achieved with numbers, letters, text or color. Sound indicators can also be used. The working area indicators may comprise multiple smaller parts that are then selected to form the whole indicator for a given task. The indicators may also be shaped by hand, e.g. by stretching from an original form or drawing. As described earlier, the different working areas may be subdivided to sections and these sections may be labeled e.g. with labels like "ΑΓ to "L1 2". These labels may then be used in e.g. guiding the task at hand by referring in the instructions or scoring to these labels.
Fig. 7 shows a view of the forest machine in training mode according to an embodiment of the invention. In addition to the visible results of the work like cut trees 71 0, the training mode provides a clear scoring system that can be defined to follow the student's progress. There may be different criteria 720 according to which the performance is judged. These criteria are measured with values 730, and from the values, scores 740 for the criteria are computed. The total score is an indicator of the student's performance. Different performance criteria have been listed in Table 1 .
Table 1 . Performance criteria
Figure imgf000018_0001
The use of working area indicators may provide advantages for learning. For example, a good working method can be trained by the simulator. The trees are removed from the side, not from the front. The trees in the front will come to optimal working area (side) when the machine moves forward. As another example, the needed driving track (width) can be shown, and minimum and maximum reach of the boom, as well as an optimal operating range are shown. The use of working area indicator makes it possible to continue the learning in a simulator for a longer time without the need to move to a real machine, since less "false learning" happens. This may provide cost savings and accelerate learning.
The working area indicator may also help to learn the reach of the boom so that this knowledge is then usable in real world. The working area indicator helps to understand the effect of the boom to machine stability - in a simulator, this is otherwise challenging to understand for a student. Visible information on widths and radiuses helps to learn proper positioning of the harvester machine and proper use of boom in a loader machine.
Fig. 8 shows a view of editing a simulated terrain in the forest machine for the training mode according to an embodiment of the invention. The simulator software may comprise, for example, the following three modes that can be implemented as separate, interconnected programs or different views in a single program, or any combination of the same. The first mode may include editors for the surrounding terrain, trees, roads and other objects in the scenery - it may be called a scenery editor. The scenery editor allows to shape the terrain in terms of surface elevation, and it allows to position different objects onto the terrain (or possibly into the terrain). The different objects may be positioned either automatically or manually, or semi-automatically as a group. The scenery editor may allow loading a pre-created scenery from a file, and it may allow storing a part of the scenery or the whole scenery to a file for later use. The second mode may be a scoring editor, where points for different operations can be defined. The third mode may be the simulation mode, as shown e.g. in Fig. 5.
In Fig. 8, the editor has a terrain 81 0 where different objects can be placed. There may be trees, rocks and other objects 840 on the map. Roads 820 and terrain forms 830 can be created onto the map. There are buttons for adding different objects like groups of trees 842, electric lines 844, surface forms 846 and roads 848. The scenery can be created manually e.g. tree-by-tree or semi-automatically e.g. as groups of trees filling a certain area and having a certain density characteristics. The benefit of such a terrain editor is that teaching scenes can be created beforehand and stored for later use. Since the forest exists in the computer, it can be grown back in a few seconds after being cut. The computerized teaching material therefore stays up to date and can be easily reused. Figs 9a and 9b show two types of forest machines whose operation can be modeled in the simulator according to an embodiment of the invention. In Fig. 9a, a harvester is shown, comprising two main parts, the engine part 910 and the cabin part, connected with a swivel 925. The harvester may have wheels 930 to run on the terrain, or other means of supporting it on the ground. The harvester has a boom with a saw head for cutting the trees and delimbing them. In Fig. 9b, a loader machine for transporting the trunks from the forest is shown. The loader machine has two parts, the cabin part 950 and the cage part 960. The loader machine has a boom and a grabber head attached to the boom for collecting the trunks to the cage 980. The loader machine may also run on wheels 990. Other types of machines are also possible in the simulator.
Fig. 10 shows an example of displaying a working area of a work machine on a display inside the working machine. The display is arranged to show to the operator of the work machine information on the work machine status 1030 as well as information on the task at hand 1040, 1 050 and 1055. For example, the display may show the engine and battery status of a forest machine or a harvester, and it may indicate to the operator the dimensions of the tree being cut and the kind of tree being cut (e.g. here "KUUSI" or "SPRUCE"). In order to help operating the machine, the display may be adapted to show the working area indicators on the screen. There may also be information like the operator's name (here "Pekka"), and the machine may record the produced output related to the person operating the machine, allowing multiple persons to successively operate the machine.
The display may show the position and attitude of the machine 1010, as well as the direction, length and movement of the boom 1 095. The left working area 1025, the right working area 1020 as well as the front working area 1022 may be indicated, with the respective sub-areas and the sub-area labels. Showing the working area indicators on the screen may help the operator to operate the machine optimally. The terrain may be shown in the work area indicator on the screen with the help of camera or scanner means, e.g. attached to the boom. The working area indicator may also be shown on the real controller screen in a simulator, in addition to showing the working area indicators in the simulated world.
The working area indicator may also be brought to the screen at times when the boom is being moved by the operator, and removed from screen at other times. This helps to use the display space optimally and allows the operator to observe only the most important matters on the screen. For example, when the tree is being cut, and the boom is not moving, the operator may appreciate to see the data related to the cutting. Therefore, at times when the tree is being cut, indicators like the length of the trunk and volume of prepared logs may be shown. At times of other operations, other data may be shown, and the working area indicator may be removed from the screen or minimized to occupu less screen area. When the operator is moving the boom, the working area indicator may be brought onto the screen, or it may be enlarged from a small indicator to a larger indicator to be better visible, or the working area indicators may fill essentially the entire screen. Showing the boom movement and position may require sensors on the boom for sensing the direction, length (extension) and movement of the boom.
Fig. 1 1 shows an example of using a head-up display for indicating the working area of a work machine in the real world. The operator of the work machine may have been trained in a simulator by showing the working area indicators in the simulated world, and possibly at the same time on the real controller screen as explained with reference to Fig. 1 0. The real work machine, like a forest machine or a harvester or a loader machine, may be fitted with a head-up display for showing the working area indicators familiar from the training indicator. The real work machine has a cabin with windows 1 1 1 0 (right window), 1 1 20 (left window) and 1 1 30 (front window). The work machine may have the same controls 1 1 40, 1 1 45, and 1 1 50 as in the simulator, and the control screen 1 1 45 may show the same working area indicators as in the simulator. To even further improve the usefulness of the working area indicators, they may be shown projected onto the windows with the help of head-up displays. There may be multiple head-up displays, for example three or four or more, or just one. In an exemplary setup, the right working area indicator 1 1 18 is projected onto the right window 1 1 1 0 with the help of a head-up display 1 1 15. The left working area indicator 1 128 is projected onto the left window 1 120 with the help of a head-up display 1 125. The front working area indicator 1 138 is projected onto the front window 1 130 with the help of a head-up display 1 135. This way, the working area indicators may be seen by the operator as if they were on top of the terrain. The head-up displays may be adapted to take into account the height of the operator so that the working area indicators are seen at the right place on top of the terrain. The head-up displays may also be able to adapt to the movement of the operator's head, and they may adapt to the tilt of the machine by changing the projection onto the window.
Other arrangements for projecting the working area indicators onto the terrain in the visual field of the operator may be used. Basically, any means that are able to form an image of the working area indicators so that it appears to be on top of the terrain may be used. Such arrangements may be a special helmet that follows head movements, transparent video goggles that detect how the operator is oriented, or even ray projection arrangements may be used. The image may be formed on any surface, even directly on the surface of the eye's retina. The various embodiments of the invention can be implemented with the help of computer program code that resides in a memory and causes the relevant apparatuses to carry out the invention. For example, a terminal device may comprise circuitry and electronics for handling, receiving and transmitting data, computer program code in a memory, and a processor that, when running the computer program code, causes the terminal device to carry out the features of an embodiment. Yet further, a network device may comprise circuitry and electronics for handling, receiving and transmitting data, computer program code in a memory, and a processor that, when running the computer program code, causes the network device to carry out the features of an embodiment. It is obvious that the present invention is not limited solely to the above- presented embodiments, but it can be modified within the scope of the appended claims.

Claims

Claims:
1 . A method for displaying a working area of a boom pivotally mounted on a work machine at an apparatus for controlling the boom and/or the work machine, the method comprising:
- electronically determining a dimension of at least part of the working area of the work machine, and
- electronically indicating the working area of the forest machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the work machine.
2. A method according to claim 1 , wherein the apparatus for controlling a forest machine is a forest work machine simulator, the method further comprising:
- modeling the forest work machine by a computer so that at least one part of the forest machine is a computer-modeled part, and
- displaying movement of the computer-modeled part of the forest work machine to a user of the apparatus.
3. A method according to claim 1 or 2, further comprising:
- electronically modeling the terrain around the work machine with a terrain form, and
- electronically indicating the working area of the work machine on top of the terrain form, and
- electronically indicating the working area with the help of a texture.
4. A method according to claim 1 , 2 or 3, further comprising:
- aligning the orientation of the indication of the working area to one of the orientation of the whole machine, the orientation of a part of the machine, the orientation of the cabin or to the orientation of the boom pivot.
5. A method according to any of the claims 1 to 4, further comprising:
- determining information on the forestry task being performed by the user, and
- electronically indicating the working area of the forest machine based on said information on the forestry task.
6. A method according to any of the claims 1 to 5, wherein said indication of the working area comprises at least one of the group of indicator for movement track of the machine, indicator for closest reach of the boom, indicator for furthest reach of the boom, positive working area indicator, negative working area indicator, a working cone indicator, a working square indicator, and an asymmetric working area indicator based on attitude and position of the work machine.
7. A method according to any of the claims 1 to 6, wherein the control means are configured to be used with a simulated machine or a real- world machine.
8. An apparatus comprising a processor, memory including computer program code, the memory and the computer program code configured to, with the processor, cause the apparatus to perform at least the following:
- electronically determining a dimension of at least part of the working area of a boom pivotally mounted on a work machine, and
- electronically indicating the working area of the work machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the work machine.
9. An apparatus according to claim 8, further comprising computer program code configured to, with the processor, cause the apparatus to perform at least the following:
- modeling the work machine by a computer so that at least one part of the work machine is a computer-modeled part, and
- displaying movement of the computer-modeled part of the work machine to a user of the apparatus.
1 0. An apparatus according to claim 8 or 9, further comprising computer program code configured to, with the processor, cause the apparatus to perform at least the following:
- electronically modeling the terrain around the machine with a terrain form, and - electronically indicating the working area of the machine on top of the terrain form, and
- electronically indicating the working area with the help of a texture.
1 1 . An apparatus according to claim 8, 9 or 1 0, further comprising computer program code configured to, with the processor, cause the apparatus to perform at least the following:
- aligning the orientation of the indication of the working area to one of the orientation of the whole machine, the orientation of a part of the machine, the orientation of the cabin or to the orientation of the boom pivot.
1 2. An apparatus according to any of the claims 8 to 1 1 , further comprising computer program code configured to, with the processor, cause the apparatus to perform at least the following:
- determining information on the forestry task being performed by the user, and
- electronically indicating the working area of the forest machine based on said information on the forestry task.
1 3. An apparatus according to any of the claims 8 to 1 2, wherein said indication of the working area comprises at least one of the group of indicator for movement track of the machine, indicator for closest reach of working means, indicator for furthest reach of working means, positive working area indicator, negative working area indicator, a working cone indicator and a working square indicator, and an asymmetric working area indicator based on attitude and position of the work machine.
1 4. An apparatus according to any of the claims 8 to 1 3, wherein the control means are configured to be used with a simulated machine or a real-world machine.
15. An apparatus comprising a processor, memory including computer program code, the memory and the computer program code configured to, with the processor, cause the apparatus to perform the method according to any of the claims 1 to 7.
1 6. A computer program product stored on a computer readable medium and executable in a data processing device, wherein the computer program product comprises
- a computer program code section for determining a dimension of at least part of the working area of a boom pivotally mounted on a work machine, and
- a computer program code section for indicating the working area of the work machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the work machine.
1 7. A computer program product stored on a computer readable medium and executable in a data processing device, wherein the computer program product comprises computer program code sections for carrying out the method according to any of the claims 1 to 7.
1 8. An apparatus comprising :
- control means of a boom pivotally mounted on a work machine for receiving control input from a user and producing control signals for controlling the apparatus,
- means for processing said control signals from the control unit,
- means for determining a dimension of at least part of the working area of the forest machine, and
- means for indicating the working area of the work machine so that the indication of the working area is at least partly visible on top of a terrain adjacent to the forest machine.
1 9. An apparatus according to claim 1 8, further comprising:
- means for modeling the forest machine by a computer so that at least one part of the work machine is a computer-modeled part, and
- means for displaying movement of the computer-modeled part of the work machine to a user of the apparatus.
20. A system comprising at least a first apparatus and a second apparatus according to any of the claims 8 to 1 5, wherein the first apparatus and the second apparatus are connected with a data connection, and the first apparatus and the second apparatus are configured to show information indicative of the operation of the second apparatus at the first apparatus.
21 . A system according to claim 20, wherein the first apparatus is configured to display the graphical form of the second apparatus in the virtual scenery of the first apparatus.
22. A method according to any of the claims 1 to 7, wherein the work machine is a real-world work machine such as a forestry machine or a loader having a mechanical boom.
23. A method according to claim 22, further comprising indicating the working area using a head-up display.
24. An apparatus according to any of the claims 8 to 1 5, wherein the work machine is a real-world work machine such as a forestry machine or a loader having a mechanical boom.
25. An apparatus according to claim 24, further comprising a head-up display for indicating the working area.
26. A work machine comprising a movable boom, a display, a processor, memory including computer program code, the memory and the computer program code configured to, with the processor, cause the work machine to perform at least the following :
- electronically indicating a working area of the boom on the display.
27. A work machine according to claim 26, the work machine is configured to indicate the working area in response to operating the boom, and to remove the working area indicator when the boom is not being operated.
28. A work machine according to claim 26 or 27, further comprising a head-up display for indicating the working area.
PCT/FI2009/050975 2009-12-03 2009-12-03 A method and a system for a work machine with a boom WO2011067456A1 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
SE1250574A SE1250574A1 (en) 2009-12-03 2009-12-03 Procedure and system for a working machine with a boom
BR112012013204A BR112012013204A2 (en) 2009-12-03 2009-12-03 method for displaying a desktop, device, computer program product stored in a computer readable medium and executable on a data processing device, system and forest machine
PCT/FI2009/050975 WO2011067456A1 (en) 2009-12-03 2009-12-03 A method and a system for a work machine with a boom
RU2012127569/11A RU2524737C2 (en) 2009-12-03 2009-12-03 Method and system for working machine with boom
FI20125702A FI129431B (en) 2009-12-03 2009-12-03 Method, apparatus and system for a simulated forest machine with a boom

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/FI2009/050975 WO2011067456A1 (en) 2009-12-03 2009-12-03 A method and a system for a work machine with a boom

Publications (1)

Publication Number Publication Date
WO2011067456A1 true WO2011067456A1 (en) 2011-06-09

Family

ID=44114641

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/FI2009/050975 WO2011067456A1 (en) 2009-12-03 2009-12-03 A method and a system for a work machine with a boom

Country Status (5)

Country Link
BR (1) BR112012013204A2 (en)
FI (1) FI129431B (en)
RU (1) RU2524737C2 (en)
SE (1) SE1250574A1 (en)
WO (1) WO2011067456A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015042155A1 (en) * 2013-09-17 2015-03-26 Caterpillar Inc. Training apparatus
EP2799630A4 (en) * 2011-12-26 2015-07-29 Sumitomo Heavy Industries Image display device for backhoe
WO2020043948A1 (en) * 2018-08-30 2020-03-05 Ponsse Oyj Method and arrangement for assisting control of forest work machine

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102016000353A1 (en) * 2016-01-14 2017-07-20 Liebherr-Components Biberach Gmbh Crane, construction machine or industrial truck simulator

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH01312129A (en) * 1988-06-13 1989-12-15 Komatsu Ltd Monitoring device for arm-type work machine
US5404661A (en) * 1994-05-10 1995-04-11 Caterpillar Inc. Method and apparatus for determining the location of a work implement
US5646844A (en) * 1994-04-18 1997-07-08 Caterpillar Inc. Method and apparatus for real-time monitoring and coordination of multiple geography altering machines on a work site
US5751576A (en) * 1995-12-18 1998-05-12 Ag-Chem Equipment Co., Inc. Animated map display method for computer-controlled agricultural product application equipment
US20050027420A1 (en) * 2002-09-17 2005-02-03 Kazuo Fujishima Excavation teaching apparatus for construction machine
GB2414010A (en) * 2004-05-11 2005-11-16 Bamford Excavators Ltd Head-up display for excavator
US20080180523A1 (en) * 2007-01-31 2008-07-31 Stratton Kenneth L Simulation system implementing real-time machine data
US20090219199A1 (en) * 2008-02-29 2009-09-03 Caterpillar Inc. Positioning system for projecting a site model

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH01312129A (en) * 1988-06-13 1989-12-15 Komatsu Ltd Monitoring device for arm-type work machine
US5646844A (en) * 1994-04-18 1997-07-08 Caterpillar Inc. Method and apparatus for real-time monitoring and coordination of multiple geography altering machines on a work site
US5404661A (en) * 1994-05-10 1995-04-11 Caterpillar Inc. Method and apparatus for determining the location of a work implement
US5751576A (en) * 1995-12-18 1998-05-12 Ag-Chem Equipment Co., Inc. Animated map display method for computer-controlled agricultural product application equipment
US20050027420A1 (en) * 2002-09-17 2005-02-03 Kazuo Fujishima Excavation teaching apparatus for construction machine
GB2414010A (en) * 2004-05-11 2005-11-16 Bamford Excavators Ltd Head-up display for excavator
US20080180523A1 (en) * 2007-01-31 2008-07-31 Stratton Kenneth L Simulation system implementing real-time machine data
US20090219199A1 (en) * 2008-02-29 2009-09-03 Caterpillar Inc. Positioning system for projecting a site model

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BURMAN L. ET AL: "Human-machine interaction improvements of forest machines", 39TH NORDIC ERGONOMICS SOCIETY CONFERENCE (NES2007), 1 October 2007 (2007-10-01) - 3 October 2007 (2007-10-03), LYSEKIL, SWEDEN, Retrieved from the Internet <URL:http://www.nordiskergonomi.org/nes2007/CD_NES_2007/papers/A26_Burman.pdf> [retrieved on 20100917] *
SIMLOG: "Simlog Personal Simulators", PRODUCT PRESENTATION, 17 March 2006 (2006-03-17), Retrieved from the Internet <URL:http://web.archive.org/web/20080701212659/www.simlog.com/personal-excavator.html> [retrieved on 20100917] *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2799630A4 (en) * 2011-12-26 2015-07-29 Sumitomo Heavy Industries Image display device for backhoe
US9909283B2 (en) 2011-12-26 2018-03-06 Sumitomo Heavy Industries, Ltd. Image display apparatus for shovel
US11072911B2 (en) 2011-12-26 2021-07-27 Sumitomo Heavy Industries, Ltd. Image display apparatus for shovel
WO2015042155A1 (en) * 2013-09-17 2015-03-26 Caterpillar Inc. Training apparatus
US10056009B2 (en) 2013-09-17 2018-08-21 Caterpillar Inc. Training apparatus
WO2020043948A1 (en) * 2018-08-30 2020-03-05 Ponsse Oyj Method and arrangement for assisting control of forest work machine
US11357157B2 (en) 2018-08-30 2022-06-14 Ponsse Oyj Method and arrangement for assisting control of forest work machine

Also Published As

Publication number Publication date
BR112012013204A2 (en) 2016-03-01
SE1250574A1 (en) 2012-07-03
FI129431B (en) 2022-02-15
RU2012127569A (en) 2014-01-10
FI20125702A (en) 2012-06-25
RU2524737C2 (en) 2014-08-10

Similar Documents

Publication Publication Date Title
US10854098B1 (en) Adaptive visual overlay wound simulation
Craig et al. Developing virtual reality applications: Foundations of effective design
CN108701429B (en) Method, system, and storage medium for training a user of a robotic surgical system
US8915740B2 (en) Virtual reality pipe welding simulator
US20100156906A1 (en) Shot generation from previsualization of a physical environment
US11861062B2 (en) Blink-based calibration of an optical see-through head-mounted display
EP1257996B1 (en) Method and apparatus for displaying musical data in a three dimensional environment
ES2705059T3 (en) GTAW virtual reality solder and pipe solder configuration and configuration
JP4288449B2 (en) Image display device, image processing device, and image display system
US11514673B2 (en) Systems and methods for augmented reality
Linowes Unity 2020 virtual reality projects: Learn VR development by building immersive applications and games with Unity 2019.4 and later versions
Linowes Unity virtual reality projects: Learn virtual reality by developing more than 10 engaging projects with unity 2018
US20100190556A1 (en) Information storage medium, game program, and game system
WO2011067456A1 (en) A method and a system for a work machine with a boom
Glover et al. Complete Virtual Reality and Augmented Reality Development with Unity: Leverage the power of Unity and become a pro at creating mixed reality applications
WO1996016389A1 (en) Medical procedure simulator
US20210375025A1 (en) Systems and methods performing object occlusion in augmented reality-based assembly instructions
Wang et al. Reflections on using a game engine to develop a virtual training system for construction excavator operators
Geig Unity Game Development in 24 Hours, Sams Teach Yourself
Shindev et al. Exploration of intention expression for robots
KR101525414B1 (en) Image processing device for visually realizing topographic information of green
Falomir et al. Q3D-Game: A Tool for Training Users' 3D Spatial Skills
JP6982207B1 (en) Magnetic field visualization system
JPH11328437A (en) Game machine and image processing method of game machine
Pereira Jr Development of a Harvester Machine Simulator in Virtual Reality

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09851809

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 20125702

Country of ref document: FI

WWE Wipo information: entry into national phase

Ref document number: 2012127569

Country of ref document: RU

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112012013204

Country of ref document: BR

122 Ep: pct application non-entry in european phase

Ref document number: 09851809

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 112012013204

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20120531