WO1999056241A1

WO1999056241A1 - Method and apparatus for producing a signal from an image

Info

Publication number: WO1999056241A1
Application number: PCT/GB1999/001259
Authority: WO
Inventors: Charles Cooke
Original assignee: The Royal College Of Art
Priority date: 1998-04-23
Filing date: 1999-04-23
Publication date: 1999-11-04
Also published as: GB9808651D0; AU3619799A

Abstract

The invention relates to a method and apparatus for producing a signal from an image (10). The image is formed from an array of symbols (12). Each symbol (12) represents a number of bits, and so information such as audio signals can be decoded from the image (10). The symbols (12) are designed so that they may be incorporated into pictures in children's books. These pictures may be scanned and sounds encoded by the symbols (12) are played via a loudspeaker or a computer. Picture books incorporating such images are useful in teaching children skills such as reading, counting, and identifying colours, shapes and other objects.

Description

1

METHOD AND APPARATUS FOR PRODUCING A SIGNAL FROM AN IMAGE

The present invention relates to a method and apparatus for producing a signal from an image.

Most children start learning to read at about the age of four. Nevertheless, picture books, pop-up books, interactive books, and even CD ROMs can be used as educational aids well before this age. However, all of these require adult supervision and intervention. What is required is a form of book whereby very young children who are not yet able to read can acquire skills such as counting, identifying shapes and colours, and learning vocabulary. Books having bar codes printed next to pictures have been developed, so that by scanning the bar code additional information, such as sounds, are presented to the child. These books have not proved popular as they can be difficult for young children to use unsupervised.

An educational aid is disclosed in UK Patent Application No. GB 2273388 (A. Szmidla). Codes which may be read by a scanner are contained in documents. The code is associated with an object in the document, so that when the code is scanned, the name of the object is sounded. In order to associate the code with a particular object, and so that the user is not confused as to which code relates to which object, the code is in the form of a series of dark and light lines arranged as a series of rings which enclose the object. The disadvantage with this system is that the series of rings is not pleasing to the eye.

In addition to the aforedescribed code, there are other types of code which may be used. One example is the "Snowflake" code, developed by Electronic Automation Limited, and is shown in Figure la. "Snowflakes" can be read in any orientation using a scanner called a "Snow Gun" which is plugged directly into a PC. The "Snow Gun" takes about one second to read and decode each "Snowflake". These codes are designed to hold a standard amount of bar code data in a very small area. As a result, a single "Snowflake" can only hold about 0.2 seconds of sound, although it can be printed very small. A further example is the "DataGlyph" technology developed by Xerox. A mass of tiny lines (known as glyphs) whose orientation is varied to represent digital information may be printed onto books, goods etc. Each glyph element consists of a small 45 degree diagonal line - as short as l/100^th of an inch or less, depending on the resolution of the printing and scanning that is used. Each glyph represents a single binary 0 or 1, depending on whether it slopes to the left or right. Sequences of these can be used to encode numeric, textural or other information. The system is specifically designed to store large quantities of data on paper that also carries text and graphics. "DataGlyphs" appear as an area of grey on the page, and an example is shown in Figure lb.

A 2D bar code is shown in Figure lc. This bar code is based on the PDF417 2D bar code system devised by Symbol Technologies Limited. A single PDF417 bar code can contain anything up to 1108 bytes of data. This equates to just over 11 seconds of sound if the latest sound compression algorithms which are currently used in voice mail systems, are employed.

A disadvantage of all of these codes, and of standard ID bar codes, is that they are very difficult to integrate into, and associate with, images. This is especially the case where the images are to be used as educational aids for young children.

An aim of the present invention is to provide a method of producing signals, particularly audio signals, from images. Another aim of the invention is to provide a method of encoding signals in images.

According to a first aspect of the invention there is provided a method of producing a signal from an image, the image including a plurality of symbols which contain encoded information, the method comprising the steps of: reading the symbols, recognising the symbols, and decoding the information encoded by the symbols to produce a signal.

The signal may then be communicated to a user.

Preferably the symbols are read using a scanner. 3 The symbols may be arranged in a two dimensional array. The symbols may represent digital data. For example, each symbol may represent 1, 2, 3 or more bits. There may be

4 different symbols, representing 4 2-bit numbers, or there may be 8 different symbols, representing 8 3-bit numbers. Preferably the symbols are machine readable.

A symbol may be a polygon. A symbol may also be an area defined by a circle, or part of the area defined by a circle. A symbol may be an area defined by a semi-circle. A symbol may be a regular or an irregular polygon. The polygon may be concave or convex. A symbol may be part of the area defined by an ellipse.

Preferably one or more symbols has no centre, point or axis of symmetry. As well as containing encoded information, the symbol may thus be used as an orientation symbol. This ensures that the direction from which the symbols are scanned may be derived. For example, a symbol may be the area defined by a circle minus the lower right quadrant. If the symbol is rotated 90 degrees clockwise, it will be seen as the area defined by a circle minus the lower left quadrant. Using processing hardware or software, it is then possible to deduce that this symbol, and hence the image, has been read from right to left, rather than from left to right. Alternatively, one or more symbols indicating the correct orientation of the image may be included in the image. For example, symbols such as arrows may be included in the image to indicate in which order the symbols should be interpreted to form the data signal.

The image may contain embedded synchronization information. That is, a repeating fixed pattern of symbols which mark the boundaries of the coded area of the image, the pattern serving as a clocking track to improve the reliability of reading the coded image area.

Preferably the signal is an audio signal. It may, however, be a signal representing an address of, for example, a location in a look-up table. The signal may also be an electrical signal, or an image signal.

If the signal is an audio signal, attributes of a sound may be assigned to control data parameters. The symbols may then represent the control data parameters. These control data parameters may, for example, be the length of an audio clip, the number of times 4 the clip is to be played, the volume of the clip, etc. The symbols may also be used to encode information as to the identity of the audio clip to be played. Digital information encoded in the symbols may represent the address of a data storage site where a digitised sound track is archived. The address may include a world wide web address, or an LP (Internet Protocol) address.

Sounds may also be generated by a synthesizer, and therefore the control parameters could be attack time, decay time, sustain, sound envelope curve, or percussion information. Alternatively, primitive sound signals or simple tunes may be encoded by the symbols. For example, an individual symbol may represent a musical note played at a pre-set volume.

The symbols may be used to encode compressed signals. The compression may be carried out using, for example, the TVC-3 algorithm developed by Cybernetics Infotech Inc. in the USA. This compression algorithm is used in voice mail systems, and one second of digital sound can be compressed to about 800 bits.

The symbols may be used to encode wav file data, or data in the MIDI file format. Alternatively, each symbol may represent a phoneme. A phoneme is the smallest unit in the sound system of a language, capable of signalling a difference of meaning between words. For example, the English words pail and tail are distinguished by the initial consonant phonemes /p/ and /t/. The phonemes can be synthesized to produce words.

Control information may also be represented by the symbols. This control information may communicate instructions to the scanner to affect its operation - controlling, for example, the volume or tone of a sound.

According to a second aspect of the invention there is provided an apparatus for producing a signal from an image, the image including a plurality of symbols, the apparatus including: reading means for reading a symbol, symbol recognition means for recognising a symbol, and decoding means for converting the information encoded in the symbol into a signal. 5 The apparatus may also include communication means for communicating the decoded signal to a user, or to an electronic device. The communication means may be a transducer, such as a loudspeaker, a transceiver, or a transmitter.

The apparatus may be disposed within a housing. The housing is preferably made of a tough durable plastic material such as ABS. Alternatively, it may be made of polypropylene.

Preferably the signal is an audio signal. The signal may, however, be an image signal, an address in a look-up table, or other electrical signal.

The reading means may be a still camera, a video camera, a scanner, or any other device that converts optical information into electronic information. Most preferably the scanner includes a plurality of light emitting diodes which produce a beam of light. The light emitting diodes may be arranged in a one-dimensional or two-dimensional array. The scanner may also be a laser scanner.

The symbol recognition means may be an integrated circuit, or, more particularly, a dedicated image processing microchip. The symbol recognition means may perform functions such as thresholding, matching, and other image processing functions in order to recognise a symbol. Each symbol, when read, may be matched with a stored representation, or template, of all the symbols in order to recognise the symbol.

The detecting means may be a symbol-to-bit converter, such that once recognised, the symbol may be associated with a value, such as a 2-bit or 3-bit number. A sequence of these numbers thus forms a digital signal.

Once decoded from the symbols, the signal may be passed from the apparatus to an electronic system, or a storage device, via a cable or a lead. Alternatively, the signals may be transmitted to an electronic system (or storage device) using wireless means such as infra-red or radio waves. The electronic system may be a computer, a hi-fi, a synthesizer, microprocessor, etc. Raw data from the symbols may be transmitted from the apparatus, or the data may be processed via on-board hardware or software before being transmitted. 6 For example, for data in the MIDI file format, instructions encoded from the symbols may be passed to a MIDI device. The MIDI device then uses the instructions to play the required sound. The MLDI device may be a chip located in the apparatus, or may be located remote from the apparatus.

The apparatus may have a number of pre-loaded samples stored in a storage device, such as RAM. Information decoded from the symbols may then be used to select the sample to be played from memory.

The apparatus preferably includes a power supply such as a battery.

The apparatus may be in the form of a children's toy. The toy may be a hand-held, easy- to-use object that contains a laser scanner and a loudspeaker. It may be designed to decode digital sound files from an image and play them to a listener via the loudspeaker. The digital sound files may be compressed or uncompressed files.

The toy may have a number of wheels, so that it may be rolled over an area containing an image. The toy may have a switch, preferably a rotary switch, for switching it on. The switch may also function as a volume adjuster. A rotation sensor may be mounted in a wheel so that the scanner is activated when the toy is rolled along. The rotation sensor is preferably magnetic.

The apparatus may also include a microphone. The microphone may be used to record soundtracks. These soundtracks may be digitised and the resulting sequence of bits may be converted into a sequence of symbols to be printed out by, for example, a printer. The printed soundtracks may appear purely as a set of symbols with no particular meaning for the human viewer, or the symbols may printed in the form of an image.

Soundtracks recorded by the microphone may be associated with a picture taken by a digital camera and stored in the onboard storage device, or in a remote data storage site. The apparatus may also include information processing means. The information processor means recognises that the digital camera has taken the same picture again, and the soundtrack is played back through the loudspeaker. 7 Soundtracks recorded by the microphone may be stored in a remote data storage site.

Digital information relating to the address of the remote data storage site may be represented by the symbols and may be printed out by a printer.

According to another aspect of the invention there is provided a method of encoding a signal in an image, the method comprising the steps of: acquiring a signal, associating a symbol with at least part of the signal, and arranging a plurality of symbols to form an image.

The signal may be a digital signal, or an analogue signal. If the signal is an analogue signal, it is preferably converted to a digital signal using an analogue-to-digital converter. The signal may be acquired by a transceiver, or other suitable means such as a computer, a digital camera, or a microphone.

The digital signal may encode audio, visual, or other information. The signal may be a compressed signal. The signal may be decompressed before it is associated with a symbol, or compressed signals may be encoded by the symbols.

A symbol may be associated with a number of bits, as described previously. The digital signal may then be encoded using a plurality of symbols.

The symbols are preferably arranged in the form of an array. Preferably the shape of the symbols is as previously described.

The symbols may be printed onto a dedicated area in which no other text or image is present, or the symbols may be part of a larger image, or area of text. The symbols may be printed using ultra-violet ink, or ink containing a magnetic material, so that they may share an area with other printed text and/or images and yet be invisible to the human eye. The symbols may be incorporated into, for example, black and white, greyscale, or colour images.

Symbols may be printed in half-tone (small black dots whose size is varied to vary tone). The image may be a colour image, in which case the symbols may be printed in CMYK half-tone (small black dots whose size is varied to vary tone, and dots printed in a similar fashion in cyan, magenta and yellow). 8 Alternatively, the symbols may be stored in electronic form for further use.

An embodiment of the invention will now be described, by way of example only, with reference to the accompanying Figures, in which: -

Figures la, lb and lc show bar codes, according to the prior art;

Figure 2 shows an image containing encoded information;

Figures 3a and 3b are tables showing symbols and their corresponding values;

Figure 4a shows a side view of a scanner device in the form of a children's toy;

Figure 4b shows the top view of the scanner device of Figure 4a;

Figure 4c shows a perspective view of the scanner device of Figure 4a; and

Figure 5 shows a block diagram of the operation of the scanner device.

Referring now to Figure 2, there is shown an image (10) of a fish. The image is composed of a 7 by 5 array of thirty five symbols (12). The table of Figure 3a shows these symbols (12) associated with sequences of bits. The right hand column of the table shows every possible sequence of two bits, that is, 00, 01, 10, and 00. Thus, the fish image (10) may be scanned, and seventy bits recovered from the thirty five symbols (12).

In addition to the symbols shown in the table of Figure 3a, further symbols (12) may be used so that each symbol (12) represents three bits. This is shown in Figure 3b. The right hand column of the table contains every possible arrangement of three bits. The left hand column contains eight symbols (12) that are recognisable to a scanner. Each symbol represents 3 bits because there are 8 symbols to choose from (2³=8). If there were 16 symbols to choose from, each symbol would represent 4 bits, 32 symbols, 5 bits per symbol, and so on.

This system is better suited to carrying images than the bar codes described in the prior art as the symbols can be included in images. The symbols are especially designed for 9 inclusion in images as the symbols fill a large amount of image space, leaving only a small proportion of the image blank.

A scanner (20) in the form of a children's toy is shown in Figure 4. The toy (20) has a body (22) containing electronics (not shown) (24), and a loudspeaker (26) attached thereto. The electronics includes a detecting element, a symbol-to-bit converter (48), a symbol recognition element (46), an information processor (50), and memory (56).

The body (22) is made from ABS. It is injection moulded. Body (22) parts are fixed together using screws and snap fits. Two legs (30a,b) extend from the side of the body, and a wheel (32a, b) is attached to each leg. A handle (34) is fixed to the loudspeaker section, so that the scanner (20) may be held and wheeled across the, area to be scanned. The wheeled unit fixes the height of the scanner at the optimum level for reading the symbols. It also allows children to nudge the scanner into place and so alleviates problems caused by children's tight grip and jerky movements. The wheels (32a, b) have a large diameter so that the scanner can be moved reasonably freely over obstacles such as the edge of a book. The wheels also discourage sideways movement and encourage a forwards and backwards motion. This motion is required for the scanner (20) to work efficiently.

A scan engine (28) is disposed within scanner (20) and includes a diode (36) (not shown), a mirror (38), a motor (40) (not shown), and collimating lenses (42) (not shown). A beam of light (44) is generated by the diode (36), and the beam (44) passes out of a window (60) located on the same side of the body as the legs, as shown in Figure 4c. The beam is moved from left to right, or up and down, by mirror (38) which is moved by motor (40).

Scanner (20) as shown in Figure 4 was designed for children, and it is very easy to use. The beam (44) is activated by a movement detector within the scanner. When the scanning beam (44) is activated, it passes between the two wheels (32a,b). The scanning beam (44) remains activated for 6 seconds after the last detected movement of the scanner. Moving the scanner (20) over the coded image (10) so that the code passes between the two wheels (32a,b) results in a successful scan. Successful scans are 10 followed by a beep and a short pause (less than a second) before the sound track is played back in full through the on-board loudspeaker (26).

The operation of the scanner will now be described with reference to the block diagram shown in Figure 5. The detector element (not shown) of scanner (20) receives the reflected beam (44) from the area that is being scanned, and this information is passed to a symbol recognition element (46). This operates in a similar way to conventional optical character recognition system, except that it is designed to operate with specially formulated symbols rather than text. The symbols, once recognised, are then converted to digital information by the symbol to bit converter (48). This digital information may be processed by an on-board processing element (50), or it may be passed to a remote processor (52) via a transceiver (54). Alternatively, the digital information may be passed to on-board memory (56), or to a printer. The digital information in the printed symbols (12) is, in this example, a digitised soundtrack. The soundtrack is then played through the loudspeaker (26).

Although the invention has been described by one embodiment only, variation may be made to the aforementioned embodiment without departing from the scope of the invention. For example, symbols representing encoded information can be printed on toys such as cars and tractors, on cereal packets and on clothes. There are 148 000 registered blind or partially sighted in the UK and many more who experience difficulty reading because of deteriorating vision. Publications including the invention will literally be able to read themselves out loud to these people. Images incorporating the symbols shown herein can be printed on standard presses which makes it cheap and opens up the possibility of printed sound gas bills, phone bills, timetables, packaging, books, newspapers and magazines.

Claims

11Claims

1. A method of producing a signal from an image, the image including a plurality of symbols which contain encoded information, the method comprising the steps of: reading the symbols, recognising the symbols, and decoding the information encoded by the symbols to produce a signal.

2. A method of encoding a signal in an image, the method comprising the steps of: acquiring a signal, associating a symbol with at least part of the signal, and arranging a plurality of symbols to form an image.

3. A method according to claims 1 and 2 wherein the signal is an audio signal.

4. A method according to claims 1 and 2 wherein the signal is an address in a look-up table.

5. A method according to any preceding claim wherein the signal is a compressed signal.

6. A method according to any preceding claim wherein the symbols represent digital data.

7. A method according to claim 6 wherein a symbol represents a 2-bit number.

8. A method according to claim 6 wherein a symbol represents a 3-bit number.

9. A method according to any preceding claim wherein the symbols are arranged in a two dimensional array.

10. A method according to any preceding claim wherein the symbols are machine readable.

11. A method according to any preceding claim wherein a symbol is a polygon.

12. A method according to claim 11 wherein the polygon is concave. 12

13. A method according to claim 11 wherein the polygon is convex.

14. A method according to any of claims 11 to 13 wherein the polygon is an irregular polygon.

15. A method according to any of claims 11 to 13 wherein the polygon is a regular polygon.

16. A method according to claim 11 wherein a symbol is at least part of the area defined by an ellipse.

17. A method according to claim 16 wherein the ellipse is a circle.

18. A method according to any of claims 2 to 17 wherein the symbols are printed onto a surface.

19. A method according to claim 18 wherein the symbols are printed using ultra-violet ink.

20. A method according to claim 19 wherein the symbols are printed using ink which includes magnetic material.

21. An image produced by the method of any of claims 2 to 20.

22. A book including the image of claim 21.

23. Packaging including the image of claim 21.

24. Clothing including the image of claim 21.

25. Apparatus for producing a signal from an image, the image including a plurality of symbols, the apparatus including: reading means for reading a symbol, symbol recognition means for recognising a symbol, and decoding means for converting the information encoded in the symbol into a signal.

26. Apparatus according to claim 25 wherein the reading means is a scanner. 13

27. Apparatus according to claim 26 wherein the scanner includes a plurality of light emitting diodes.

28. Apparatus according to claim 25 wherein the symbol recognition means is an integrated circuit.

29. Apparatus according to claim 25 wherein the decoding means is a symbol-to-bit converter.

30. Apparatus according to any of claims 25 to 29 further including a loudspeaker.

31. Apparatus according to any of claims 25 to 30 further including a transceiver.

32. A method of producing a signal from an image substantially as described herein with reference to the Figures.

33. A method of encoding a signal in an image substantially as herein described with reference to the Figures.

34. Apparatus substantially as described herein with reference to the Figures.