WO2005010634A2 - Digital pen function control - Google Patents

Digital pen function control Download PDF

Info

Publication number
WO2005010634A2
WO2005010634A2 PCT/IB2004/003218 IB2004003218W WO2005010634A2 WO 2005010634 A2 WO2005010634 A2 WO 2005010634A2 IB 2004003218 W IB2004003218 W IB 2004003218W WO 2005010634 A2 WO2005010634 A2 WO 2005010634A2
Authority
WO
WIPO (PCT)
Prior art keywords
digital
digital pen
module
pen device
handwriting
Prior art date
Application number
PCT/IB2004/003218
Other languages
French (fr)
Other versions
WO2005010634A3 (en
Inventor
Jean-Luc Dupraz
Florian M. Kehlstadt
Emmanuel Dubois
Fabrice Lauper
Original Assignee
Logitech Europe S.A.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Logitech Europe S.A. filed Critical Logitech Europe S.A.
Priority to DE112004001404T priority Critical patent/DE112004001404T5/en
Publication of WO2005010634A2 publication Critical patent/WO2005010634A2/en
Publication of WO2005010634A3 publication Critical patent/WO2005010634A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03545Pens or stylus

Definitions

  • the present invention relates generally to digital writing input devices for computer systems and more particularly to a multi-function digital pen with function control features.
  • One type of conventional prior art digital pen device includes a digital pen and tablet configuration.
  • the digital pen and tablet couple together.
  • the tablet couples with a personal computer.
  • the tablet includes electronics to recognize character strokes when the digital pen is in contact with the tablet.
  • the tablet may include special function areas on which a user may tap with the digital pen to invoke a function.
  • the function may be pre- configured or pre-programmed.
  • the function can be preprogrammed to include a keystroke function such as open file, e.g., ⁇ CNTLxO>, or print file, e.g., ⁇ CNTL> ⁇ P>.
  • the digital pen and tablet configuration includes a number of drawbacks.
  • the digital pen and tablet are bulky units. They require a large surface area for use and are too heavy to be considered portable. In addition, the pen is not useful without the tablet and vice versa.
  • Another conventional digital pen device has appeared on the market, the Anoto Digital Pen, which is manufactured by a subsidiary of C Technologies AB of Sweden, Anoto AB of Sweden.
  • the Anoto Digital Pen includes a digital camera, an advanced image-processing unit and a Bluetooth radio transceiver. It also contains an ink cartridge so a user can see what is written or drawn.
  • the image capturing mechanism of the Anoto Digital Pen is activated by removing a cap and deactivated by replacing the cap again.
  • the image processing capability of the Anoto Digital Pen is operational only when the Anoto Digital Pen is used with a specially designed paper.
  • the specially designed paper must include a custom pattern that consists of very small dots slightly displaced from a grid structure and a special transfer box (sometimes referred to as magic "boxes").
  • magic "boxes" As a user writes or draws on the specially designed paper with the Anoto Digital Pen, the Anoto Digital Pen creates a digital trace of whatever is done. This information is stored in the pen until the transfer indication box is selected.
  • One drawback of the Anoto Digital Pens and other types of digital pens that capture handwriting strokes is that the digital pens themselves lack functional buttons or displays to capture additional handwritten gestures such as highlighting, holding, or color change. For example, when a user handwrites notes and then uses a highlighter to emphasize particular text or patterns, the conventional digital pen device or system such as those described above, only capture the original handwritten information, but are unable to capture the emphasized text or pattern.
  • an attempt to go back and, for example, highlight text, using a conventional digital pen device or system may result in canceling or crossing out previously written text.
  • Some other drawbacks of some digital pens that capture handwriting strokes are that the digital pens do not have added functionalities which can make these pens useful in other ways to a user.
  • a digital pen with an alarm functionality could be used by a user to remind herself of certain important appointments etc.
  • Such notification to the user could be provided by sound (e.g., a beep, a buzz, etc.), by vibration of the pen, and so on.
  • Another example is a digital pen that can function as a calculator, which could be used to perform quick mathematical calculations.
  • a writing system is configured for capturing image data associated with writing gestures.
  • the writing system also provides additional formatting functionality, for example, holding, highlighting, underlining, italicizing, and the like. Further, the writing system may include other features that conveniently complement the handwriting capturing functions, such as, for example, a digital camera, a pocket calculator, a user notification feature, alarms, timers, calendar function, biometric sensors, and the like.
  • the writing system according to the present invention includes a digital pen device with selection mechanisms for a user to select among the different alternative functions provided, for example, highlighting, holding, underlining, and the like. According to another aspect of the present invention, it is provided a user interface for the management of data files generated by a system according to one embodiment of the invention.
  • a method includes the capture of information relating to a user's writing.
  • a method includes formatting of captured information.
  • the formatting may include holding, underlining, highlighting, color change, among other features that may be applied to the captured handwriting.
  • the formatting features can be selected at different times with respect to the capturing of the handwriting gestures, for example, a holding selection can be made prior to capturing the handwriting text to be bolded, or alternatively, once some handwriting has been captured, it can be selected for applying highlighting to it.
  • several embodiments are presented with additional functions provided.
  • one embodiment of the present invention includes a notification feature that allows a user to preset a trigger or alarm for the notification to take place, such as, for example, a timer or a calendar entry.
  • a notification feature that allows a user to preset a trigger or alarm for the notification to take place, such as, for example, a timer or a calendar entry.
  • the notification feature including an audible alarm, vibration, visual display or the like.
  • Other convenient functions presented include still or video capturing devices, audio input and output devices, calculator, biometrics, and the like.
  • Figure 1 is an illustration of a digital pen device according to one embodiment of the present invention.
  • Figure 2 is a functional block diagram of a digital pen device according to one embodiment of the present invention.
  • Figure 3 illustrates a sample function control subsystem software within a software stack for a digital pen device according to one embodiment of the present invention.
  • Figure 4 illustrates a sample process operation of a function control subsystem according to one embodiment of the present invention.
  • Figures 5a, 5b, and 5c show an illustration of one example of an operation according to one embodiment of the present invention.
  • Figure 6a is sample screen shot of a user interface according to one embodiment of the present invention.
  • Figure 6b is a sample detailed view of a screen shot of a user interface according to one embodiment of the present invention.
  • DETAILED DESCRIPTION OF THE INVENTION [0021] The Figures and the following description relate to preferred embodiments of the present invention by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of the claimed invention.
  • One embodiment of the present invention comprises a digital pen device that includes function control selection mechanisms, e.g., buttons, pressure switches, or the like.
  • a user that is writing text, patterns, or other handwritten expression on a writing medium, e.g., paper, may operate the digital pen device.
  • the digital pen device captures the handwritten expression, for example, as images, and stores the handwritten expression as data. If additional expression is added to the handwritten expression, e.g., highlighting a word, the function control buttons are configured to capture this additional expression as data too.
  • Figure 1 illustrates a digital pen device 100 in accordance with one embodiment of the present invention.
  • the digital pen device 100 includes a pen barrel 110, an optional grip area 115, a pen (or ink) tip 120, a character recognition lens 125, a function control box 130 and a display area 135.
  • the ink tip 120 and the character recognition lens 125 are in close proximity to each other.
  • the function control box 130 may be comprised of one or more mechanical or electrical (e.g., touch sensitive or optical) switches 130a- 130c.
  • the function control box 130 may be configured for placement along a side of the pen barrel 110.
  • Figure 2 illustrates a functional block diagram of a digital pen device 100 in accordance with one embodiment of the present invention.
  • the digital pen device 100 includes a character recognition subsystem 210, a function control subsystem 220, a processor (or controller) subsystem 230, and a memory subsystem 240.
  • the character recognition subsystem 210 may include a character recognition lens 125, and may couple with the processor subsystem 230 and the memory subsystem 240.
  • the function control subsystem 220 couples with the processor subsystem 230 and may also couple with the memory subsystem 240 and/or the character recognition subsystem 210.
  • the character recognition subsystem 210 may be a digital camera or may be an optical character recognition system that includes pattern recognition sensors, e.g., photosensors. In some embodiments, the digital pen 100 includes a separate mini camera 290.
  • the digital pen 100 can be used as a camera as well as a pen, and can be used capture images or video sequences.
  • the character recognition subsystem 210 when the character recognition subsystem 210 is a camera, the character recognition subsystem 210 itself may be used to capture images or video sequences.
  • the character recognition subsystem 210 is configured to capture handwriting gestures, e.g., handwritten text, numbers, symbols, drawings, or other impressions that may appear on a written medium.
  • the function control subsystem 220 may include dedicated (or pre-defined) or configurable (e.g., user definable) functional operations that may, for example, be used in association with one or more application programs.
  • the function control subsystem 220 may be configured to identify an area that a user has highlighted using a highlighter and transmit that information back to an application associated with the character recognition subsystem 210.
  • the processor subsystem 230 provides conventional processor functions, including fetching and executing instructions to carry out functions, for example, functions defined by the character recognition subsystem 210 and the function control subsystem 220.
  • the memory subsystem 240 may include volatile memory (e.g., dynamic and/or static random access memory) and/or non-volatile memory (e.g., flash memory and/or electrically erasable programmable read only memory).
  • the digital pen device may also include an optional ink cartridge, configured in close proximity to the character recognition device, so that impression written by a user is immediately picked up by the character recognition subsystem 210.
  • the digital pen device may include a communication interface subsystem 250.
  • the communication interface subsystem 250 may be for wired (tethered) or wireless (non-tethered) communication.
  • the communication interface subsystem 250 may be, for example, a radio frequency transceiver or a universal serial bus interface.
  • the communication interface subsystem 250 may be an Infra Red communication system.
  • the communication interface subsystem 250 couples with, for example, the processor subsystem 230 to transmit information to, for example, a host computer in either real-time or batch mode (e.g., synchronization process such as a docking connection or universal serial bus connection on the pen, or the like). It is also noted that the digital pen device 100 components discussed above may all couple together via a data bus 260 in the digital pen device 100.
  • Figure 3 illustrates one embodiment of the function control subsystem 220 software within a software stack for a digital pen device 100 in accordance with one embodiment of the present invention.
  • the software stack includes a pen device driver 310 and an application 320.
  • the software 220a for the function control subsystem 220 can reside on top of the pen device driver 310, an operating system, or within each individually or in combination.
  • the pen device driver 220 couples with the data bus 260.
  • the pen data bus 260 may include an electrical connection with the function control box 130 (or switches 130a- 130c).
  • a user may configure the selection mechanisms 130a-130c (130n generally), to correspond to a particular function, e.g., highlighting, holding, italicizing, underlining, changing font color, or the like.
  • the configuration may be provided on the digital pen 100 itself, or may be done, for example, via a software application on a personal computer that is transmitted to the digital pen 100.
  • the selection mechanisms 130n send a function selection signal back to the pen data bus 260 to the processor subsystem 230, to the pen device control, and then to the function control subsystem 220.
  • the function control subsystem 220 interprets the function selected and signals the operating system or the application 320 regarding the selected function.
  • the handwriting gestures with which the formatting is associated can be determined in several ways.
  • a user can actuate a selection mechanism 13 On to select a formatting function prior to handwriting the gestures to which the formatting applies.
  • the user may actuate the same or another selection mechanism to indicate the end of the handwriting gestures to which the formatting applies.
  • the user may produce the handwriting gestures first, then select a subset of handwriting gestures, and actuate a selection mechanism 130n to apply the formatting to the selected handwriting gestures.
  • patterned paper comprising machine-readable marks or "glyphs" provides a coordinate system that can be interpreted to uniquely identify the location of each individual glyph.
  • This coordinate or location information is typically captured along with the handwriting gestures.
  • a user may indicate the coordinate information (e.g., selecting a set of glyphs) of two opposite corners of a rectangular area on the patterned paper and using the captured coordinate information the processor can determine the subset of handwriting gestures located within the selected rectangular area.
  • Figures 5a through 5c illustrate one example of an operation in accordance with one embodiment of the present invention.
  • existing handwriting gestures are selected by indicating the coordinates of the top left beginning of a handwriting subset and the coordinates of the bottom right end of the subset. Then, the user actuates function control switch 130a, configured for example to "highlight” handwriting gestures. The digital pen 100 then transmits the function ("highlight") and coordinate information back to the processor subsystem 230, onto the pen device driver 310 and the function control subsystem 220. The function control subsystem 220 would provide information to the application 320 that tells it to highlight the selected area provided by the position coordinates. [0031] According to another embodiment of the present invention, the selection of a subset of captured handwriting gestures for applying a selected formatting function to includes an image processing function.
  • handwriting gestures may be captured using an optical imaging system, for example, a photosensor array.
  • the handwriting gestures are captured as bitmaps or similar images created based on the light reflected from a writing surface.
  • a user may actuate a selection mechanism, e.g. a button, and recapture a subset of handwriting gestures to which a formatting function should be applied.
  • a processor can compare previously captured images with the currently captured images to find a match thereby determining the subset of captured handwriting gestures to which the selected formatting function applies.
  • a user may change the tip of the digital pen 100 on a multiple-tip pen embodiment from a ball pen tip, to a highlighter tip and the digital pen 100 would automatically recognize the highlighting function from the use of the highlighter tip and capture the highlighting information accordingly.
  • a single selection mechanism 130a can be pressed to perform various distinct functions, based on the level or pressure/force applied. For instance, a slight pressure applied to control switch 130a might result in highlighting of the text, some more pressure applied to control switch 130a might result in holding of the text, while still more pressure applied to control switch 130a might result in changing the color of the text.
  • the pressure applied by the user can be sensed using a Force Sensitive Resistor ("FSR") located at the tip of the digital pen 100.
  • FSR Force Sensitive Resistor
  • the selection mechanism can be configured as an n-way switch, where n is any integer and corresponds to the number of states (e.g., functions that may be available).
  • a function control switch 130a is configured for voice recording.
  • an apparatus in accordance with the present invention is a dual mode pocket memo recorder that offers both written- and voice- memo download to a host.
  • Such a digital pen 100 can be used, for example, to record to-do list items in the digital pen's memory. Moreover, such a digital pen 100 can be especially useful in situations when a user cannot write, or when the user has no special paper available at hand. For example, a user may be driving a car.
  • Voice recording by the digital pen 100 can have several other benefits as well. For example, in one embodiment of the present invention, voice recording is linked to written material. For instance, a user could draw on paper using a digital pen 100, and record a verbal description of the drawing. Such a verbal description, along with the linked drawing would provide for a very accurate and clear capture of the thoughts of the user.
  • voice recording is used in conjunction with voice recognition.
  • the digital pen 100 includes a Coder-Decoder ("CODEC") 295 to perform functions such as converting a voice analog signal into a digital signal.
  • the CODEC 295 can be implemented in hardware, software, or firmware, or in any combination of these.
  • compression can be used to store the digital voice data in its stand alone memory (non-volatile) in a more effective way, in order to implement a longer recording time.
  • the compression algorithm used is compatible with PC standards.
  • a voice-to-text conversion module 270 is also present in the digital pen 100.
  • the voice-to-text conversion module 270 can be used to convert voice data into text with a specific voice recognition software application, as available today with some voice recorders.
  • the digital pen 100 may also have an audio input 265 and an audio output 285.
  • the audio output 285 could be, for example, a speaker or earphone jack. Both the audio input 265 and the audio output 285 may be processed by a CODEC 295.
  • Such a digital pen 100 can be used as a stand alone voice recording and playback system, i.e., without necessarily downloading to a host.
  • a biometric sensor module 280 is also present in the digital pen 100.
  • this biometric sensor module could be a fingerprint sensor.
  • Such a biometric sensor module 280 can be used for identification of the user of the pen. For some applications, this could be in lieu of a password.
  • biometric sensing could also be used to retrieve each user's preferred settings when that user starts using the pen.
  • preferences may also be associated automatically with a user, by identifying the user via biometric sensing.
  • the biometric sensor in one embodiment may be functional with the optional grip area 115 or in lieu of the optional grip area 115.
  • FIG. 1 Another example of a functionality that can be available in the digital pen 100 is that of a pocket calculator. Such a digital pen 100 would provide the user with a "pocket calculator" which is easily available, and is small and convenient to carry. [0040] In one embodiment, the numbers and the operators could be captured by handwriting. Thus no keyboard would be needed. In another embodiment, the capture may also be implemented by using special paper with printed numbers and operators. Further, in one embodiment, the result of the calculation could be shown on an on-board display 135, e.g., a liquid crystal display (“LCD").
  • LCD liquid crystal display
  • a functionality that can be available in the digital pen 100 is that of a user notification function, e.g., an alarm notification or the like, provided in response to a user programmed triggering event.
  • the special paper may include conventional patterned paper, e.g., having glyphs, and the "magic boxes" may be preset patterns corresponding to a function once that pattern image is detected.
  • special paper with specific fields (“magic boxes") can be used to enter an alarm.
  • different triggering events for the user notifications can be preprogrammed depending on the functionality available in the digital pen. .
  • the triggering event may be an appointment at a certain date and time or a reminder for a special occasion, or the like.
  • the triggering event may be a timer/countdown alarm, or a wake-up alarm, or the like.
  • the user notification can take place in several different ways. For example, some possible user notifications include without limitation, an alarm via the display 135, a buzzer (sound feedback), a vibration of the digital pen 100, or via one or more light emitting diodes ("LEDs").
  • FIG. 4 illustrates a process operation of the function control subsystem 220 in accordance with one embodiment of the present invention. The process starts 410 and a particular function is selected 420 through a selection mechanism, e.g. 130a. A position on a writing surface, for example, a piece of paper, is identified 430 on which the functional operation is to occur.
  • a selection mechanism e.g. 130a.
  • this includes identifying a first location, pi, and a second location, p2 (see, e.g., Figure 5b) by moving the digital pen over or on those two areas to trigger a selection mechanism (e.g., a switch or a sensor).
  • the function control subsystem 220 then calculates 440 an area, A, using the two position data points. This information may then be passed to, for example, the application 320 to apply the function, e.g., highlighting previously written text as shown in Figures 5a through 5c.
  • the process then ends 460 by shutting off or waiting for the next function operation.
  • a writing system in accordance with one embodiment of the present invention may also be used with a special paper such that ink does not adhere to the paper.
  • a card which fits into a wallet could be carried by a user.
  • the user could check of a "magic box” on this card every time he wanted to perform a specific function (e.g., transfer to the host data written on special paper, transfer to the host data written only on a specified page, etc.).
  • the special paper on such a card could be treated such that ink will not adhere-to it.
  • the "magic boxes" are predefined and may be Intelligent Character Recognition ("ICR") fields.
  • the page number could be "written” into an ICR field on the card.
  • a visual keyboard could be visible on the card, and the appropriate "key” could be tapped on in order to perform a specific function.
  • ICR Another alternative to using ICR is the use of graffiti, as is well-known in the context of Personal Digital Assistants ("PDAs").
  • the "magic box" may be a predetermined pattern (e.g., a character) that is assigned a particular function or operation such that it is performed once the optical system recognizes the predetermined pattern.
  • magic boxes are on the card described above. In other embodiments, the magic boxes are on the paper on which the user is writing. In addition, in some embodiments, ink may adhere to such magic boxes, while in other embodiments, ink may not adhere to such magic boxes.
  • the speed-mail notion is analogous to the speed-dial feature available on most telephone instruments today. Specific e-mail addresses can be preprogrammed into specific numbers (e.g., "1" corresponds to mom's email address, etc.), and then the specific number could be written into a magic box to send the data to the specified email address.
  • Fig. 6a is a screenshot of an embodiment of a user interface 600 on a host, which can make it easier for users to find, view, and integrate digital writing documents into other applications.
  • all documents created with the pen are listed under a folder entitled "My Pen Documents.” It is to be noted that in one embodiment, this destination folder can be renamed or changed by the user as desired.
  • the created documents can be sorted by different criteria (e.g., date, type, etc.).
  • Fig. 6b provides a more detailed view of the user interface 600.
  • the user interface 600 allows the user to use the documents directly (e.g., print them, send as an attachment in an email, send directly in an email, etc.).
  • some tags may be inserted into them (e.g., date, time, author's name, sub-title, keyword etc.).
  • the insertion of such tags may be automatic.
  • the insertion of such tags may be manual.
  • the insertion of such tags may be a combination of automatic and manual.
  • the user interface 600 may include a list of templates for each type of document.
  • the various "types" of documents can include emails, faxes, letters, meeting minutes, notes, to-do lists, drawings, calendar entries, and so on).
  • the user can automatically insert the document into a fax template.
  • the document can them be faxed using third party software.
  • the user interface 600 permits the user to seamlessly integrate with third party software.
  • the user may be able to assign certain rules to a specific document, or to a type of document.
  • a user can assign rules such as which application to use for the email (e.g., Microsoft OutlookTM, Lotus NotesTM, etc.), the way the document will appear (e.g., in draft form, etc.), and so on. Further, such rules can be assigned to the specific email in question, or to all emails created with the digital pen 100. Moreover, a rule may be set as a permanent rule or as a temporary rule. [0052] In one embodiment of the present invention, a user can set up such rules by using a "Rule-setup Wizard" which assists the user in defining and assigning rules.
  • the digital pen 100 may also be configured to provide bi-directional (or alternatively unidirectional) communication with another device, e.g., a host computer, through its communication subsystem 250.
  • a host computer can be configured to retrieve digital pen status information.
  • the digital pen 100 can be configured to transmit pen status information to the host computer.
  • the host computer can display the pen status information.
  • Pen status information may include, for example, battery status, available memory, used memory, pen serial number, pen identifier, user settings (e.g., configuration of selection mechanism), and the like.
  • pen information may also include commands transmitted from the host computer to the digital pen 100, for example, setting an indicator (e.g., vibrator, LED) ON/OFF or setting power-off delays, or synchronizing a calendar, or the like.
  • an indicator e.g., vibrator, LED
  • pen information may also include commands transmitted from the host computer to the digital pen 100, for example, setting an indicator (e.g., vibrator, LED) ON/OFF or setting power-off delays, or synchronizing a calendar, or the like.

Abstract

A digital writing system according to one embodiment of the present invention captures handwritten gestures in addition to text and patterns. It may include, among other things, a digital pen, special erasable ink and paper, and application software for the managing of captured files with a corresponding user interface. A digital pen according to one embodiment of the present invention may include character recognition subsystem, function control subsystem with selection mechanisms, a processor subsystem, memory subsystems, and wired or wireless communication subsystems. In addition, embodiments of a digital pen according to the present invention may include additional functional features including, for example, a user notification (e.g., alarm) feature, a digital camera feature, voice recording and recognition features, calendar features, calculator features, biometric sensor features, and the like.

Description

DIGITAL PEN FUNCTION CONTROL FIELD OF THE INVENTION [0001] The present invention relates generally to digital writing input devices for computer systems and more particularly to a multi-function digital pen with function control features.
BACKGROUND OF THE INVENTION [0002] Conventional digital pen devices that capture handwriting strokes are known. One type of conventional prior art digital pen device includes a digital pen and tablet configuration. The digital pen and tablet couple together. The tablet couples with a personal computer. The tablet includes electronics to recognize character strokes when the digital pen is in contact with the tablet. In addition, the tablet may include special function areas on which a user may tap with the digital pen to invoke a function. The function may be pre- configured or pre-programmed. For example, the function can be preprogrammed to include a keystroke function such as open file, e.g., <CNTLxO>, or print file, e.g., <CNTL><P>. [0003] The digital pen and tablet configuration includes a number of drawbacks. For example, the digital pen and tablet are bulky units. They require a large surface area for use and are too heavy to be considered portable. In addition, the pen is not useful without the tablet and vice versa. To help address these shortcomings, another conventional digital pen device has appeared on the market, the Anoto Digital Pen, which is manufactured by a subsidiary of C Technologies AB of Sweden, Anoto AB of Sweden. [0004] The Anoto Digital Pen includes a digital camera, an advanced image-processing unit and a Bluetooth radio transceiver. It also contains an ink cartridge so a user can see what is written or drawn. The image capturing mechanism of the Anoto Digital Pen is activated by removing a cap and deactivated by replacing the cap again. However, the image processing capability of the Anoto Digital Pen is operational only when the Anoto Digital Pen is used with a specially designed paper. The specially designed paper must include a custom pattern that consists of very small dots slightly displaced from a grid structure and a special transfer box (sometimes referred to as magic "boxes"). As a user writes or draws on the specially designed paper with the Anoto Digital Pen, the Anoto Digital Pen creates a digital trace of whatever is done. This information is stored in the pen until the transfer indication box is selected. It is then forwarded from your digital pen directly to your nearby personal computer, or by a Bluetooth™ device - such as a Bluetooth™ enabled mobile phone - to any computer, mobile phone or information bank in the world via the Internet. [0005] One drawback of the Anoto Digital Pens and other types of digital pens that capture handwriting strokes is that the digital pens themselves lack functional buttons or displays to capture additional handwritten gestures such as highlighting, holding, or color change. For example, when a user handwrites notes and then uses a highlighter to emphasize particular text or patterns, the conventional digital pen device or system such as those described above, only capture the original handwritten information, but are unable to capture the emphasized text or pattern. Moreover, an attempt to go back and, for example, highlight text, using a conventional digital pen device or system may result in canceling or crossing out previously written text. [0006] Some other drawbacks of some digital pens that capture handwriting strokes are that the digital pens do not have added functionalities which can make these pens useful in other ways to a user. For instance, a digital pen with an alarm functionality could be used by a user to remind herself of certain important appointments etc. Such notification to the user could be provided by sound (e.g., a beep, a buzz, etc.), by vibration of the pen, and so on. Another example is a digital pen that can function as a calculator, which could be used to perform quick mathematical calculations. With such a digital pen, a user would be able to dispense with pocket calculators. [0007] Yet another drawback of conventional digital pens is that they do not have a voice- recording functionality. Such a voice-recording functionality would be useful in several situations. For instance, such a feature would be very useful in instances when the user cannot write (e.g., when the user is driving a car), or when the user does not have any special paper available at hand. [0008] Still another drawback of conventional digital pens is that the special paper and the "magic boxes" on such a special paper described above are not reusable, since ink adheres to the paper when the pen is used. In certain situations, it may be desirable to not have the ink adhere to the paper, so that the paper can then be reused. For instance, a user may desire to use over and over again, a "magic box" to transmit written material to a host. [0009] Therefore, there is a need for (1) a digital writing system that captures additional handwritten gestures in addition to handwritten text and patterns; (2) a digital writing system with additional functionalities; (3) a digital writing system with a voice recording and recognition capability; and (4) a digital writing system where, in certain situations, the ink does not adhere to the paper. SUMMARY OF THE INVENTION [0010] In accordance with one illustrative embodiment of the present invention, a writing system is configured for capturing image data associated with writing gestures. The writing system also provides additional formatting functionality, for example, holding, highlighting, underlining, italicizing, and the like. Further, the writing system may include other features that conveniently complement the handwriting capturing functions, such as, for example, a digital camera, a pocket calculator, a user notification feature, alarms, timers, calendar function, biometric sensors, and the like. In one embodiment, the writing system according to the present invention includes a digital pen device with selection mechanisms for a user to select among the different alternative functions provided, for example, highlighting, holding, underlining, and the like. According to another aspect of the present invention, it is provided a user interface for the management of data files generated by a system according to one embodiment of the invention. [0011] A method according to one embodiment of the present invention includes the capture of information relating to a user's writing. For example, in response to selection signals, a method according to this embodiment includes formatting of captured information. The formatting may include holding, underlining, highlighting, color change, among other features that may be applied to the captured handwriting. According to one embodiment of the present invention, the formatting features can be selected at different times with respect to the capturing of the handwriting gestures, for example, a holding selection can be made prior to capturing the handwriting text to be bolded, or alternatively, once some handwriting has been captured, it can be selected for applying highlighting to it. [0012] In addition, according to another aspect of the present invention, several embodiments are presented with additional functions provided. For example, one embodiment of the present invention includes a notification feature that allows a user to preset a trigger or alarm for the notification to take place, such as, for example, a timer or a calendar entry. Several alternatives are also shown for the notification feature including an audible alarm, vibration, visual display or the like. Other convenient functions presented include still or video capturing devices, audio input and output devices, calculator, biometrics, and the like. [0013] The features and advantages described in the specification are not all inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. BRIEF DESCRIPTION OF THE DRAWINGS [0014] Figure 1 is an illustration of a digital pen device according to one embodiment of the present invention. [0015] Figure 2 is a functional block diagram of a digital pen device according to one embodiment of the present invention. [0016] Figure 3 illustrates a sample function control subsystem software within a software stack for a digital pen device according to one embodiment of the present invention. [0017] Figure 4 illustrates a sample process operation of a function control subsystem according to one embodiment of the present invention. [0018] Figures 5a, 5b, and 5c show an illustration of one example of an operation according to one embodiment of the present invention. [0019] Figure 6a is sample screen shot of a user interface according to one embodiment of the present invention. [0020] Figure 6b is a sample detailed view of a screen shot of a user interface according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION [0021] The Figures and the following description relate to preferred embodiments of the present invention by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of the claimed invention. [0022] One embodiment of the present invention comprises a digital pen device that includes function control selection mechanisms, e.g., buttons, pressure switches, or the like. A user that is writing text, patterns, or other handwritten expression on a writing medium, e.g., paper, may operate the digital pen device. The digital pen device captures the handwritten expression, for example, as images, and stores the handwritten expression as data. If additional expression is added to the handwritten expression, e.g., highlighting a word, the function control buttons are configured to capture this additional expression as data too. [0023] Figure 1 illustrates a digital pen device 100 in accordance with one embodiment of the present invention. The digital pen device 100 includes a pen barrel 110, an optional grip area 115, a pen (or ink) tip 120, a character recognition lens 125, a function control box 130 and a display area 135. In one embodiment, the ink tip 120 and the character recognition lens 125 are in close proximity to each other. In addition, the function control box 130 may be comprised of one or more mechanical or electrical (e.g., touch sensitive or optical) switches 130a- 130c. The function control box 130 may be configured for placement along a side of the pen barrel 110. [0024] Figure 2 illustrates a functional block diagram of a digital pen device 100 in accordance with one embodiment of the present invention. The digital pen device 100 includes a character recognition subsystem 210, a function control subsystem 220, a processor (or controller) subsystem 230, and a memory subsystem 240. The character recognition subsystem 210 may include a character recognition lens 125, and may couple with the processor subsystem 230 and the memory subsystem 240. The function control subsystem 220 couples with the processor subsystem 230 and may also couple with the memory subsystem 240 and/or the character recognition subsystem 210. [0025] The character recognition subsystem 210 may be a digital camera or may be an optical character recognition system that includes pattern recognition sensors, e.g., photosensors. In some embodiments, the digital pen 100 includes a separate mini camera 290. In such embodiments, the digital pen 100 can be used as a camera as well as a pen, and can be used capture images or video sequences. In other embodiments, when the character recognition subsystem 210 is a camera, the character recognition subsystem 210 itself may be used to capture images or video sequences. [0026] The character recognition subsystem 210 is configured to capture handwriting gestures, e.g., handwritten text, numbers, symbols, drawings, or other impressions that may appear on a written medium. The function control subsystem 220 may include dedicated (or pre-defined) or configurable (e.g., user definable) functional operations that may, for example, be used in association with one or more application programs. For example, the function control subsystem 220 may be configured to identify an area that a user has highlighted using a highlighter and transmit that information back to an application associated with the character recognition subsystem 210. The processor subsystem 230 provides conventional processor functions, including fetching and executing instructions to carry out functions, for example, functions defined by the character recognition subsystem 210 and the function control subsystem 220. The memory subsystem 240 may include volatile memory (e.g., dynamic and/or static random access memory) and/or non-volatile memory (e.g., flash memory and/or electrically erasable programmable read only memory). [0027] The digital pen device may also include an optional ink cartridge, configured in close proximity to the character recognition device, so that impression written by a user is immediately picked up by the character recognition subsystem 210. In addition, the digital pen device may include a communication interface subsystem 250. The communication interface subsystem 250 may be for wired (tethered) or wireless (non-tethered) communication. The communication interface subsystem 250 may be, for example, a radio frequency transceiver or a universal serial bus interface. Alternatively, the communication interface subsystem 250 may be an Infra Red communication system. The communication interface subsystem 250 couples with, for example, the processor subsystem 230 to transmit information to, for example, a host computer in either real-time or batch mode (e.g., synchronization process such as a docking connection or universal serial bus connection on the pen, or the like). It is also noted that the digital pen device 100 components discussed above may all couple together via a data bus 260 in the digital pen device 100. [0028] Figure 3 illustrates one embodiment of the function control subsystem 220 software within a software stack for a digital pen device 100 in accordance with one embodiment of the present invention. The software stack includes a pen device driver 310 and an application 320. The software 220a for the function control subsystem 220 can reside on top of the pen device driver 310, an operating system, or within each individually or in combination. The pen device driver 220 couples with the data bus 260. The pen data bus 260 may include an electrical connection with the function control box 130 (or switches 130a- 130c). According to one embodiment of the present invention, a user may configure the selection mechanisms 130a-130c (130n generally), to correspond to a particular function, e.g., highlighting, holding, italicizing, underlining, changing font color, or the like. The configuration may be provided on the digital pen 100 itself, or may be done, for example, via a software application on a personal computer that is transmitted to the digital pen 100. Further, different users may configure the selection mechanisms differently and a set of user profiles can be stored on the digital pen 100 and selected prior to operation, for example, using biometrics as discussed below. [0029] During operation, when the user triggers a selection mechanism, e.g., any of 130a- 130c alone or in combination, the selection or switching mechanisms 130n send a function selection signal back to the pen data bus 260 to the processor subsystem 230, to the pen device control, and then to the function control subsystem 220. The function control subsystem 220, interprets the function selected and signals the operating system or the application 320 regarding the selected function. The handwriting gestures with which the formatting is associated can be determined in several ways. For example, a user can actuate a selection mechanism 13 On to select a formatting function prior to handwriting the gestures to which the formatting applies. Once finished with the handwriting, the user may actuate the same or another selection mechanism to indicate the end of the handwriting gestures to which the formatting applies. In an alternative embodiment, the user may produce the handwriting gestures first, then select a subset of handwriting gestures, and actuate a selection mechanism 130n to apply the formatting to the selected handwriting gestures. [0030] According to the present invention, there are several possible techniques for selecting a subset of handwriting gestures in order to apply the formatting function to it. For example, in one embodiment, patterned paper comprising machine-readable marks or "glyphs" provides a coordinate system that can be interpreted to uniquely identify the location of each individual glyph. This coordinate or location information is typically captured along with the handwriting gestures. Subsequently, a user may indicate the coordinate information (e.g., selecting a set of glyphs) of two opposite corners of a rectangular area on the patterned paper and using the captured coordinate information the processor can determine the subset of handwriting gestures located within the selected rectangular area. For example, Figures 5a through 5c illustrate one example of an operation in accordance with one embodiment of the present invention. In this example, existing handwriting gestures are selected by indicating the coordinates of the top left beginning of a handwriting subset and the coordinates of the bottom right end of the subset. Then, the user actuates function control switch 130a, configured for example to "highlight" handwriting gestures. The digital pen 100 then transmits the function ("highlight") and coordinate information back to the processor subsystem 230, onto the pen device driver 310 and the function control subsystem 220. The function control subsystem 220 would provide information to the application 320 that tells it to highlight the selected area provided by the position coordinates. [0031] According to another embodiment of the present invention, the selection of a subset of captured handwriting gestures for applying a selected formatting function to includes an image processing function. According to this embodiment, handwriting gestures may be captured using an optical imaging system, for example, a photosensor array. The handwriting gestures are captured as bitmaps or similar images created based on the light reflected from a writing surface. Subsequently, a user may actuate a selection mechanism, e.g. a button, and recapture a subset of handwriting gestures to which a formatting function should be applied. Then, a processor can compare previously captured images with the currently captured images to find a match thereby determining the subset of captured handwriting gestures to which the selected formatting function applies. Several other implementations are possible to provide the same functionality. For example, rather than using a selection mechanism 130n, a user may change the tip of the digital pen 100 on a multiple-tip pen embodiment from a ball pen tip, to a highlighter tip and the digital pen 100 would automatically recognize the highlighting function from the use of the highlighter tip and capture the highlighting information accordingly. [0032] In another embodiment, it is possible that a single selection mechanism 130a can be pressed to perform various distinct functions, based on the level or pressure/force applied. For instance, a slight pressure applied to control switch 130a might result in highlighting of the text, some more pressure applied to control switch 130a might result in holding of the text, while still more pressure applied to control switch 130a might result in changing the color of the text. In one embodiment, the pressure applied by the user can be sensed using a Force Sensitive Resistor ("FSR") located at the tip of the digital pen 100. In this embodiment, it is not necessary to have a specifically dedicated button on the pen 100 for each desired functionality. In an alternative embodiment, the selection mechanism can be configured as an n-way switch, where n is any integer and corresponds to the number of states (e.g., functions that may be available). [0033] Another example is that in one embodiment, a function control switch 130a is configured for voice recording. In such an embodiment, an apparatus in accordance with the present invention is a dual mode pocket memo recorder that offers both written- and voice- memo download to a host. Such a digital pen 100 can be used, for example, to record to-do list items in the digital pen's memory. Moreover, such a digital pen 100 can be especially useful in situations when a user cannot write, or when the user has no special paper available at hand. For example, a user may be driving a car. [0034] Voice recording by the digital pen 100 can have several other benefits as well. For example, in one embodiment of the present invention, voice recording is linked to written material. For instance, a user could draw on paper using a digital pen 100, and record a verbal description of the drawing. Such a verbal description, along with the linked drawing would provide for a very accurate and clear capture of the thoughts of the user. Yet another example of a benefit of voice recording is that in one embodiment of the present invention, voice recording is used in conjunction with voice recognition. Such a combination can be used, for instance, for email address look-up. [0035] In such an embodiment, the digital pen 100 includes a Coder-Decoder ("CODEC") 295 to perform functions such as converting a voice analog signal into a digital signal. The CODEC 295 can be implemented in hardware, software, or firmware, or in any combination of these. In one embodiment, compression can be used to store the digital voice data in its stand alone memory (non-volatile) in a more effective way, in order to implement a longer recording time. In one embodiment, the compression algorithm used is compatible with PC standards. [0036] In one embodiment of the present invention, a voice-to-text conversion module 270 is also present in the digital pen 100. The voice-to-text conversion module 270 can be used to convert voice data into text with a specific voice recognition software application, as available today with some voice recorders. [0037] In one embodiment, the digital pen 100 may also have an audio input 265 and an audio output 285. The audio output 285 could be, for example, a speaker or earphone jack. Both the audio input 265 and the audio output 285 may be processed by a CODEC 295. Such a digital pen 100 can be used as a stand alone voice recording and playback system, i.e., without necessarily downloading to a host. [0038] In one embodiment of the present invention, a biometric sensor module 280 is also present in the digital pen 100. For instance this biometric sensor module could be a fingerprint sensor. Such a biometric sensor module 280 can be used for identification of the user of the pen. For some applications, this could be in lieu of a password. If multiple users may be using a digital pen 100, in some embodiments, such biometric sensing could also be used to retrieve each user's preferred settings when that user starts using the pen. In some embodiments, such preferences may also be associated automatically with a user, by identifying the user via biometric sensing. It is noted that the biometric sensor in one embodiment may be functional with the optional grip area 115 or in lieu of the optional grip area 115. [0039] Another example of a functionality that can be available in the digital pen 100 is that of a pocket calculator. Such a digital pen 100 would provide the user with a "pocket calculator" which is easily available, and is small and convenient to carry. [0040] In one embodiment, the numbers and the operators could be captured by handwriting. Thus no keyboard would be needed. In another embodiment, the capture may also be implemented by using special paper with printed numbers and operators. Further, in one embodiment, the result of the calculation could be shown on an on-board display 135, e.g., a liquid crystal display ("LCD"). [0041] Yet another example of a functionality that can be available in the digital pen 100 is that of a user notification function, e.g., an alarm notification or the like, provided in response to a user programmed triggering event. The special paper may include conventional patterned paper, e.g., having glyphs, and the "magic boxes" may be preset patterns corresponding to a function once that pattern image is detected. In addition, in one embodiment, special paper with specific fields ("magic boxes") can be used to enter an alarm. In one embodiment, different triggering events for the user notifications can be preprogrammed depending on the functionality available in the digital pen. . For example, in a digital pen equipped with a digital calendar feature, the triggering event may be an appointment at a certain date and time or a reminder for a special occasion, or the like. Alternatively, in a digital pen that includes a digital watch or a digital timer, the triggering event may be a timer/countdown alarm, or a wake-up alarm, or the like. The user notification can take place in several different ways. For example, some possible user notifications include without limitation, an alarm via the display 135, a buzzer (sound feedback), a vibration of the digital pen 100, or via one or more light emitting diodes ("LEDs"). [0042] In one embodiment with a digital pen having a digital calendar feature, based on the appointments, meetings, reminders, or the like, entered with the digital pen 100, the calendar could be synchronized between the digital pen 100 and a host computer. For example, the synchronizing may be done using the communication module of the digital pen either automatically or at user selected intervals. [0043] Figure 4 illustrates a process operation of the function control subsystem 220 in accordance with one embodiment of the present invention. The process starts 410 and a particular function is selected 420 through a selection mechanism, e.g. 130a. A position on a writing surface, for example, a piece of paper, is identified 430 on which the functional operation is to occur. In one embodiment, this includes identifying a first location, pi, and a second location, p2 (see, e.g., Figure 5b) by moving the digital pen over or on those two areas to trigger a selection mechanism (e.g., a switch or a sensor). The function control subsystem 220 then calculates 440 an area, A, using the two position data points. This information may then be passed to, for example, the application 320 to apply the function, e.g., highlighting previously written text as shown in Figures 5a through 5c. The process then ends 460 by shutting off or waiting for the next function operation. [0044] A writing system in accordance with one embodiment of the present invention may also be used with a special paper such that ink does not adhere to the paper. Such non- adherence of the ink could result in reusability. In one embodiment, a card which fits into a wallet (e.g., sized like a credit card) could be carried by a user. The user could check of a "magic box" on this card every time he wanted to perform a specific function (e.g., transfer to the host data written on special paper, transfer to the host data written only on a specified page, etc.). In one embodiment, the special paper on such a card could be treated such that ink will not adhere-to it. [0045] In one embodiment, the "magic boxes" are predefined and may be Intelligent Character Recognition ("ICR") fields. Thus, for example, in order to transfer data written on a specified page, the page number could be "written" into an ICR field on the card. In another embodiment, a visual keyboard could be visible on the card, and the appropriate "key" could be tapped on in order to perform a specific function. This could be an alternative to using ICR. Another alternative to using ICR is the use of graffiti, as is well-known in the context of Personal Digital Assistants ("PDAs"). In an alternative embodiment, the "magic box" may be a predetermined pattern (e.g., a character) that is assigned a particular function or operation such that it is performed once the optical system recognizes the predetermined pattern. [0046] Yet another use of the magic boxes is for purposes of "speed-mail." It is to be noted that in some embodiments, such magic boxes are on the card described above. In other embodiments, the magic boxes are on the paper on which the user is writing. In addition, in some embodiments, ink may adhere to such magic boxes, while in other embodiments, ink may not adhere to such magic boxes. The speed-mail notion is analogous to the speed-dial feature available on most telephone instruments today. Specific e-mail addresses can be preprogrammed into specific numbers (e.g., "1" corresponds to mom's email address, etc.), and then the specific number could be written into a magic box to send the data to the specified email address. [0047] Once documents are created using the digital pen 100, and these are transferred to a host, it should be easy for a user to find them and use them. Fig. 6a is a screenshot of an embodiment of a user interface 600 on a host, which can make it easier for users to find, view, and integrate digital writing documents into other applications. [0048] As can be seen from Fig. 6a, in one embodiment, all documents created with the pen are listed under a folder entitled "My Pen Documents." It is to be noted that in one embodiment, this destination folder can be renamed or changed by the user as desired. The created documents can be sorted by different criteria (e.g., date, type, etc.). Further, this user interface provides for a preview of the various documents without the need for any third party software. It is also possible to search the documents using a search form. [0049] Fig. 6b provides a more detailed view of the user interface 600. In one embodiment, the user interface 600 allows the user to use the documents directly (e.g., print them, send as an attachment in an email, send directly in an email, etc.). Moreover, when the documents are transferred to the host, some tags may be inserted into them (e.g., date, time, author's name, sub-title, keyword etc.). In one embodiment, the insertion of such tags may be automatic. In another embodiment, the insertion of such tags may be manual. In yet another embodiment, the insertion of such tags may be a combination of automatic and manual. [0050] In one embodiment, the user interface 600 may include a list of templates for each type of document. (The various "types" of documents can include emails, faxes, letters, meeting minutes, notes, to-do lists, drawings, calendar entries, and so on). For instance, if the document is a fax document, the user can automatically insert the document into a fax template. The document can them be faxed using third party software. In one embodiment, the user interface 600 permits the user to seamlessly integrate with third party software. [0051] In one embodiment, the user may be able to assign certain rules to a specific document, or to a type of document. For instance, if a document is an email, a user can assign rules such as which application to use for the email (e.g., Microsoft Outlook™, Lotus Notes™, etc.), the way the document will appear (e.g., in draft form, etc.), and so on. Further, such rules can be assigned to the specific email in question, or to all emails created with the digital pen 100. Moreover, a rule may be set as a permanent rule or as a temporary rule. [0052] In one embodiment of the present invention, a user can set up such rules by using a "Rule-setup Wizard" which assists the user in defining and assigning rules. [0053] The digital pen 100 may also be configured to provide bi-directional (or alternatively unidirectional) communication with another device, e.g., a host computer, through its communication subsystem 250. For example, in one embodiment when the digital pen 100 is connected with a host computer, the host computer can be configured to retrieve digital pen status information. Alternatively, the digital pen 100 can be configured to transmit pen status information to the host computer. In either case, the host computer can display the pen status information. Pen status information may include, for example, battery status, available memory, used memory, pen serial number, pen identifier, user settings (e.g., configuration of selection mechanism), and the like. Moreover, pen information may also include commands transmitted from the host computer to the digital pen 100, for example, setting an indicator (e.g., vibrator, LED) ON/OFF or setting power-off delays, or synchronizing a calendar, or the like. [0054] While particular embodiments and applications of the present invention have been illustrated and described herein, it is to be understood that the invention is not limited to the precise construction and components disclosed herein and that various modifications, changes, and variations may be made in the arrangement, operation, and details of the methods and apparatuses of the present invention without departing from the spirit and scope of the invention as it is defined in the appended claims.

Claims

What is claimed is: 1. A digital writing system for entering handwriting data in a host system, the digital writing system comprising: a digital pen device for capturing handwriting gestures, the digital pen device including a function selection mechanism to select a formatting function for association with the handwriting gestures; and a communication system for coupling the digital pen device to the host system to communicate the captured handwriting gestures, including the selected formatting function, from the digital pen device to the host system.
2. The digital writing system of claim 1, wherein the formatting function further comprises one from a group consisting of holding, italicizing, highlighting, underlining, and changing font colors.
3. The digital writing system of claim 2, further comprising a function module comprising one from the group consisting of a user notification module, a calculator module, a digital camera module, an audio input/output module, a biometric sensor module, a calendar module, and a digital timer module.
4. The digital writing system of claim 1, wherein the communication system comprises a wireless transmitter configured to transmit data associated with the captured handwriting gestures between the digital pen device and the host system.
5. The digital writing system of claim 1, further comprising an application software system for execution in the host system to communicatively couple the communication system to receive handwriting data from the digital pen device and to enable processing of the captured handwriting gesture on the host system.
6. The digital writing system of claim 1, further configured for use with a writing surface and ink, the ink housed within the digital pen device for writing on the writing surface and wherein the ink does not permanently adhere to the writing surface.
7. A digital pen device for capturing handwriting gestures and converting them into digital data for transmission to a computer system, the digital pen device comprising: a pen barrel; a character recognition module housed within the pen barrel for capturing the handwriting gestures; a processor housed within the pen barrel and coupled to the character recognition module for converting the captured handwriting gestures into a first computer readable data; a function selection module at least partially housed within the pen barrel and coupled to the processor, the function selection module including function selection mechanisms for selecting a formatting function to associate with the handwriting gestures, the processor furthered configured to associate the formatting function with the handwriting gestures as formatting information and to convert the formatting information into a second computer readable data; and a communication module at least partially housed within the pen barrel and coupled to the processor for transmitting the first and the second computer readable data to the computer system.
8. The digital pen device of claim 7, further comprising a feature module at least partially housed within the pen barrel for providing a second feature wherein the feature module comprises one from the group consisting of a user notification module, a digital camera, a voice recorder, a calculator, a voice recognition module, an audio file player, a calendar, a timer, and a biometric sensor.
9. The digital pen device of claim 8, wherein the user notification module is configured to provide at least one from the group consisting of an audible alarm, a display message, a vibration, and a visible alarm.
10. The digital pen device of claim 7, wherein the character recognition module comprises a character recognition lens and a photosensor array, the character recognition lens to project light reflected from the handwriting gestures and a writing surface to the photosensor array.
11. The digital pen device of claim 7, wherein the selection mechanisms comprise at least one from the group consisting of a button, a force sensitive resistor, and a switch.
12. The digital pen device of claim 7, wherein the communication module comprises a wireless transmission system.
13. The digital pen device of claim 13, wherein the wireless transmission system communicates using one of the group consisting of radio frequency signals and infrared signals.
14. The digital pen device of claim 7, wherein the communication module comprises a Universal Serial Bus (USB) link.
15. A method for digitally capturing handwriting gestures with a digital writing system comprising a digital pen device, the method comprising: capturing handwriting gestures produced by a user on a writing surface using the digital pen device; converting the captured handwriting gestures into computer readable image data; receiving a formatting function selection signal from a selection mechanism located on the digital pen device indicating a format for application to at least a subset of the handwriting gestures; in response to receiving the formatting function selection signal generating formatting data to apply to the computer readable image data corresponding to the subset of the handwriting gestures; and transmitting the computer readable image data and the formatting data to a host system.
16. The method of claim 15, further including storing the computer readable image data and the formatting data in a memory device prior to transmitting to the host system.
17. The method.of claim 15, wherein the transmitting takes place substantially concurrently with the converting the captured handwriting gestures into computer readable image data.
18. The method of claim 15, wherein the transmitting comprises transmission of the computer readable image data and the formatting data through one from a group consisting of a Universal Serial Bus, a radio frequency transmitter, and an Infrared port.
19. The method of claim 15, further comprising providing a user notification in response to an occurrence of a triggering event, the user notification comprising one from the group consisting of an audible notification, a visible notification, and a vibrating notification.
20. The method of claim 19, wherein the triggering event is one from the group consisting of an appointment, a calendar reminder, a meeting reminder, a clock alarm, and a timer count.
PCT/IB2004/003218 2003-07-30 2004-07-29 Digital pen function control WO2005010634A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
DE112004001404T DE112004001404T5 (en) 2003-07-30 2004-07-29 Digital pen function control

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/631,434 2003-07-30
US10/631,434 US20050024346A1 (en) 2003-07-30 2003-07-30 Digital pen function control

Publications (2)

Publication Number Publication Date
WO2005010634A2 true WO2005010634A2 (en) 2005-02-03
WO2005010634A3 WO2005010634A3 (en) 2005-04-21

Family

ID=34104105

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2004/003218 WO2005010634A2 (en) 2003-07-30 2004-07-29 Digital pen function control

Country Status (3)

Country Link
US (1) US20050024346A1 (en)
DE (1) DE112004001404T5 (en)
WO (1) WO2005010634A2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2428124A (en) * 2005-07-07 2007-01-17 Hewlett Packard Development Co Digital pen correlating handwriting with speech
GB2428125A (en) * 2005-07-07 2007-01-17 Hewlett Packard Development Co Digital pen with speech input
EP2045695A1 (en) * 2007-10-01 2009-04-08 Siemens Aktiengesellschaft Intuitive data transmission to a mobile storage medium
US8243028B2 (en) 2008-06-13 2012-08-14 Polyvision Corporation Eraser assemblies and methods of manufacturing same
EP3147815A1 (en) 2015-09-23 2017-03-29 Politechnika Gdanska A system for hand-written signature and verification of its authenticity

Families Citing this family (94)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7916124B1 (en) 2001-06-20 2011-03-29 Leapfrog Enterprises, Inc. Interactive apparatus using print media
US7478756B1 (en) * 2003-03-04 2009-01-20 Data Recognition Corporation Method and apparatus for pen based data entry and storage
ES2387569T3 (en) * 2004-01-13 2012-09-26 The University Of Toledo Polarimeter sensor compensated for non-invasive birefringence
TW200531042A (en) * 2004-03-12 2005-09-16 Lite On It Corp Compact disc writer
US20060066591A1 (en) * 2004-03-17 2006-03-30 James Marggraff Method and system for implementing a user interface for a device through recognized text and bounded areas
US20060078866A1 (en) * 2004-03-17 2006-04-13 James Marggraff System and method for identifying termination of data entry
GB2412153A (en) * 2004-03-20 2005-09-21 Hewlett Packard Development Co Digital pen with a memory tag reader/writer
GB0417069D0 (en) * 2004-07-30 2004-09-01 Hewlett Packard Development Co Methods, apparatus and software for validating entries made on a form
US7240849B2 (en) * 2004-08-27 2007-07-10 Hewlett-Packard Development Company, L.P. Glyph pattern generation and glyph pattern decoding
US20060109262A1 (en) * 2004-11-19 2006-05-25 Ming-Hsiang Yeh Structure of mouse pen
US7460111B2 (en) * 2005-03-02 2008-12-02 Microsoft Corporation Computer input device
US8433751B2 (en) * 2005-03-08 2013-04-30 Hewlett-Packard Development Company, L.P. System and method for sharing notes
FI20055111A0 (en) * 2005-03-11 2005-03-11 Nokia Corp Creating information for a calendar application in an electronic device
US8102383B2 (en) 2005-03-18 2012-01-24 The Invention Science Fund I, Llc Performing an action with respect to a hand-formed expression
US8749480B2 (en) * 2005-03-18 2014-06-10 The Invention Science Fund I, Llc Article having a writing portion and preformed identifiers
US8229252B2 (en) 2005-03-18 2012-07-24 The Invention Science Fund I, Llc Electronic association of a user expression and a context of the expression
US8232979B2 (en) 2005-05-25 2012-07-31 The Invention Science Fund I, Llc Performing an action with respect to hand-formed expression
US8823636B2 (en) 2005-03-18 2014-09-02 The Invention Science Fund I, Llc Including environmental information in a manual expression
US7873243B2 (en) 2005-03-18 2011-01-18 The Invention Science Fund I, Llc Decoding digital information included in a hand-formed expression
US8340476B2 (en) * 2005-03-18 2012-12-25 The Invention Science Fund I, Llc Electronic acquisition of a hand formed expression and a context of the expression
US7809215B2 (en) 2006-10-11 2010-10-05 The Invention Science Fund I, Llc Contextual information encoded in a formed expression
US8640959B2 (en) * 2005-03-18 2014-02-04 The Invention Science Fund I, Llc Acquisition of a user expression and a context of the expression
US8290313B2 (en) * 2005-03-18 2012-10-16 The Invention Science Fund I, Llc Electronic acquisition of a hand formed expression and a context of the expression
JP2007072555A (en) * 2005-09-05 2007-03-22 Sony Corp Input pen
US20070009184A1 (en) * 2005-07-05 2007-01-11 Bhogal Kulvir S Associating attributes to captured input within a digital pen
US7922099B1 (en) 2005-07-29 2011-04-12 Leapfrog Enterprises, Inc. System and method for associating content with an image bearing surface
US20070030257A1 (en) * 2005-08-04 2007-02-08 Bhogal Kulvir S Locking digital pen
US7281664B1 (en) * 2005-10-05 2007-10-16 Leapfrog Enterprises, Inc. Method and system for hierarchical management of a plurality of regions of an encoded surface used by a pen computer
GB2432341B (en) * 2005-10-29 2009-10-14 Hewlett Packard Development Co Marking material
GB2432233B (en) * 2005-10-29 2011-04-20 Hewlett Packard Development Co User-interface system, method & apparatus
GB2432929A (en) * 2005-11-25 2007-06-06 Hewlett Packard Development Co Paper calendar employing digital pen input provides notification of appointment conflicts
US7411518B2 (en) * 2006-02-06 2008-08-12 Novation Science, Llc Parking location reminder device
US8261967B1 (en) 2006-07-19 2012-09-11 Leapfrog Enterprises, Inc. Techniques for interactively coupling electronic content with printed media
US7760915B2 (en) * 2006-10-09 2010-07-20 International Business Machines Corporation Method, system, and program product for encrypting information
US8000532B2 (en) * 2007-02-28 2011-08-16 Medtronic, Inc. Digital pen to capture data in ambulatory monitored patients
US20080294457A1 (en) * 2007-05-25 2008-11-27 Cordery Robert A Real-time medical records
US20090021495A1 (en) * 2007-05-29 2009-01-22 Edgecomb Tracy L Communicating audio and writing using a smart pen computing system
US8638319B2 (en) * 2007-05-29 2014-01-28 Livescribe Inc. Customer authoring tools for creating user-generated content for smart pen applications
KR20100029219A (en) * 2007-05-29 2010-03-16 라이브스크라이브, 인크. Multi-modal smartpen computing system
US8374992B2 (en) 2007-05-29 2013-02-12 Livescribe, Inc. Organization of user generated content captured by a smart pen computing system
JP2009193323A (en) * 2008-02-14 2009-08-27 Sharp Corp Display apparatus
US20090251441A1 (en) * 2008-04-03 2009-10-08 Livescribe, Inc. Multi-Modal Controller
US20090251440A1 (en) * 2008-04-03 2009-10-08 Livescribe, Inc. Audio Bookmarking
US8944824B2 (en) * 2008-04-03 2015-02-03 Livescribe, Inc. Multi-modal learning system
US9058067B2 (en) * 2008-04-03 2015-06-16 Livescribe Digital bookclip
US20090251338A1 (en) * 2008-04-03 2009-10-08 Livescribe, Inc. Ink Tags In A Smart Pen Computing System
US20090277697A1 (en) * 2008-05-09 2009-11-12 Smart Technologies Ulc Interactive Input System And Pen Tool Therefor
US20090309854A1 (en) * 2008-06-13 2009-12-17 Polyvision Corporation Input devices with multiple operating modes
US9280913B2 (en) * 2009-07-10 2016-03-08 Lincoln Global, Inc. Systems and methods providing enhanced education and training in a virtual reality environment
US9134798B2 (en) 2008-12-15 2015-09-15 Microsoft Technology Licensing, Llc Gestures, interactions, and common ground in a surface computing environment
JP5531412B2 (en) * 2009-01-14 2014-06-25 シャープ株式会社 Electronic device and information processing method
US8441668B2 (en) * 2009-04-08 2013-05-14 Palo Alto Research Center Incorporated Permanence estimation and policy enforcement for transient printing
US20110041052A1 (en) * 2009-07-14 2011-02-17 Zoomii, Inc. Markup language-based authoring and runtime environment for interactive content platform
WO2012005688A1 (en) * 2010-07-06 2012-01-12 T-Data Systems (S) Pte Ltd Data storage device with data input function
US9760241B1 (en) * 2010-11-05 2017-09-12 Amazon Technologies, Inc. Tactile interaction with content
US8982045B2 (en) 2010-12-17 2015-03-17 Microsoft Corporation Using movement of a computing device to enhance interpretation of input events produced when interacting with the computing device
US9244545B2 (en) 2010-12-17 2016-01-26 Microsoft Technology Licensing, Llc Touch and stylus discrimination and rejection for contact sensitive computing devices
US8988398B2 (en) 2011-02-11 2015-03-24 Microsoft Corporation Multi-touch input device with orientation sensing
US8660978B2 (en) 2010-12-17 2014-02-25 Microsoft Corporation Detecting and responding to unintentional contact with a computing device
US8994646B2 (en) 2010-12-17 2015-03-31 Microsoft Corporation Detecting gestures involving intentional movement of a computing device
US9201520B2 (en) 2011-02-11 2015-12-01 Microsoft Technology Licensing, Llc Motion and context sharing for pen-based computing inputs
WO2012115106A1 (en) * 2011-02-25 2012-08-30 シャープ株式会社 Pen tablet device, and pen tablet system
US20160147314A1 (en) * 2011-04-11 2016-05-26 Ali Mohammad Bujsaim Talking notebook
US20160148517A1 (en) * 2011-04-11 2016-05-26 Ali Mohammad Bujsaim Talking notebook with projection
US8902181B2 (en) 2012-02-07 2014-12-02 Microsoft Corporation Multi-touch-movement gestures for tablet computing devices
US9013425B2 (en) * 2012-02-23 2015-04-21 Cypress Semiconductor Corporation Method and apparatus for data transmission via capacitance sensing device
KR20130107473A (en) * 2012-03-22 2013-10-02 삼성전자주식회사 Capactive type touch pen
US8963891B2 (en) 2012-04-26 2015-02-24 Blackberry Limited Method and apparatus for drawing tool selection
KR102043146B1 (en) * 2012-08-02 2019-11-11 엘지전자 주식회사 Mobile terminal and electronic note system using the mobile terminal
KR101973634B1 (en) * 2012-08-23 2019-04-29 엘지전자 주식회사 Mobile terminal and control method thereof
KR102078748B1 (en) * 2013-01-29 2020-02-19 삼성전자주식회사 Method for inputting for character in flexible display an electronic device thereof
US9075464B2 (en) * 2013-01-30 2015-07-07 Blackberry Limited Stylus based object modification on a touch-sensitive display
JP5987726B2 (en) * 2013-02-22 2016-09-07 富士ゼロックス株式会社 Information processing apparatus and information processing program
US9478146B2 (en) * 2013-03-04 2016-10-25 Xerox Corporation Method and system for capturing reading assessment data
US9891722B2 (en) * 2013-03-11 2018-02-13 Barnes & Noble College Booksellers, Llc Stylus-based notification system
US11269431B2 (en) * 2013-06-19 2022-03-08 Nokia Technologies Oy Electronic-scribed input
US20150116283A1 (en) * 2013-10-24 2015-04-30 Livescribe Inc. Paper Strip Presentation Of Grouped Content
US9232409B2 (en) 2013-12-12 2016-01-05 Microsoft Technology Licensing, Llc Binding of an apparatus to a computing device
US9402169B2 (en) * 2014-03-20 2016-07-26 Google Technology Holdings, LLC Method on a writing instrument having wireless communication capability
US9827809B2 (en) * 2014-05-21 2017-11-28 Lauren Michelle Neubauer Digital pen with enhanced educational feedback
US9727161B2 (en) 2014-06-12 2017-08-08 Microsoft Technology Licensing, Llc Sensor correlation for pen and touch-sensitive computing device interaction
US9870083B2 (en) 2014-06-12 2018-01-16 Microsoft Technology Licensing, Llc Multi-device multi-user sensor correlation for pen and computing device interaction
KR102336983B1 (en) * 2014-12-30 2021-12-08 엘지전자 주식회사 Pen type multi-media device processing image data by handwriting input and control method trerof
US20160250706A1 (en) * 2015-02-27 2016-09-01 Illinois Tool Works Inc. Welding system providing remote storage of video weld data
KR102333720B1 (en) * 2015-04-09 2021-12-01 삼성전자주식회사 Digital Pen, Touch System, and Method for providing information thereof
US11016581B2 (en) 2015-04-21 2021-05-25 Microsoft Technology Licensing, Llc Base station for use with digital pens
WO2016204338A1 (en) * 2015-06-19 2016-12-22 엘지전자(주) Electronic device
KR20170001036A (en) * 2015-06-25 2017-01-04 엘지전자 주식회사 Electronic device and control method of the same
US10877575B2 (en) * 2017-03-06 2020-12-29 Microsoft Technology Licensing, Llc Change of active user of a stylus pen with a multi user-interactive display
US10657326B2 (en) * 2017-05-23 2020-05-19 International Business Machines Corporation Removable spell checker device
US20210216185A1 (en) * 2017-07-07 2021-07-15 Hewlett-Packard Development Company, L.P. Electronic pens with sensors coupled to communicative tips
JP1612493S (en) * 2017-07-10 2018-09-03
US10198094B1 (en) 2017-08-02 2019-02-05 Microsoft Technology Licensing, Llc Active stylus
IT202100022136A1 (en) * 2021-08-20 2023-02-20 Webgas S R L Pen for writing

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4751741A (en) * 1984-07-19 1988-06-14 Casio Computer Co., Ltd. Pen-type character recognition apparatus
US5247137A (en) * 1991-10-25 1993-09-21 Mark Epperson Autonomous computer input device and marking instrument
US6081261A (en) * 1995-11-01 2000-06-27 Ricoh Corporation Manual entry interactive paper and electronic document handling and processing system
US6232962B1 (en) * 1998-05-14 2001-05-15 Virtual Ink Corporation Detector assembly for use in a transcription system
WO2001061454A1 (en) * 2000-02-18 2001-08-23 Anoto Ab Controlling an electronic device
US6502756B1 (en) * 1999-05-28 2003-01-07 Anoto Ab Recording of information
WO2003058394A2 (en) * 2001-12-26 2003-07-17 Firooz Ghassabian Stylus computer and method of manipulating data

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5345543A (en) * 1992-11-16 1994-09-06 Apple Computer, Inc. Method for manipulating objects on a computer display
US5578813A (en) * 1995-03-02 1996-11-26 Allen; Ross R. Freehand image scanning device which compensates for non-linear movement
US5686720A (en) * 1995-03-02 1997-11-11 Hewlett Packard Company Method and device for achieving high contrast surface illumination
US5973677A (en) * 1997-01-07 1999-10-26 Telxon Corporation Rechargeable, untethered electronic stylus for computer with interactive display screen
US6563494B1 (en) * 1998-10-08 2003-05-13 International Business Machines Corporation Cut and paste pen for pervasive computing devices
US7015901B2 (en) * 1999-10-25 2006-03-21 Silverbrook Research Pty Ltd Universal pen with code sensor
US6380927B1 (en) * 1999-11-17 2002-04-30 Microsoft Corporation Determining the position of a detented optical encoder
US6529189B1 (en) * 2000-02-08 2003-03-04 International Business Machines Corporation Touch screen stylus with IR-coupled selection buttons
US6462330B1 (en) * 2000-03-24 2002-10-08 Microsoft Corporation Cover with integrated lens for integrated chip optical sensor
US7176906B2 (en) * 2001-05-04 2007-02-13 Microsoft Corporation Method of generating digital ink thickness information
US7109979B2 (en) * 2002-02-08 2006-09-19 Virtual Ink Corporation System and method for recording writing performed on a surface
US7136052B1 (en) * 2002-02-28 2006-11-14 Palm, Inc. Bi-stable stylus for use as an input aid
US7158675B2 (en) * 2002-05-14 2007-01-02 Microsoft Corporation Interfacing with ink
US7137076B2 (en) * 2002-07-30 2006-11-14 Microsoft Corporation Correcting recognition results associated with user input
US7009594B2 (en) * 2002-10-31 2006-03-07 Microsoft Corporation Universal computing device
US6985138B2 (en) * 2003-08-29 2006-01-10 Motorola, Inc. Input writing device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4751741A (en) * 1984-07-19 1988-06-14 Casio Computer Co., Ltd. Pen-type character recognition apparatus
US5247137A (en) * 1991-10-25 1993-09-21 Mark Epperson Autonomous computer input device and marking instrument
US6081261A (en) * 1995-11-01 2000-06-27 Ricoh Corporation Manual entry interactive paper and electronic document handling and processing system
US6232962B1 (en) * 1998-05-14 2001-05-15 Virtual Ink Corporation Detector assembly for use in a transcription system
US6502756B1 (en) * 1999-05-28 2003-01-07 Anoto Ab Recording of information
WO2001061454A1 (en) * 2000-02-18 2001-08-23 Anoto Ab Controlling an electronic device
WO2003058394A2 (en) * 2001-12-26 2003-07-17 Firooz Ghassabian Stylus computer and method of manipulating data

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2428124A (en) * 2005-07-07 2007-01-17 Hewlett Packard Development Co Digital pen correlating handwriting with speech
GB2428125A (en) * 2005-07-07 2007-01-17 Hewlett Packard Development Co Digital pen with speech input
US7676092B2 (en) 2005-07-07 2010-03-09 Hewlett-Packard Development Company, L.P. Data input apparatus and method
GB2428124B (en) * 2005-07-07 2010-04-14 Hewlett Packard Development Co Data input apparatus and method
EP2045695A1 (en) * 2007-10-01 2009-04-08 Siemens Aktiengesellschaft Intuitive data transmission to a mobile storage medium
US8243028B2 (en) 2008-06-13 2012-08-14 Polyvision Corporation Eraser assemblies and methods of manufacturing same
EP3147815A1 (en) 2015-09-23 2017-03-29 Politechnika Gdanska A system for hand-written signature and verification of its authenticity

Also Published As

Publication number Publication date
WO2005010634A3 (en) 2005-04-21
DE112004001404T5 (en) 2006-06-29
US20050024346A1 (en) 2005-02-03

Similar Documents

Publication Publication Date Title
US20050024346A1 (en) Digital pen function control
JP5285091B2 (en) Note pad
US7773377B2 (en) Integrated personal information management system
JP5470051B2 (en) Note capture device
US9648287B2 (en) Note capture device
EP3291075B1 (en) Hand-written information inputting apparatus and hand-written information inputting method
US20080181501A1 (en) Methods, Apparatus and Software for Validating Entries Made on a Form
US8577148B2 (en) Image combining apparatus and image combining program
US20030005003A1 (en) Information processing apparatus
CA2532447A1 (en) System and method for identifying termination of data entry
JP2004164609A (en) Universal input device
JP2018037033A5 (en)
US20140152543A1 (en) System, data providing method and electronic apparatus
WO2003058394A2 (en) Stylus computer and method of manipulating data
JP2004110571A (en) Procedure system, its server device, and business form for electronic pen
JP5633448B2 (en) Entry support system, terminal device, program
JP2008181510A (en) Handwritten entry information collection and management system using digital pen
EP1374012A2 (en) Stylus computer
JP2004110499A (en) Time information input system, time information acquiring device and program, and business form for electronic pen
JP5678757B2 (en) Entry support system, terminal device, program
JP5776382B2 (en) Computer apparatus and program, and notebook
JP4990704B2 (en) Document position information processing system and document position information processing method
JP5768523B2 (en) Information processing system and program
SE516739C2 (en) Notepad for information management system, has activation icon that enables a position code detector to initiate a predetermined operation that utilizes the recorded information
JP2013025612A (en) User interface creation device and user interface creation program

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase