US20150127256A1 - Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers - Google Patents

Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers Download PDF

Info

Publication number
US20150127256A1
US20150127256A1 US14/533,519 US201414533519A US2015127256A1 US 20150127256 A1 US20150127256 A1 US 20150127256A1 US 201414533519 A US201414533519 A US 201414533519A US 2015127256 A1 US2015127256 A1 US 2015127256A1
Authority
US
United States
Prior art keywords
blind
wayfinding
user
landmarks
ready
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/533,519
Inventor
Joseph Cioffi
Philip Agee
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
InTouch Graphics Inc
Original Assignee
InTouch Graphics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by InTouch Graphics Inc filed Critical InTouch Graphics Inc
Priority to US14/533,519 priority Critical patent/US20150127256A1/en
Publication of US20150127256A1 publication Critical patent/US20150127256A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00Appliances for aiding patients or disabled persons to walk about
    • A61H3/06Walking aids for blind persons
    • A61H3/061Walking aids for blind persons with electronic detecting or guiding means
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20Instruments for performing navigational calculations
    • G01C21/206Instruments for performing navigational calculations specially adapted for indoor navigation
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34Route searching; Route guidance
    • G01C21/3453Special cost functions, i.e. other than distance or default speed limit of road segments
    • G01C21/3476Special cost functions, i.e. other than distance or default speed limit of road segments using point of interest [POI] information, e.g. a route passing visible POIs
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34Route searching; Route guidance
    • G01C21/36Input/output arrangements for on-board computers
    • G01C21/3605Destination input or retrieval
    • G01C21/3608Destination input or retrieval using speech input, e.g. using speech recognition
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34Route searching; Route guidance
    • G01C21/36Input/output arrangements for on-board computers
    • G01C21/3626Details of the output of route guidance instructions
    • G01C21/3629Guidance using speech or audio output, e.g. text-to-speech
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3863Structures of map data
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H2201/00Characteristics of apparatus not provided for in the preceding codes
    • A61H2201/50Control means thereof
    • A61H2201/5023Interfaces to the user
    • A61H2201/5048Audio interfaces, e.g. voice or music controlled

Definitions

  • Various embodiments of the present invention concern systems and methods for providing wayfinding information to blind and visually impaired pedestrians.
  • RFID radio-frequency identification
  • GPS Global Positioning System
  • the present inventors have recognized a need for better ways of providing routing information to blind and visually impaired pedestrians.
  • One exemplary system provides a database of preconfigured narrative wayfinding data suitable for blind navigation.
  • the system receives user input identifying a starting landmark and corresponding narrative wayfinding data, and then outputs the wayfinding ending landmark in a particular geographic region, searches the database for the information, in the form of text or audio, for guiding a blind pedestrian from the starting landmark to the ending landmark.
  • blind users select the geographic region as well as the starting and ending landmark from a voice-driven telephonic menu system, accessible via a mobile telephone, personal digital assistant, or computer.
  • the system also provides indoor wayfinding data to help blind users walk from an entry point of a landmark building or facility to one or more offices, retail shops, offices, automatic teller machines, elevators, restrooms, or other points of interest within facility. Still other embodiments provide options for the user to access voice-driven food menus for restaurants or other types of menus, such as government service menus within a facility.
  • the exemplary system is provided as a free service to all users with an internet connection, telephone, or cell phone. Facility and building owners provide financial support for the system.
  • FIG. 1 is a block diagram of an exemplary blind pedestrian wayfinding system corresponding to one or more embodiments of the present invention.
  • FIG. 2 is a flow chart of an exemplary method of operating a blind pedestrian wayfinding system, corresponding to one or more embodiments of the present invention.
  • FIG. 3 is a facsimile of an exemplary graphical user interface 300 , which corresponds to one or more embodiments of the present invention.
  • FIG. 4 is a facsimile of an exemplary graphical user interface 400 , which corresponds to one or more embodiments of the present invention.
  • Orientation and mobility refers to a body of knowledge and the specific techniques within the “blindness education” domain that involve skills needed for a blind traveler to develop an awareness of the environment, and the ability to travel through it safely and independently.
  • Cognitive map refers to the mental processing involved in acquiring, storing, and retrieving information having to do with one's physical or spatial environment.
  • Constant contact technique refers to method of using a long (white) cane technique whereby the cane tip makes continuous contact with the walking surface as it moves rhythmically side-to-side over in a distance slightly greater than the physical body width of the individual. This technique is used to guarantee that a blind traveler will detect any changes of elevation in the walking path.
  • Wayfinding refers to the navigational methods used by an individual to travel from one location to another. With respect to “blindness wayfinding”, it refers to the specific landmarking and safety techniques that a blind individual needs in order to accomplish this kind of deliberate navigation safely and independently.
  • Landmarks and landmarking are an object that has an auditory, olfactory, proprioceptive or tactile quality and that has a known permanent location in an environment.
  • a landmark must be constant and permanent to qualify as a useful orientation aid for a blind traveler. Once these landmarks are identified and brought to the attention of a blind traveler, they can be used: to establish and maintain directional orientation; as a reference point to establish distance relationships in locating a specific object, to orient or reorient oneself to an area, to determine parallel or perpendicular alignment for straight line travel.
  • Narrative map also referred to as verbal, text-based, or auditory maps:
  • a narrative map is a verbal or text-based description that assists someone in developing a mental representation of a particular environment, or provides the step-by-step instructions required to follow and maintain orientation along a walking route.
  • Trailing Involves the use of one's fingers, a cane, or some residual vision, to follow a surface or guideline along a walking route.
  • Exemplary surfaces include walls, grassline edges, driveways, curbs, etc.
  • Tactile map an image that uses raised surfaces and textures that represent an indoor or outdoor environment and that assist in the development of a spatial map.
  • Tactile maps can be designed with both Braille and print features.
  • FIG. 1 shows an exemplary blind wayfinding data system 100 .
  • System 100 includes blind wayfinding databases 110 , a blind wayfinding data servers 120 , and access devices 130 .
  • Blind wayfinding data sources 110 include an indoor-outdoor blind wayfinding database 112 , and a destination menu database 114 .
  • Indoor-outdoor blind wayfinding database 112 includes indoor-outdoor blind wayfinding route data in the form of narrative walking routes between two outdoor points or landmarks (such as intersections, buildings, facilities) or between two indoor points or facility features, such as entry doors, offices, elevators, restrooms. In sonic embodiments, routes can encompass both indoor and outdoor landmarks or features.
  • Database 112 which for example takes the form of a SQL database, includes one or more narrative maps.
  • each narrative map takes the form of a set of one or more textual and/or audio instructions, and is prepared by blind mobility specialists, incorporating terminology, technique recommendations, and landmarking cues that work for blind travelers.
  • An exemplary 6-step narrative map is provided in the table below.
  • Narrative map data is stored in tables in a relational database, which are generally representative of narrative map data structures, of which data structure 1121 is generally representative.
  • Data structure 1121 includes a location identifier 1121 A, a starting point identifier 1121 B, an end or destination point identifier 1121 C, and narrative text instructions 1121 D.
  • Location identifier 1121 A uniquely identifies a geographic region or facility, such as University of Minnesota, which is associated with starting point identifier 1121 B and ending point identifiers 1121 C.
  • Narrative text instructions 1121 D is itself logically associated with a total steps indicator for indicating the number of steps in the instructional sequence, a time stamp indicator for indicating the last time the narrative instructions were updated, an author indicator for indicating the author(s) of the narrative instruction, as well as one or more narrative step instructions, such as step 1 . Portions of each instructions are further associated with tags or corresponding fields to identify portions of instructions that are intended to support blind pedestrians.
  • each step instruction is associated with a GPS coordinates to facilitate use of other data and functions that may be correlated to such coordinates.
  • this alloy s synchronized output of the instruction as text, Braille, or audio based on real-time measured or estimated position of the user.
  • other embodiments allow the user to advance, pause, or backup to replay presentation of a narrative map using a voice or manually input command.
  • Some embodiments support GPS interfacing. This enables users having mobile phones or other devices with GPS or analogous capability to user such GPS data as an input to the wayfinding system. For example, if a user is positioned at a landmark correlated to current positional coordinates, the system can receive a generic command such as “from here, how do I get to the library,” and have the system be able to correlate based on GPS or cell phone tower location data or Wife location data to the appropriate set of directions, with appropriate user confirmation input or preference data to resolve any ambiguities. In some instances, the positional data can be used to negate or defeat presentation of certain menu options.
  • the narrative maps are converted by a multimedia encoding system, into multimedia files using encoding servers as part of the textual data entry process for the narrative maps.
  • the encoding system creates multi-media files of the step-by-step directions and other location information using codecs for MPEG (mp3).
  • MPEG MPEG
  • Adobe Flash flv
  • Microsoft Windows Media wmv
  • Apple Quicktime mov
  • Destination menu database 114 stores one or more textual restaurant (or more generally government or business service) menus.
  • the menus are stored in the form of one or more data structures, of which menu data structure 1141 is generally representative.
  • Data structure 1141 includes a menu identifier field 1141 A, a location identifier 1141 B for associating the menu with a particular geographic region, an end point identifier 1141 C for associating the menu with a particular endpoint or destination within the geographic region, and a time stamp identifier for indicating when the menu was last updated.
  • the data structure includes a menu narrative field 1141 E including the text of one of more items, such as food or beverage items.
  • Each listed item is associated with one or more category tags, such as entrée, soup, salad, beverage, wheat, eggs, (other known allergens), and as well as pricing data tags.
  • category tags such as entrée, soup, salad, beverage, wheat, eggs, (other known allergens), and as well as pricing data tags.
  • these allow for sorting and eliminating menu items from presentation to a given user based on stored dietary and price preferences.
  • Some embodiments also store the narrative menus as audio files, or in association with a preferred text-to-speech synthesizer for the restaurant or business associated with the menu.
  • Data sources 110 are coupled or couplable via a wireless or wireline communications network, to wayfinding server 120 .
  • Wayfinding data server 120 which provides blind wayfinding and menu data to blind and visually impaired pedestrian users, includes a processor module 121 , a memory module 122 , a subscriber database module 123 , a wayfinding service module 124 , and a menu module 125 .
  • Processor module 121 includes one or more local or distributed processors, controllers, or virtual machines. In the exemplary embodiment, processor module 121 assumes any convenient or desirable form. In some embodiments, one or more of the processors are incorporated into servers.
  • Memory module 122 which takes the exemplary form of one or more electronic, magnetic, or optical data-storage devices, stores machine-readable instructions that when executed by one or more processors, performs one or more of the processes and/or methods as described herein.
  • subscriber module 123 includes one or more sets of machine-readable and/or executable instructions for collecting and storing user account or subscriber data for blind users. (Some embodiments also include sighted or non-blind users.) To this end, module includes one or more data structures, of which subscriber data structure 1231 is representative. Data structure 123 includes a unique identification portion 1231 A, which is logically associated with one or more fields, such as fields 1231 B, 1231 C, 1231 D, 1231 E, and 1231 F.
  • Field 1231 B includes a user account data, such as username and passwords, contact data (such as mobile telephone number and email address), and credit card billing, information; field 1231 C includes travel preference information, such as preferred locations (geographic regions), starting points or landmarks, and ending points or destinations.
  • Field 1231 D includes other preferences, such as dietary preferences, price preferences, user allergens, and so forth.
  • Field 1231 E includes user generated or private narrative map information, which are not generally available to other users.
  • Field 1231 F includes a vision status field, which designates the user as sighted or blind, and enables the system to filter out or leave in blind portions of narrative map data.
  • wayfinding service module 124 includes one or more sets of machine-readable and/or executable instructions for receiving user requests for wayfinding data, searching databases 110 , and outputting wayfinding data (narrative maps) to an access device.
  • wayfinding service module 124 includes a speech recognizer 1241 , a search engine 1242 , a text-to-speech converter 1243 , a telephony module 1244 , and a web/http interface module 1245 .
  • Speech Recognizer/responder 1241 receives voice commands and requests from users, for example locations, starting points, and destinations and provides query structures to search engine 124 .
  • Search engine 124 communicates the requests to databases 112 , receives the results in textual form, for example, and forward them to text-to-speech module 1243 for conversion and output to telephony module 1244 for communication with an access device having a telephone capability.
  • Exemplary telephony capabilities include Voice-Over-Internet-Protocol (VOIP) and automated voice response systems.
  • Web interface module 1245 provides web interface functionality and related graphical user interfaces for receives and fulfilling requests via an HTTP protocol. Text-based and graphical interfaces including web pages consisting of HTML, AJAX, Javascript, CSS over HTTP. Web interface module 1245 also supports entering and displaying narrative map data using HTML forms. Hyperlinks on web pages provide access to multimedia files for downloading, podcasts, RSS text feeds, and RSS audio feeds. Web pages also provide access to streaming of multimedia map data.
  • Menu module 125 includes one or more sets of machine-readable and/or executable instructions for receiving user requests for destination menu data, searching databases 110 (specifically destination menu data 114 ), and outputting menu data to an access device, as discussed herein, based on user preferences. Additionally, in some embodiments, menu module 125 includes instructions for playing back selected menu options, calculating purchases, and conducting secure credit card transactions based on user selected menu options.
  • Server 120 interacts via a wireless or wireline communications network with one or more accesses devices, such as access device 130 .
  • Access device 130 is generally representative of one or more access devices.
  • access device 130 takes the form of a personal computer, workstation, personal digital assistant, mobile telephone, or any other device capable of providing an effective user interface with a server or database.
  • access device 130 includes a processor module 131 one or more processors (or processing circuits) 131 , a memory 132 , a display/loudspeaker 133 , a keypad or board 134 , and user input devices 135 , such as graphical pointer or selector 135 A and microphone 135 B.
  • Processor module 131 includes one or more processors, processing circuits, or controllers. In the exemplary embodiment, processor module 131 takes any convenient or desirable form. Coupled to processor module 131 is memory 132 .
  • Memory 132 stores code (machine-readable or executable instructions) for an operating system 136 , a browser 137 , and a graphical user interface (GUI) 138 .
  • operating system 136 takes the form of a version of the Microsoft Windows operating system
  • browser 137 takes the form of a version of Microsoft Internet Explorer.
  • Operating system 136 and browser 137 not only receive inputs from keyboard 134 and selector 135 , but also support rendering of GUI 138 on display 133 .
  • GUI 138 presents data in association with one or more interactive control features (or user-interface elements), as shown for example in FIGS. 3 and 4 and further described below.
  • the exemplary embodiment defines one or more portions of interface 138 using applets or other programmatic objects or structures from server 120 to implement the interfaces shown or described elsewhere in this description.
  • FIG. 2 shows a flow chart 200 of one or more exemplary methods of operating a system, such as system 100 .
  • Flow chart 200 includes blocks 210 - 299 , which are arranged and described in a serial execution sequence in the exemplary embodiment.
  • other embodiments execute two or more blocks in parallel using multiple processors or processor-like devices or a single processor organized as two or more virtual machines or sub processors.
  • Other embodiments also alter the process sequence or provide different functional partitions to achieve analogous results.
  • some embodiments may alter the client-server, allocation of functions, such that functions shown and described on the server side are implemented in whole or in part on the client side, and vice versa.
  • still other embodiments implement the blocks as two or more interconnected hardware modules with related control and data signals communicated between and through the modules.
  • the exemplary process flow applies to software, hardware, and firmware implementations.
  • the exemplary method begins with collecting and organizing narrative map and restaurant menu data.
  • this entails expert collection and drafting of narrative map data for various locations.
  • the narrative map data is uploaded as text into the database.
  • various portions of the narrative map data is tagged to facilitate use, pursuant to data structure 1121 .
  • the particular map itself is tagged with starting and ending landmarks, a time stamp, author, total number of steps.
  • Each step is also separately tagged or labeled with a sequence number.
  • label or tag portions of the text within each step to indicate for example that the tagged portion is a distance quantity or that the tagged portion is a blind wayfinding instruction or description. This latter tag facilitates filtering of the narrative map for use by a sighted person.
  • the exemplary embodiment also collects textual menu data and structures it according data structure 1141 . Exemplary execution continues at block 220 .
  • Block 210 entails receiving a wayfinding request from a blind pedestrian user.
  • this entails user making a selection from either a graphical user interface or via voice command menu, using a mobile telephone or personal digital assistant or personal computer.
  • the exemplary embodiment uses interfaces as shown in FIG. 3 . These interfaces guide a user to select or identify a location, such as city, state, university, airport, shopping mall other defined geographic region, or to o initiate a search of database for relevant blind-ready walking map data. Or, alternatively user selects from a drop down menu or list of predefined or dynamically determined locations.
  • the list and menus are presented after a user login, which allows the lists and menus to be based on stored user preference information, such as a stored set of favorite or most recently used locations or regions. Additionally, some embodiments are responsive to current location information, for example, from a Global Positioning System (GPS), to determine which of a set of available lists and menu options are appropriate for presentation to a user, reducing the number of selections for a user to consider or eliminating the seed for a user to make a selection at all.
  • GPS Global Positioning System
  • This screen also includes a “change page for low vision” command feature to adjust the size of the text display and/or enable voice output as the user scrolls over the selection features on the page.
  • the starting point is selected automatically based on current user position, which can be determined using positional information in the client access device associated with the user, for example, mobile telephone with WiFi or GPS positional capability. In some embodiments, this starting point is taken automatically to be the last user selected destination. Again, it may also be selected automatically based on stored user preference information.
  • Some embodiments provide wheel-chair or impaired walker accessible travel directions which direct users to ramps, railings, and other walking surfaces that are designed for use by wheel-chair or other disabled travelers. Some embodiments also allow users to look for other travel features, such as restrooms, family restrooms, diaper changing stations.
  • the selections shown here on a graphical user interface are presented as audio to the user via telephone and voice-activated technologies. Also, some embodiments allow output of the directions in audio form for selected playback on a mobile client device, as MP3 or podcast. In some embodiments, the playback of the audio directions is synched to real-time positional information from a GPS system or to n command signal, such as next given verbally or via a keypad or other manual entry from the user.
  • Block 230 entails outputting wayfinding instructions (blind ready wayfinding instructions to an access device carried or used by a blind pedestrian.
  • the output is provided as audio output via a telephony interface or as textual output via a web interface.
  • FIG. 4 show an exemplary web interface displaying textual representation of blind-ready wayfinding instructions.
  • Block 240 entails receiving a request for a menu associated with a given landmark or destination.
  • the destination is by default the destination associated with the most recently fullfilled wayfinding data request.
  • the menu is associated with GPS coordinates and the system uses the user's current GPS or similar positional information to determine the most likely menus.
  • the system output a destination menu to the user in audio and/or text form.
  • the menu is output based, on user preferences, such as dietary and/or price preferences to reduce data transfer demands and improve useability of complex menu options. Additionally some embodiments receive voice commands to filter and sort menu options. Some embodiments also allow users to eliminate items or reserve items as maybes as they are played out. This enables the user to rapidly converge on desired selections and ultimately make decisions, and/or place orders via the system.

Abstract

To support the independence and mobility of blind pedestrians, the present inventors devised, among other things, free systems, methods, and software for providing narrative blind-ready wayfinding information. One exemplary system receives user input identifying a starting landmark and ending landmark in a particular selected geographic region, such as a city, university campus, government building, shopping mall, or airport. The system then searches a database for the corresponding narrative wayfinding instructions, and outputs them in the form of text or audio to guide a blind pedestrian from the starting landmark to the ending landmark. In the exemplary system, blind users select the geographic region as well as the starting and ending landmark from a voice-driven telephonic menu system and receive audible wayfinding instruction via mobile telephone. In some embodiments, the system also provides access to voice-driven restaurant menus.

Description

    RELATED APPLICATION
  • The present application claims priority to U.S. Provisional Patent Application 61/161,356, which was filed on Mar. 18, 2009. This application is incorporated herein by reference.
  • COPYRIGHT NOTICE AND LIMITED PERMISSION
  • A portion of this patent document contains material subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyrights whatsoever. The following notice applies to this document: Copyright ©C 2009, InTouch Graphics, Inc.
  • TECHNICAL FIELD
  • Various embodiments of the present invention concern systems and methods for providing wayfinding information to blind and visually impaired pedestrians.
  • BACKGROUND
  • Blind pedestrians use a variety of travel aids. Chief among these are white canes and guide dogs. However, recent years have seen the emergence of navigational aid systems based on newer technologies, such as radio-frequency identification (RFID) tags and the Global Positioning System (GPS).
  • Although these high-tech systems provide some benefits, the present inventors have recognized that they suffer from disadvantages that have prevented widespread adoption. For example, these devices typically require purchase of specialized hardware, such as RFID readers, that is beyond the means of many in the blind and visually impaired community. Additionally, the street-based routing information provided by these systems is set up for users with normal vision, and therefore of minimal to no value to blind and visually impaired travelers.
  • Accordingly, the present inventors have recognized a need for better ways of providing routing information to blind and visually impaired pedestrians.
  • SUMMARY
  • To address this and/or other needs, the present inventors devised, among other things, systems, methods, and software for providing narrative wayfinding information to blind and visually impaired pedestrians. One exemplary system provides a database of preconfigured narrative wayfinding data suitable for blind navigation. The system receives user input identifying a starting landmark and corresponding narrative wayfinding data, and then outputs the wayfinding ending landmark in a particular geographic region, searches the database for the information, in the form of text or audio, for guiding a blind pedestrian from the starting landmark to the ending landmark. In the exemplary system, blind users select the geographic region as well as the starting and ending landmark from a voice-driven telephonic menu system, accessible via a mobile telephone, personal digital assistant, or computer. In some embodiments, the system also provides indoor wayfinding data to help blind users walk from an entry point of a landmark building or facility to one or more offices, retail shops, offices, automatic teller machines, elevators, restrooms, or other points of interest within facility. Still other embodiments provide options for the user to access voice-driven food menus for restaurants or other types of menus, such as government service menus within a facility.
  • Additionally, the exemplary system is provided as a free service to all users with an internet connection, telephone, or cell phone. Facility and building owners provide financial support for the system.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a block diagram of an exemplary blind pedestrian wayfinding system corresponding to one or more embodiments of the present invention.
  • FIG. 2 is a flow chart of an exemplary method of operating a blind pedestrian wayfinding system, corresponding to one or more embodiments of the present invention.
  • FIG. 3 is a facsimile of an exemplary graphical user interface 300, which corresponds to one or more embodiments of the present invention.
  • FIG. 4 is a facsimile of an exemplary graphical user interface 400, which corresponds to one or more embodiments of the present invention.
  • DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENT(S)
  • This document, which incorporates the drawings and the appended claims, describes one or more specific embodiments of an invention. These embodiments, offered not to limit but only to exemplify and teach the invention, are shown and described in sufficient detail to enable those skilled in the art to implement or practice the invention. Thus, where appropriate to avoid obscuring the invention, the description may omit certain information known to those of skill in the art.
  • Exemplary Definitions
  • As an aid to understanding the following description, the following exemplary definitions are provided:
  • Orientation and mobility refers to a body of knowledge and the specific techniques within the “blindness education” domain that involve skills needed for a blind traveler to develop an awareness of the environment, and the ability to travel through it safely and independently.
  • Cognitive map: refers to the mental processing involved in acquiring, storing, and retrieving information having to do with one's physical or spatial environment.
  • “constant contact technique”refers to method of using a long (white) cane technique whereby the cane tip makes continuous contact with the walking surface as it moves rhythmically side-to-side over in a distance slightly greater than the physical body width of the individual. This technique is used to guarantee that a blind traveler will detect any changes of elevation in the walking path.
  • Wayfinding—refers to the navigational methods used by an individual to travel from one location to another. With respect to “blindness wayfinding”, it refers to the specific landmarking and safety techniques that a blind individual needs in order to accomplish this kind of deliberate navigation safely and independently.
  • Landmarks and landmarking (as they relate to wayfinding for the blind): A landmark is an object that has an auditory, olfactory, proprioceptive or tactile quality and that has a known permanent location in an environment. A landmark must be constant and permanent to qualify as a useful orientation aid for a blind traveler. Once these landmarks are identified and brought to the attention of a blind traveler, they can be used: to establish and maintain directional orientation; as a reference point to establish distance relationships in locating a specific object, to orient or reorient oneself to an area, to determine parallel or perpendicular alignment for straight line travel.
  • Narrative map (also referred to as verbal, text-based, or auditory maps): A narrative map is a verbal or text-based description that assists someone in developing a mental representation of a particular environment, or provides the step-by-step instructions required to follow and maintain orientation along a walking route.
  • Trailing: Involves the use of one's fingers, a cane, or some residual vision, to follow a surface or guideline along a walking route. Exemplary surfaces include walls, grassline edges, driveways, curbs, etc.
  • Tactile map: an image that uses raised surfaces and textures that represent an indoor or outdoor environment and that assist in the development of a spatial map. Tactile maps can be designed with both Braille and print features.
  • Exemplary Blind Wayfinding Data System
  • FIG. 1 shows an exemplary blind wayfinding data system 100. System 100 includes blind wayfinding databases 110, a blind wayfinding data servers 120, and access devices 130.
  • Exemplary Blind Wayfinding Data Sources
  • Blind wayfinding data sources 110 include an indoor-outdoor blind wayfinding database 112, and a destination menu database 114. Indoor-outdoor blind wayfinding database 112 includes indoor-outdoor blind wayfinding route data in the form of narrative walking routes between two outdoor points or landmarks (such as intersections, buildings, facilities) or between two indoor points or facility features, such as entry doors, offices, elevators, restrooms. In sonic embodiments, routes can encompass both indoor and outdoor landmarks or features. Database 112, which for example takes the form of a SQL database, includes one or more narrative maps.
  • In the exemplary embodiment, each narrative map takes the form of a set of one or more textual and/or audio instructions, and is prepared by blind mobility specialists, incorporating terminology, technique recommendations, and landmarking cues that work for blind travelers. An exemplary 6-step narrative map is provided in the table below.
  • Route: Coffman Memorial Hall to Moos Tower
  • 1. Exit main front doors of Coffman Memorial Hall. You are facing
    North in the direction of Washington Ave. This is a 10-foot wide pathway
    with grass edges on both left and right sides as you walk in a perpendic-
    ular direction towards Washington Ave. Proceed straight until you reach
    the Washington Ave sidewalk in 30 feet.
    2. Turn right at this intersecting sidewalk, now facing East on Washington
    Ave. Continue straight and in 50 feet you will reach the first down curb at
    the T-intersection of Church and Washington.
    3. This is a light controlled T-intersection with 2-way traffic. Cross
    straight and continue along Washington Ave.
    4. At mid-block, while following the ride side edge of this sidewalk
    (there is a curb guideline along this right edge), the sidewalk increases
    significantly in width, from 10 feet to 30 feet. This is at the 200-foot
    marker of this block.
    5. Walk striaght for another 20 feet and make a right turn, now facing
    South. Continue straight until you reach ascending stairs. Take these 10
    steps to the top landing, and walk straight 20 feet until you find the
    perpendicular wall of a building.
    6. This is Moos Tower. Turn left and here and trail until you feel the
    double door entrance in 20 feet.
  • Narrative map data is stored in tables in a relational database, which are generally representative of narrative map data structures, of which data structure 1121 is generally representative. Data structure 1121 includes a location identifier 1121A, a starting point identifier 1121B, an end or destination point identifier 1121C, and narrative text instructions 1121D. Location identifier 1121A uniquely identifies a geographic region or facility, such as University of Minnesota, which is associated with starting point identifier 1121B and ending point identifiers 1121C. (It is assumed in the exemplary embodiment that pedestrians will be traveling within a single location or region; however, other embodiments may provide for starting point and ending points that are in separate zones by, for example, allowing a starting or ending point to be associated with more than one location identifier.) Narrative text instructions 1121D is itself logically associated with a total steps indicator for indicating the number of steps in the instructional sequence, a time stamp indicator for indicating the last time the narrative instructions were updated, an author indicator for indicating the author(s) of the narrative instruction, as well as one or more narrative step instructions, such as step 1. Portions of each instructions are further associated with tags or corresponding fields to identify portions of instructions that are intended to support blind pedestrians.
  • Additionally, one or more portions of each step instruction are associated with a GPS coordinates to facilitate use of other data and functions that may be correlated to such coordinates. For example, this alloys synchronized output of the instruction as text, Braille, or audio based on real-time measured or estimated position of the user. However, other embodiments allow the user to advance, pause, or backup to replay presentation of a narrative map using a voice or manually input command.
  • Some embodiments support GPS interfacing. This enables users having mobile phones or other devices with GPS or analogous capability to user such GPS data as an input to the wayfinding system. For example, if a user is positioned at a landmark correlated to current positional coordinates, the system can receive a generic command such as “from here, how do I get to the library,” and have the system be able to correlate based on GPS or cell phone tower location data or Wife location data to the appropriate set of directions, with appropriate user confirmation input or preference data to resolve any ambiguities. In some instances, the positional data can be used to negate or defeat presentation of certain menu options.
  • In some embodiments, the narrative maps are converted by a multimedia encoding system, into multimedia files using encoding servers as part of the textual data entry process for the narrative maps. The encoding system creates multi-media files of the step-by-step directions and other location information using codecs for MPEG (mp3). Adobe Flash (flv), Microsoft Windows Media (wmv), and Apple Quicktime (mov).
  • Destination menu database 114 stores one or more textual restaurant (or more generally government or business service) menus. The menus are stored in the form of one or more data structures, of which menu data structure 1141 is generally representative. Data structure 1141 includes a menu identifier field 1141A, a location identifier 1141B for associating the menu with a particular geographic region, an end point identifier 1141C for associating the menu with a particular endpoint or destination within the geographic region, and a time stamp identifier for indicating when the menu was last updated. Additionally, the data structure includes a menu narrative field 1141E including the text of one of more items, such as food or beverage items. Each listed item is associated with one or more category tags, such as entrée, soup, salad, beverage, wheat, eggs, (other known allergens), and as well as pricing data tags. In the exemplary embodiment these allow for sorting and eliminating menu items from presentation to a given user based on stored dietary and price preferences. (Some embodiments also store the narrative menus as audio files, or in association with a preferred text-to-speech synthesizer for the restaurant or business associated with the menu.) Data sources 110 are coupled or couplable via a wireless or wireline communications network, to wayfinding server 120.
  • Exemplary Wayfinding Server
  • Wayfinding data server 120, which provides blind wayfinding and menu data to blind and visually impaired pedestrian users, includes a processor module 121, a memory module 122, a subscriber database module 123, a wayfinding service module 124, and a menu module 125.
  • Processor module 121 includes one or more local or distributed processors, controllers, or virtual machines. In the exemplary embodiment, processor module 121 assumes any convenient or desirable form. In some embodiments, one or more of the processors are incorporated into servers.
  • Memory module 122, which takes the exemplary form of one or more electronic, magnetic, or optical data-storage devices, stores machine-readable instructions that when executed by one or more processors, performs one or more of the processes and/or methods as described herein.
  • In the exemplary embodiment, subscriber module 123 includes one or more sets of machine-readable and/or executable instructions for collecting and storing user account or subscriber data for blind users. (Some embodiments also include sighted or non-blind users.) To this end, module includes one or more data structures, of which subscriber data structure 1231 is representative. Data structure 123 includes a unique identification portion 1231A, which is logically associated with one or more fields, such as fields 1231B, 1231C, 1231D, 1231E, and 1231F. Field 1231B includes a user account data, such as username and passwords, contact data (such as mobile telephone number and email address), and credit card billing, information; field 1231C includes travel preference information, such as preferred locations (geographic regions), starting points or landmarks, and ending points or destinations. Field 1231D includes other preferences, such as dietary preferences, price preferences, user allergens, and so forth. Field 1231E includes user generated or private narrative map information, which are not generally available to other users. Field 1231F includes a vision status field, which designates the user as sighted or blind, and enables the system to filter out or leave in blind portions of narrative map data.
  • In some embodiments, wayfinding service module 124 includes one or more sets of machine-readable and/or executable instructions for receiving user requests for wayfinding data, searching databases 110, and outputting wayfinding data (narrative maps) to an access device. In particular wayfinding service module 124 includes a speech recognizer 1241, a search engine 1242, a text-to-speech converter 1243, a telephony module 1244, and a web/http interface module 1245. Speech Recognizer/responder 1241 receives voice commands and requests from users, for example locations, starting points, and destinations and provides query structures to search engine 124. Search engine 124 communicates the requests to databases 112, receives the results in textual form, for example, and forward them to text-to-speech module 1243 for conversion and output to telephony module 1244 for communication with an access device having a telephone capability. Exemplary telephony capabilities include Voice-Over-Internet-Protocol (VOIP) and automated voice response systems. Web interface module 1245 provides web interface functionality and related graphical user interfaces for receives and fulfilling requests via an HTTP protocol. Text-based and graphical interfaces including web pages consisting of HTML, AJAX, Javascript, CSS over HTTP. Web interface module 1245 also supports entering and displaying narrative map data using HTML forms. Hyperlinks on web pages provide access to multimedia files for downloading, podcasts, RSS text feeds, and RSS audio feeds. Web pages also provide access to streaming of multimedia map data.
  • Menu module 125 includes one or more sets of machine-readable and/or executable instructions for receiving user requests for destination menu data, searching databases 110 (specifically destination menu data 114), and outputting menu data to an access device, as discussed herein, based on user preferences. Additionally, in some embodiments, menu module 125 includes instructions for playing back selected menu options, calculating purchases, and conducting secure credit card transactions based on user selected menu options.
  • Server 120 interacts via a wireless or wireline communications network with one or more accesses devices, such as access device 130.
  • Exemplary Access Device
  • Access device 130 is generally representative of one or more access devices. In the exemplary embodiment, access device 130 takes the form of a personal computer, workstation, personal digital assistant, mobile telephone, or any other device capable of providing an effective user interface with a server or database. Specifically, access device 130 includes a processor module 131 one or more processors (or processing circuits) 131, a memory 132, a display/loudspeaker 133, a keypad or board 134, and user input devices 135, such as graphical pointer or selector 135A and microphone 135B.
  • Processor module 131 includes one or more processors, processing circuits, or controllers. In the exemplary embodiment, processor module 131 takes any convenient or desirable form. Coupled to processor module 131 is memory 132.
  • Memory 132 stores code (machine-readable or executable instructions) for an operating system 136, a browser 137, and a graphical user interface (GUI) 138. In the exemplary embodiment, operating system 136 takes the form of a version of the Microsoft Windows operating system, and browser 137 takes the form of a version of Microsoft Internet Explorer. Operating system 136 and browser 137 not only receive inputs from keyboard 134 and selector 135, but also support rendering of GUI 138 on display 133. Upon rendering, GUI 138 presents data in association with one or more interactive control features (or user-interface elements), as shown for example in FIGS. 3 and 4 and further described below. (The exemplary embodiment defines one or more portions of interface 138 using applets or other programmatic objects or structures from server 120 to implement the interfaces shown or described elsewhere in this description.)
  • Exemplary Method(s) of Operation
  • FIG. 2 shows a flow chart 200 of one or more exemplary methods of operating a system, such as system 100. Flow chart 200 includes blocks 210-299, which are arranged and described in a serial execution sequence in the exemplary embodiment. However, other embodiments execute two or more blocks in parallel using multiple processors or processor-like devices or a single processor organized as two or more virtual machines or sub processors. Other embodiments also alter the process sequence or provide different functional partitions to achieve analogous results. For example, some embodiments may alter the client-server, allocation of functions, such that functions shown and described on the server side are implemented in whole or in part on the client side, and vice versa. Moreover, still other embodiments implement the blocks as two or more interconnected hardware modules with related control and data signals communicated between and through the modules. Thus, the exemplary process flow applies to software, hardware, and firmware implementations.
  • At block 210, the exemplary method begins with collecting and organizing narrative map and restaurant menu data. In the exemplary embodiment, this entails expert collection and drafting of narrative map data for various locations. The narrative map data is uploaded as text into the database. In some embodiments, various portions of the narrative map data is tagged to facilitate use, pursuant to data structure 1121. For example, the particular map itself is tagged with starting and ending landmarks, a time stamp, author, total number of steps. Each step is also separately tagged or labeled with a sequence number. Moreover, some embodiments label or tag portions of the text within each step to indicate for example that the tagged portion is a distance quantity or that the tagged portion is a blind wayfinding instruction or description. This latter tag facilitates filtering of the narrative map for use by a sighted person. The exemplary embodiment also collects textual menu data and structures it according data structure 1141. Exemplary execution continues at block 220.
  • Block 210 entails receiving a wayfinding request from a blind pedestrian user. In the exemplary embodiment, this entails user making a selection from either a graphical user interface or via voice command menu, using a mobile telephone or personal digital assistant or personal computer. In the case of a graphical user interface, the exemplary embodiment uses interfaces as shown in FIG. 3. These interfaces guide a user to select or identify a location, such as city, state, university, airport, shopping mall other defined geographic region, or to o initiate a search of database for relevant blind-ready walking map data. Or, alternatively user selects from a drop down menu or list of predefined or dynamically determined locations. In some embodiments, the list and menus are presented after a user login, which allows the lists and menus to be based on stored user preference information, such as a stored set of favorite or most recently used locations or regions. Additionally, some embodiments are responsive to current location information, for example, from a Global Positioning System (GPS), to determine which of a set of available lists and menu options are appropriate for presentation to a user, reducing the number of selections for a user to consider or eliminating the seed for a user to make a selection at all. This screen also includes a “change page for low vision” command feature to adjust the size of the text display and/or enable voice output as the user scrolls over the selection features on the page.
  • After selection of University of Minnesota from map region definition page, User selects a starting point, landmark. In some embodiments, the starting point is selected automatically based on current user position, which can be determined using positional information in the client access device associated with the user, for example, mobile telephone with WiFi or GPS positional capability. In some embodiments, this starting point is taken automatically to be the last user selected destination. Again, it may also be selected automatically based on stored user preference information.
  • User then selects a destination from the drop down menu and selects go to initiate a search of database for the blind-ready map data to travel (e.g. walk) from the starting point to the destination. (Some embodiments provide wheel-chair or impaired walker accessible travel directions which direct users to ramps, railings, and other walking surfaces that are designed for use by wheel-chair or other disabled travelers. Some embodiments also allow users to look for other travel features, such as restrooms, family restrooms, diaper changing stations.
  • In some embodiments, the selections shown here on a graphical user interface, are presented as audio to the user via telephone and voice-activated technologies. Also, some embodiments allow output of the directions in audio form for selected playback on a mobile client device, as MP3 or podcast. In some embodiments, the playback of the audio directions is synched to real-time positional information from a GPS system or to n command signal, such as next given verbally or via a keypad or other manual entry from the user.
  • Block 230 entails outputting wayfinding instructions (blind ready wayfinding instructions to an access device carried or used by a blind pedestrian. In the exemplary embodiment the output is provided as audio output via a telephony interface or as textual output via a web interface. FIG. 4 show an exemplary web interface displaying textual representation of blind-ready wayfinding instructions.
  • Block 240 entails receiving a request for a menu associated with a given landmark or destination. In the exemplary embodiment, the destination is by default the destination associated with the most recently fullfilled wayfinding data request. However, in other embodiments, the menu is associated with GPS coordinates and the system uses the user's current GPS or similar positional information to determine the most likely menus.
  • In block 250, the system output a destination menu to the user in audio and/or text form. In some embodiments, the menu is output based, on user preferences, such as dietary and/or price preferences to reduce data transfer demands and improve useability of complex menu options. Additionally some embodiments receive voice commands to filter and sort menu options. Some embodiments also allow users to eliminate items or reserve items as maybes as they are played out. This enables the user to rapidly converge on desired selections and ultimately make decisions, and/or place orders via the system.

Claims (22)

1. A blind traveler navigational data system having a server, the server comprising:
a processor for receiving information identifying first and second landmarks from a set of predefined landmarks within a predefined geographic region;
a database for accessing and retrieving, blind-ready wayfinding instructions to for guiding a blind or visually impaired traveler from the first identified landmark to the second identified landmark; and
at least one communications interface for outputting the retrieved blind-ready wayfinding instructions to a blind or visually impaired user wherein the blind-ready wayfinding instructions include acoustic information specific to an environment associated with the first and second landmarks.
2. The system of claim 1, wherein the blind-ready wayfinding instructions includes walking path surface information for facilitating trailing.
3. The system of claim 1, wherein the means for identifying two of a set of landmarks includes means for receiving a selection of at least one of the first and second landmarks via a telephone-operated voice-response system.
4. The system of claim 1, wherein the means for receiving information identifying first and second landmarks includes a graphical user interface at least partially defined by the server.
5. The system of claim 1, wherein the second landmark is a building, and wherein the server further comprises means, responsive to a user command, for outputting a listing of indoor destination options associated with the second identified landmark; and means, responsive to selection of one of the indoor destination options, for accessing and retrieving from a database, blind ready wayfinding data for guiding a blind or visually impaired (legally blind) traveler from an entry door of the building to the selected one of the indoor destination options.
6. The system of claim 5, wherein the one of indoor destination option is a business that leases space within the building.
7. The system of claim 5, wherein the one of the indoor destination options is an automatic teller machine in the building.
8. The system of claim 5, wherein the one of the indoor destination options is a restroom within the building.
9. The system of claim 5, wherein the one of the indoor destination options is a wheelchair ramp within the building.
10. The system of claim 5, wherein the one of the indoor destination options is a tactile map within the building.
11. The system of claim 5, wherein the one of the indoor destination options is an elevator within the building.
12. The system of claim 5, wherein the one of the indoor destination options is an escalator within the building.
13. The system of claim 5, wherein the one of the indoor destination options is a water fountain within the building.
14. The system of claim 5, wherein the one of the indoor destination options is an restaurant within the building, and the server further includes means, responsive to a user command, for presenting at least a portion of a voice navigable voice menu of food items for the restaurant.
15. The system of claim 14, wherein the server further includes:
means, responsive to a user voice command, for receiving one or more menu selections;
means responsive to a subsequent user voice command for playing back the one or more menu selections; and
means responsive a subsequent user voice command for communicating an order based on the one or more menu selections to an order management system within the restaurant.
16. The system of claim 14, wherein the server further includes means, responsive to a user command, for eliminating one or more menu items from presentation during an access session.
17. The system of claim 16, wherein the means for eliminating is configured to receive an elimination command from a user during a presentation of a first portion of the voice menu, and during a subsequent presentation of the voice menu for the user (during the session) the first portion is not presented.
18. The system of claim 14, wherein the server includes dietary preference data for a user and means, responsive to stored dietary preference data for the user, for inhibiting playback of one or more portions of a voice navigable voice menu.
19. A method implemented using at least one processor and memory, the method comprising:
receiving into the memory information identifying first and second landmarks from a set of predefined landmarks within a predefined geographic region;
in response to the identified first and second landmarks, using the processor to access and retrieve from a database, blind-ready wayfinding instructions for guiding a blind or visually impaired traveler from the first identified landmark to the second identified landmark; and
outputting the retrieved blind-ready wayfinding instructions to wireless device carried by a blind or visually impaired user wherein the blind-ready wayfinding instructions include acoustic information specific to an environment associated with the first and second landmarks.
20. The method of claim 19, wherein the wireless device is a mobile telephone and the wayfinding instructions comprise a text-to-speech synthesis of textual wayfinding instructions.
21. A machine-readable data storage medium comprising machine-executable instructions for:
receiving into memory information identifying first and second landmarks from a set of predefined landmarks within a predefined geographic region;
in response to the identified first and second landmarks, causing a processor to access and retrieve from a database, blind-ready wayfinding instructions for guiding a blind or visually impaired traveler from the first identified landmark to the second identified landmark; and
causing a device associated with the processor to output the retrieved blind-ready wayfinding instructions to a wireless device carried by a blind or visually impaired user wherein the blind-ready wayfinding instructions include acoustic information specific to an environment associated with the first and second landmarks.
22. The machine-readable data storage medium of claim 21, wherein the wireless device is a mobile telephone and the wayfinding instructions comprise a text-to-speech synthesis of textual wayfinding instructions.
US14/533,519 2009-03-18 2014-11-05 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers Abandoned US20150127256A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/533,519 US20150127256A1 (en) 2009-03-18 2014-11-05 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US16135609P 2009-03-18 2009-03-18
US12/727,206 US8594935B2 (en) 2009-03-18 2010-03-18 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers
US14/088,817 US8886462B1 (en) 2009-03-18 2013-11-25 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers
US14/533,519 US20150127256A1 (en) 2009-03-18 2014-11-05 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US14/088,817 Continuation US8886462B1 (en) 2009-03-18 2013-11-25 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers

Publications (1)

Publication Number Publication Date
US20150127256A1 true US20150127256A1 (en) 2015-05-07

Family

ID=42227657

Family Applications (3)

Application Number Title Priority Date Filing Date
US12/727,206 Expired - Fee Related US8594935B2 (en) 2009-03-18 2010-03-18 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers
US14/088,817 Expired - Fee Related US8886462B1 (en) 2009-03-18 2013-11-25 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers
US14/533,519 Abandoned US20150127256A1 (en) 2009-03-18 2014-11-05 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US12/727,206 Expired - Fee Related US8594935B2 (en) 2009-03-18 2010-03-18 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers
US14/088,817 Expired - Fee Related US8886462B1 (en) 2009-03-18 2013-11-25 Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers

Country Status (2)

Country Link
US (3) US8594935B2 (en)
WO (1) WO2010108047A2 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106606407A (en) * 2015-10-22 2017-05-03 北京金坤科创技术有限公司 High-precision indoor navigation system for blind person
US20170370733A1 (en) * 2016-06-28 2017-12-28 International Business Machines Corporation Technology for giving users cognitive mapping capability
US9909880B2 (en) 2015-11-09 2018-03-06 International Business Machines Corporation Automated wayfinding of an accessible route
CN108139215A (en) * 2015-10-16 2018-06-08 因温特奥股份公司 Using voice signal to the route guidance of personnel
US11834838B2 (en) 2019-05-06 2023-12-05 Richard Hoffberg Wheelchair ramp

Families Citing this family (62)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4315211B2 (en) * 2007-05-01 2009-08-19 ソニー株式会社 Portable information terminal, control method, and program
WO2010108047A2 (en) 2009-03-18 2010-09-23 Intouch Graphics, Inc. Systems, methods, and software for providing orientation and wayfinding data to blind travelers
CN101886928A (en) * 2009-05-14 2010-11-17 深圳富泰宏精密工业有限公司 Portable electronic device with guiding function
US20110130956A1 (en) * 2009-11-30 2011-06-02 Nokia Corporation Method and apparatus for presenting contextually appropriate navigation instructions
EP2489342A1 (en) * 2011-02-15 2012-08-22 Alcatel Lucent Method and assistive device for navigation
US9000933B2 (en) 2011-05-12 2015-04-07 At&T Intellectual Property I, L.P. Automated allergy alerts
US9947167B2 (en) 2011-08-26 2018-04-17 Elwha Llc Treatment system and method for ingestible product dispensing system and method
US10192037B2 (en) 2011-08-26 2019-01-29 Elwah LLC Reporting system and method for ingestible product preparation system and method
US10026336B2 (en) 2011-08-26 2018-07-17 Elwha Llc Refuse intelligence acquisition system and method for ingestible product preparation system and method
US20130054255A1 (en) 2011-08-26 2013-02-28 Elwha LLC, a limited liability company of the State of Delaware Controlled substance authorization and method for ingestible product preparation system and method
US9785985B2 (en) * 2011-08-26 2017-10-10 Elwha Llc Selection information system and method for ingestible product preparation system and method
US20130330451A1 (en) 2012-06-12 2013-12-12 Elwha LLC, a limited liability company of the State of Delaware Substrate Structure Duct Treatment System and Method for Ingestible Product System and Method
US20130331981A1 (en) 2012-06-12 2013-12-12 Elwha LLC, a limited liability company of the State of Delaware Substrate Structure Deposition Treatment System And Method For Ingestible Product System And Method
US9111256B2 (en) 2011-08-26 2015-08-18 Elwha Llc Selection information system and method for ingestible product preparation system and method
US9997006B2 (en) 2011-08-26 2018-06-12 Elwha Llc Treatment system and method for ingestible product dispensing system and method
US9922576B2 (en) 2011-08-26 2018-03-20 Elwha Llc Ingestion intelligence acquisition system and method for ingestible material preparation system and method
US10121218B2 (en) 2012-06-12 2018-11-06 Elwha Llc Substrate structure injection treatment system and method for ingestible product system and method
US9240028B2 (en) 2011-08-26 2016-01-19 Elwha Llc Reporting system and method for ingestible product preparation system and method
US20180150921A1 (en) * 2012-07-26 2018-05-31 Nearby Coolleges LLC Travel planning application resulting in efficient customized grouping and coordination of selected locations and destinations based on the combination of user selected important criteria, radius searching, and real time simultaneous multi-source data gathering, analysis, and derivation executed by and presented on a computer and/or smart device for user preference
JP2014194363A (en) * 2013-03-28 2014-10-09 Fujitsu Ltd Guidance device, guidance method, and program
US9942685B2 (en) 2013-06-28 2018-04-10 Microsoft Technology Licensing, Llc Navigation with three dimensional audio effects
US9307073B2 (en) * 2013-12-31 2016-04-05 Sorenson Communications, Inc. Visual assistance systems and related methods
US10248856B2 (en) 2014-01-14 2019-04-02 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10360907B2 (en) 2014-01-14 2019-07-23 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9915545B2 (en) 2014-01-14 2018-03-13 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9578307B2 (en) 2014-01-14 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US10024679B2 (en) 2014-01-14 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9629774B2 (en) * 2014-01-14 2017-04-25 Toyota Motor Engineering & Manufacturing North America, Inc. Smart necklace with stereo vision and onboard processing
US9140554B2 (en) * 2014-01-24 2015-09-22 Microsoft Technology Licensing, Llc Audio navigation assistance
US10104452B2 (en) * 2014-05-08 2018-10-16 Paypal, Inc. Gathering unique information from dispersed users
US20150330787A1 (en) * 2014-05-19 2015-11-19 Joseph Cioffi Systems, Methods and Software for Redirecting Blind Travelers Using Dynamic Wayfinding Orientation and Wayfinding Data
US10024667B2 (en) 2014-08-01 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable earpiece for providing social and environmental awareness
US9212924B1 (en) * 2014-08-21 2015-12-15 Microsoft Technology Licensing, Llc Multimode transportation transitions
US10024678B2 (en) 2014-09-17 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable clip for providing social and environmental awareness
US9922236B2 (en) 2014-09-17 2018-03-20 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable eyeglasses for providing social and environmental awareness
USD768024S1 (en) 2014-09-22 2016-10-04 Toyota Motor Engineering & Manufacturing North America, Inc. Necklace with a built in guidance device
US9576460B2 (en) 2015-01-21 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device for hazard detection and warning based on image and audio data
US10490102B2 (en) 2015-02-10 2019-11-26 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for braille assistance
US9586318B2 (en) 2015-02-27 2017-03-07 Toyota Motor Engineering & Manufacturing North America, Inc. Modular robot with smart device
US9677901B2 (en) 2015-03-10 2017-06-13 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing navigation instructions at optimal times
US9811752B2 (en) 2015-03-10 2017-11-07 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable smart device and method for redundant object identification
US9972216B2 (en) 2015-03-20 2018-05-15 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for storing and playback of information for blind users
US9898039B2 (en) 2015-08-03 2018-02-20 Toyota Motor Engineering & Manufacturing North America, Inc. Modular smart necklace
US10024680B2 (en) 2016-03-11 2018-07-17 Toyota Motor Engineering & Manufacturing North America, Inc. Step based guidance system
US9958275B2 (en) 2016-05-31 2018-05-01 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for wearable smart device communications
US10561519B2 (en) 2016-07-20 2020-02-18 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device having a curved back to reduce pressure on vertebrae
US20180060088A1 (en) * 2016-08-31 2018-03-01 Microsoft Technology Licensing, Llc Group Interactions
US10432851B2 (en) 2016-10-28 2019-10-01 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable computing device for detecting photography
US10012505B2 (en) 2016-11-11 2018-07-03 Toyota Motor Engineering & Manufacturing North America, Inc. Wearable system for providing walking directions
US10521669B2 (en) 2016-11-14 2019-12-31 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for providing guidance or feedback to a user
US10172760B2 (en) 2017-01-19 2019-01-08 Jennifer Hendrix Responsive route guidance and identification system
CN107517441B (en) * 2017-06-27 2020-12-25 惠州Tcl云创科技有限公司 Navigation method, mobile terminal and device with storage function
CN108095987B (en) * 2017-12-06 2020-10-16 英业达科技有限公司 Vision-impaired navigation system and method thereof
CN108180901A (en) * 2017-12-08 2018-06-19 深圳先进技术研究院 Indoor navigation method, device, robot and the storage medium of blind-guidance robot
CN108534788B (en) * 2018-03-07 2020-06-05 广州大学 AGV navigation method based on kinect vision
US20220074760A1 (en) * 2018-12-10 2022-03-10 Pluxity Co., Ltd. Method, apparatus and computer readable recording medium for providing user-customized geographic information and analysis information using universal map
US10931916B2 (en) 2019-04-24 2021-02-23 Sorenson Ip Holdings, Llc Apparatus, method and computer-readable medium for automatically adjusting the brightness of a videophone visual indicator
JPWO2021090445A1 (en) * 2019-11-07 2021-05-14
US11257396B2 (en) * 2020-03-18 2022-02-22 Sas Institute Inc. User interfaces for converting geospatial data into audio outputs
CN112197777A (en) * 2020-08-28 2021-01-08 星络智能科技有限公司 Blind person navigation method, server and computer readable storage medium
US11460973B1 (en) 2022-04-11 2022-10-04 Sas Institute Inc:. User interfaces for converting node-link data into audio outputs
WO2023247984A1 (en) 2022-06-20 2023-12-28 Genima Innovations Marketing Gmbh Device and method for assisting visually impaired persons in public spaces

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6711474B1 (en) * 2000-01-24 2004-03-23 G. Victor Treyz Automobile personal computer systems
US20060293839A1 (en) * 2005-06-10 2006-12-28 Board Of Regents, The University Of Texas System System, method and apparatus for providing navigational assistance
US20090191514A1 (en) * 2008-01-25 2009-07-30 Judy Barnow Calorie Counter
US20090240428A1 (en) * 2007-06-18 2009-09-24 Maryam Shahrestani Mobile phone having gps navigation system
US20100049431A1 (en) * 2008-04-30 2010-02-25 Rafael Maya Zetune Navigation Using Portable Reading Machine
US20100125409A1 (en) * 2008-11-18 2010-05-20 Nokia Corporation User generated pedestrian and indoor shortcut routes for navigation systems

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8526971B2 (en) * 1996-08-15 2013-09-03 Snaptrack, Inc. Method and apparatus for providing position-related information to mobile recipients
US20040155815A1 (en) * 2001-05-14 2004-08-12 Motorola, Inc. Wireless navigational system, device and method
US6502032B1 (en) * 2001-06-25 2002-12-31 The United States Of America As Represented By The Secretary Of The Air Force GPS urban navigation system for the blind
US20080120029A1 (en) * 2006-02-16 2008-05-22 Zelek John S Wearable tactile navigation system
JP4787093B2 (en) 2006-06-29 2011-10-05 京セラ株式会社 Mobile station
JP2008111792A (en) * 2006-10-31 2008-05-15 Oki Electric Ind Co Ltd Guidance sentence generation system, guidance sentence generation method, and guidance sentence generation program
US20090032590A1 (en) * 2007-08-02 2009-02-05 Hopkins Billy D Location, orientation, product and color identification apparatus, system and method for the blind or visually impaired
US8090532B2 (en) * 2007-12-14 2012-01-03 Microsoft Corporation Pedestrian route production
WO2010108047A2 (en) 2009-03-18 2010-09-23 Intouch Graphics, Inc. Systems, methods, and software for providing orientation and wayfinding data to blind travelers

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6711474B1 (en) * 2000-01-24 2004-03-23 G. Victor Treyz Automobile personal computer systems
US20060293839A1 (en) * 2005-06-10 2006-12-28 Board Of Regents, The University Of Texas System System, method and apparatus for providing navigational assistance
US20090240428A1 (en) * 2007-06-18 2009-09-24 Maryam Shahrestani Mobile phone having gps navigation system
US20090191514A1 (en) * 2008-01-25 2009-07-30 Judy Barnow Calorie Counter
US20100049431A1 (en) * 2008-04-30 2010-02-25 Rafael Maya Zetune Navigation Using Portable Reading Machine
US20100125409A1 (en) * 2008-11-18 2010-05-20 Nokia Corporation User generated pedestrian and indoor shortcut routes for navigation systems

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108139215A (en) * 2015-10-16 2018-06-08 因温特奥股份公司 Using voice signal to the route guidance of personnel
CN106606407A (en) * 2015-10-22 2017-05-03 北京金坤科创技术有限公司 High-precision indoor navigation system for blind person
US9909880B2 (en) 2015-11-09 2018-03-06 International Business Machines Corporation Automated wayfinding of an accessible route
US20170370733A1 (en) * 2016-06-28 2017-12-28 International Business Machines Corporation Technology for giving users cognitive mapping capability
US10066949B2 (en) * 2016-06-28 2018-09-04 International Business Machines Corporation Technology for giving users cognitive mapping capability
US11834838B2 (en) 2019-05-06 2023-12-05 Richard Hoffberg Wheelchair ramp

Also Published As

Publication number Publication date
WO2010108047A2 (en) 2010-09-23
US20100241350A1 (en) 2010-09-23
WO2010108047A3 (en) 2010-12-29
US8886462B1 (en) 2014-11-11
US8594935B2 (en) 2013-11-26
US20140324341A1 (en) 2014-10-30

Similar Documents

Publication Publication Date Title
US8886462B1 (en) Systems, methods, and software for providing wayfinding orientation and wayfinding data to blind travelers
JP6580703B2 (en) System and method for disambiguating a location entity associated with a mobile device's current geographic location
US11243087B2 (en) Device and method for providing content to user
Krüger et al. The connected user interface: Realizing a personal situated navigation service
US20230228577A1 (en) Devices, systems, and methods for navigation and usage guidance in a navigable space using wireless communication
US20150330787A1 (en) Systems, Methods and Software for Redirecting Blind Travelers Using Dynamic Wayfinding Orientation and Wayfinding Data
US20050027705A1 (en) Mapping method and system
US20170206211A1 (en) Computerized system and method for determining location based data and communicating such data for overlay on a mapping interface
US20130006515A1 (en) Geo-spatial recommendation and discovery system
EP2549233A2 (en) Method of operating a navigation system to provide geographic location information
US20130006522A1 (en) Method for constructing geo-fences for a spatial recommendation and discovery system
US10234299B2 (en) Geo-location tracking system and method
JP2003523565A (en) An Internet system that connects client travelers to geographically relevant data
Giudice et al. Evaluation of an accessible, real-time, and infrastructure-free indoor navigation system by users who are blind in the mall of america
WO2008002732A2 (en) Driving directions with landmark data
Kameswaran et al. Understanding in-situ use of commonly available navigation technologies by people with visual impairments
US20210232819A1 (en) Method and system of determining a recommended pathway through a physical structure environment
Balata et al. Navigation problems in blind-to-blind pedestrians tele-assistance navigation
Zahabi et al. Design of navigation applications for people with disabilities: A review of literature and guideline formulation
Rajamäki et al. LaureaPOP indoor navigation service for the visually impaired in a WLAN environment
CN111694906A (en) Method, device and equipment for displaying POI in map
Marsh et al. Pharos: coupling GSM and GPS-TALK technologies to provide orientation, navigation and location-based services for the blind
Wang et al. A recommendation for designing mobile pedestrian navigation system in university campuses
Feng et al. Commute booster: a mobile application for first/last mile and middle mile navigation support for people with blindness and low vision
US11307045B2 (en) Method and system to determine navigation actions based on instructions from a directional dialogue

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION