Virtual-Blind-Road Following Based Wearable Navigation Device for Blind People
ABSTRACT
To help the blind people walk to the destination efficiently and safely in indoor environment, a novel wearable navigation device is presented in this paper. The locating, way-finding, route following and obstacle avoiding modules are the essential components in a navigation system, while it remains a challenging task to consider obstacle avoiding during route following, as the indoor environment is complex, changeable and possibly with dynamic objects. To address this issue, we propose a novel scheme which utilizes a dynamic sub-goal selecting strategy to guide the users to the destination and help them bypass obstacles at the same time. This scheme serves as the key component of a complete navigation system deployed on a pair of wearable optical see-through glasses for the ease of use of blind people’s daily walks. The proposed navigation device has been tested on a collection of individuals and proved to be effective on indoor navigation tasks. The sensors embedded are of low cost, small volume and easy integration, making it possible for the glasses to be widely used as a wearable consumer device.
Existing system
The visually impaired people usually have difficulties in walking in an unfamiliar and complex place independently. To provide them an automatic navigation device with effective guidance on their move, three problems should be considered:
1.) Where is the person? The device has to know where theperson is located in order to make a correct decision for guiding
the person. This refers to be the localization problem.
2.) Where does the person want to go? In order to help the visually impaired person reach his destination, the device has to identify the destination. This is known as goal recognition.
3.) How does the person get there? This includes way-finding, route following and obstacle detecting. Way finding is to plan a shortest path from the starting position to the destination, route following is to make sure the blind person follow the planned path and obstacle detecting is to help him avoid obstacles
So far, there are many navigation systems trying to solve the above problems, such as the low-cost white cane guide dog and ETAs (Electronic Travel Aids). However, white cane is unable to find a globally shortest path and provide the location information. Guide dog is incapable of detecting overhanging object, and needs costly training, which may be unaffordable to the visually impaired individuals . Most existing ETAs are only intended for obstacle detecting or/and feedback, and cannot provide way-finding and route following functions. Although some ETAs were designed with way-finding and route following functions, such as the cactus tree based algorithm proposed in , obstacle (especially the dynamic obstacles) detecting and avoiding are ignored. A wearable indoor navigation system based on visual marker recognition and ultrasonic obstacle perception was introduced in , but the localization precision is not high enough for guiding the blind due to the error increase of the inertial measurement sensor, and the goal recognition scheme is less efficient on planning a global path. A successful navigation system for the blind is the visual SLAM (Simultaneous Localization and Mapping) and PoI (Point of Interest)-graph based indoor navigation system presented in. However, the obstacle detection heavily relies on the white cane swaying, which is not efficient and portable.
DISADVANTAGES:
- This system not efficient.
PROPOSED SYSTEM:
This paper aims to develop an effective wearable navigation system which can locate the user, follow the virtual-blind-road and avoid obstacles at the same time, in order to provide automatic navigation for the visually impaired people. The main contribution of this paper is the proposal of a novel dynamic sub-goal selecting based virtual-blind-road following scheme which combines the obstacle avoiding algorithm for guiding the blind people to follow the globally shortest virtual-blind-road without collision. This scheme can help the visually impaired people to avoid obstacles while following a path precisely to the destination, which realizes the automatic navigation for them. As the GPS-based blind navigation technology cannot be used due to the severely degradation of GPS signal in indoor environment, the visual SLAM (e.g. ORB-SLAM (ORiented Brief-SLAM)) was adopted in this paper for the building of the virtual-blind-road and the localization. The whole system is deployed on a pair of wearable optical see-through glasses, which is able to give the visually impaired people visual hints and sound feedback.
HARDWARE CONFIGURATION :
The proposed navigation device consists of a fisheye and a depth camera, an ultrasonic rangefinder, an embedded CPU board, a pair of OST (Optical See-Through) glasses, and an earphone. These sensors are of low cost, small volume, and easy integration, and thus can be widely used in consumer market. The depth camera and the fisheye camera are used for building the virtual-blind-road and locating the user precisely with visual SLAM algorithm. Besides, the depth camera is also used for obstacle detection [13]. To compensate for the limitations of the depth camera, such as passing through the transparent objects, being absorbed by some special materials, etc., the ultrasonic rangefinder is utilized. It consists of an ultrasonic sensor and a MCU (Microprogrammed Control Unit) to measure the distance of an obstacle in front of the user. Fusing the depth camera and the ultrasonic rangefinder for obstacle detection can ensure adequate safety of the visually impaired individual. The CPU operates at 1.44 GHz with a 2 GB RAM. All the algorithms such as visual SLAM, obstacle detecting, way-finding, route following, speech synthesis, etc. are performed on the CPU. The OST-glasses have two display screens for displaying the guiding information and surrounding information to the partially sighted individual. In addition, the glasses have two loudspeakers for playing the guiding sound. The earphone is used for playing the guiding sound to the totally blind individuals when they are in a noisy indoor environment (e.g. supermarket). The hardware configuration of the proposed navigation device is illustrated in Fig. 1, and its prototype is shown in Fig. 2.
Fig. 1. The hardware configuration of the proposed navigation device
Fig. 2. The prototype of the proposed navigation device
The architecture of the proposed navigation system is illustrated in Fig. 3, which includes visual SLAM, PoI-graph, way finding, obstacle detection and route following module. The rendering module converts the guiding information produced by the navigation system into audio or/and visual cues. The obstacle detection and rendering modules have been presented in our previous work [13], and thus will not be repeated here The visual SLAM module utilizes the RGB image and depthimage to build the virtual-blind-road (offline) and locate the user (online). After the virtual-blind-road is tagged with some key positions (offline), it becomes a sparse map (i.e. PoI-graph). The way finding module plans a globally shortest path from the starting position to the destination according to the PoI-graph. The obstacle detection module provides many candidate walkable directions according to the current depth image. The route following module takes the current pose of the blind user, the globally shortest path, the candidate walkable directions, and the obstacle distance measured by the ultrasonic rangefinder as inputs, and uses a dynamic sub-goal selecting method to produce the guiding information. This allows the user to follow the globally shortest path as closely as possible and avoid obstacles at the same time.
Fig. 3. Architecture of the proposed navigation system.
ADVANTAGES
The sensors embedded on the device have the characteristics of low cost, small size and easy integration.
CONCLUSION
This paper presents a novel navigation device for the visually impaired groups to help them reach the destination safely and efficiently in indoor environment. The visual SLAM algorithm was used to solve the problems of indoor localization and virtual-blind-road building. The PoI-graph was generated to find a globally shortest virtual-blind-road by the A* based way finding algorithm. The dynamic sub-goal selecting based route following algorithm was proposed to help the blind follow the globally shortest virtual-blind-road as closely as possible and meanwhile avoid obstacles (including dynamic obstacles). Experimental results verified that the proposed navigation device was effective enough on helping the visually impaired people walk from one place to another. The sensors embedded on the device have the characteristics of low cost, small size and easy integration. Thus, it has great potential in consumer market, especially electronic travel aids market.