New apps help blind people navigate indoor spaces

Two new apps are expected to help blind people navigate indoor spaces by providing voice instructions via a smartphone. This provides a safe solution for navigating areas where GPS is ineffective.

Roberto Manduchi, a professor of computer science and engineering at UC Santa Cruz, has devoted much of his research career to creating technologies that are accessible to the blind and visually impaired. Over the years of working with these communities, he has learned that there is a particular need for tools to facilitate indoor navigation in new spaces.

“Moving independently in a place you don’t know is particularly difficult, because you have no visual reference and you get lost very easily. The idea here is to try to make this a little easier and safer for people,” Manduchi said.

In a new article published in the journal ACM Transactions on Accessible ComputingManduchi’s research group presents two smartphone apps that provide orientation indoors, navigating to a specific point and returning safely, the process of retracing a past route. The apps provide audio cues and do not require a user to hold their smartphone in front of them, which would be distracting and attract undue attention.

Safer and scalable technology

Smartphones are a great platform for hosting accessible technology because they are less expensive than a dedicated hardware system, have the support of company IT teams, and come equipped with built-in sensors and accessibility features.

Other smartphone-based wayfinding systems require a person to walk with their phone out, which can create several problems. A blind person navigating a new space often uses at least one hand for a guide dog or cane, so using the other for a phone is less than ideal. Holding a phone in your hand also makes the browser vulnerable to crime, and people with disabilities already face disproportionately higher crime rates.

Although companies like Apple and Google have developed indoor wayfinding systems for specific locations, such as large airports and stadiums, their methods rely on sensors installed inside those buildings. This makes it a much less scalable solution due to the cost of adding and maintaining additional infrastructure.

Use of integrated sensors

The Manduchi wayfinding app provides directions in the same way as GPS services like Google Maps; however, GPS systems do not work indoors because the satellite signal is distorted by the building walls. Instead, Manduchi’s system uses other sensors embedded in a smartphone to provide voice instructions for navigating an unfamiliar building.

The wayfinding app works by using a map of a building’s interior to find a path to the destination, then uses a phone’s built-in inertial sensors, accelerometers and gyroscopes, which provide features such as a step counter, to track the browser. progress on the path.

The same sensors can also track the orientation of the phone and therefore the browser. However, the estimated location and orientation are often somewhat inaccurate, so researchers have incorporated another method called particle filtering to reinforce the physical constraints of a building so that it does not interpret that the browser goes through walls or other impossible situations.

The backtracking app simply reverses a route previously taken by a browser, which is useful in situations where a blind person is guided into a room and wants to exit independently. In addition to inertial sensors, it uses the phone’s magnetometer to identify characteristic magnetic field anomalies, typically created by large appliances, which can serve as landmarks in a building.

Communicate instructions

Both systems provide instructions through voice communication and can also be used with a smart watch to supplement instructions with vibrations. Overall, the researchers tried to minimize the amount of information provided to the browser so they could focus on security.

They also rely on the browser to determine where to turn, to account for any tracking errors. The system asks a person to make their next turn five meters before predicting the turn will occur, with instructions such as “at the next intersection, turn left”, and the navigator can start finding the turn at using his cane or guide dog.

“Sharing responsibility, in my opinion, is the right approach,” Manduchi said. “As a philosophy, you can’t rely on technology alone. This is also true when you drive a car: if it says turn right, you don’t immediately turn right, you look for where the intersection is. You have to work with the system.”

While testing their systems in UC Santa Cruz’s Baskin Engineering building, the research team found that users were able to successfully navigate the many hallways and turns. The team will continue to refine its applications, which use the same interface but are separated for easier development.

In the future, they will focus on integrating AI features that could allow a browser to take a photo of its surroundings and get a description of the scene if it is in a particularly difficult area to navigate, such as an alcove of a building or a large space. . They also want to improve the ability to access and download building maps, perhaps leveraging an open source software ecosystem to do so.

“I’m very grateful to the blind community in Santa Cruz, who gave me fantastic advice. (As engineers creating technology for the blind community), you have to be very, very careful and very humble, and start by the person who is going to use the technology rather than the technology itself,” Manduchi said.