A robot trained to read braille twice as fast as humans

Researchers have developed a robotic sensor that incorporates artificial intelligence techniques to read Braille at speeds about twice that of most human readers.

The University of Cambridge research team used machine learning algorithms to teach a robotic sensor to quickly swipe across lines of braille text. The robot was able to read Braille at a speed of 315 words per minute with an accuracy close to 90%.

Although the robot braille reader was not developed as an assistive technology, researchers say the high sensitivity required to read braille makes it an ideal test for developing robot hands or prosthetics with comparable sensitivity to that of human fingertips. The results are reported in the journal IEEE Letters on Robotics and Automation.

Human fingertips are remarkably sensitive and help us gather information about the world around us. Our fingertips can detect tiny changes in the texture of a material or help us know how much force to use to grip an object: for example, picking up an egg without breaking it or a bowling ball without dropping it .

Reproducing this level of sensitivity in a robotic hand, in an energy-efficient manner, constitutes a significant engineering challenge. In the laboratory of Professor Fumiya Iida in Cambridge’s Department of Engineering, researchers are developing solutions to this and other skills that humans find easy, but robots find difficult.

“The softness of human fingertips is one of the reasons we are able to grip objects with the right amount of pressure,” said Parth Potdar of the Cambridge Department of Engineering and undergraduate student at Pembroke College, first author of the article. “For robotics, smoothness is a useful characteristic, but you also need a lot of sensor information, and it’s difficult to have both at once, especially when dealing with flexible or deformable surfaces.”

Braille is an ideal test for a robot’s “fingertip” because reading it requires high sensitivity because the dots of each representative letter pattern are very close together. Researchers used a commercially available sensor to develop a robotic braille reader that more accurately replicates human reading behavior.

“There are robotic Braille readers, but they only read one letter at a time, which is not the case with humans,” said co-author David Hardman, also of the Department of Engineering. “Existing robotic braille readers operate statically: they touch a letter pattern, read it, tear off the surface, move, lower to the next letter pattern, and so on. We want something more realistic and much more effective. “.

The robotic sensor used by the researchers has a camera on the “fingertip” and reads using a combination of information from the camera and sensors. “This is a difficult problem for roboticists because a lot of image processing needs to be done to remove motion blur, which takes time and energy,” Potdar said.

The team developed machine learning algorithms so that the robotic reader could “unblur” images before the sensor attempted to recognize the letters. They trained the algorithm on a set of sharp Braille images with fake blur applied. Once the algorithm learned to remove blur from the letters, they used a computer vision model to detect and classify each character.

Once the algorithms were integrated, the researchers tested their reader by sliding it quickly along rows of braille characters. The robotic Braille reader could read at 315 words per minute with 87% accuracy, twice as fast and about as accurate as a human Braille reader.

“Considering that we used fake blur to train the algorithm, it was surprising how accurate it was in reading braille,” Hardman said. “We found a good compromise between speed and accuracy, which is also the case with human readers.”

“Braille reading speed is an excellent way to measure the dynamic performance of tactile sensing systems. Our results could therefore be applicable beyond braille, for applications such as detecting surface textures or swipes in the robotic manipulation,” Potdar said.

In the future, researchers hope to scale this technology to the size of a humanoid hand or skin.