Computer-Simulated Realistic Sounds

Researchers at MIT have created a system that is able to simulate sounds that are so perfect that they can even fool human listeners. These are effects created by an algorythm which predicts the acoustic qualities of a sound, just by analyzing silent videoclips.

By examining videos of drumsticks hitting different objects and materials, like leaves, wood, water or plastic, the algorythm creates a realistic simulation, able to deceive the listeners. Check the video below:



To achieve such a result, PhD student Andrew Owens and his team applied a technique called "deep learning", that allows computers to identify patterns out of huge amounts of data by themselves. For several months, computers looked at data from about 1000 videos and 46000 sounds of drumsticks hitting, scratching and prodding many types of materials and objects. The drumstick's ability to produce consistent sounds was the reason for its choice.

The simulated sounds were often so good that when people were presented with two instances of the same video - one accompanied by simulated sound and another with the real sound - they failed to distinguish correctly the real one. Researchers believe that this is an important step not only in terms of technologies of simulation, but also for robotics. “A robot could look at a sidewalk and instinctively know that the cement is hard and the grass is soft, and therefore know what would happen if they stepped on either of them” said Owens.“Being able to predict sound is an important first step toward being able to predict the consequences of physical interactions with the world”.

Source: Gizmodo