{"id":170999,"date":"2021-02-05T01:16:47","date_gmt":"2021-02-04T22:16:47","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/audeo-teaches-artificial-intelligence-to-play-the-piano\/"},"modified":"2021-02-05T01:16:47","modified_gmt":"2021-02-04T22:16:47","slug":"audeo-teaches-artificial-intelligence-to-play-the-piano","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/audeo-teaches-artificial-intelligence-to-play-the-piano\/","title":{"rendered":"#’Audeo’ teaches artificial intelligence to play the piano"},"content":{"rendered":"

“#’Audeo’ teaches artificial intelligence to play the piano<\/strong>”<\/p>\n

\n
\n
\n
\"'Audeo'
\n Credit: University of Washington
\n <\/figcaption><\/figure>\n<\/div>\n<\/div>\n

Anyone who’s been to a concert knows that something magical happ<\/a>ens between the performers and their instruments. It transforms music from being just “notes on a page” to a satisfying experience.<\/p>\n

A University of Washington team wondered if artificial intelligence could recreate that delight using only visual cues\u2014a silent, top-down video of someone playing the piano. The researchers used machine learning to create a system, called Audeo, that creates audio from silent piano performances. When the group tested the music Audeo created with music-recognition apps, such as SoundHound, the apps correctly identified the piece Audeo played about 86% of the time. For comparison, these apps identified the piece in the audio tracks from the source videos 93% of the time.<\/p>\n

The researchers presented Audeo Dec. 8 at the NeurIPS 2020 conference.<\/p>\n

“To create music that sounds like it could be played in a musical performance was previously believed to be impossible,” said senior author Eli Shlizerman, an assistant professor in both the applied mathematics and the electrical and computer engineering departments. “An algorithm needs to figure out the cues, or ‘features,’ in the video frames that are related to generating music, and it needs to ‘imagine’ the sound that’s happening in between the video frames. It requires a system that is both precise and imaginative. The fact that we achieved music that sounded pretty good was a surprise.”<\/p>\n

Audeo uses a series<\/a> of steps to decode what’s happening in the video and then translate it into music. First, it has to detect which keys are pressed in each video frame to create a diagram over time. Then it needs to translate that diagram into something that a music synthesizer would actually recognize as a sound a piano would make. This second step cleans up the data and adds in more information, such as how strongly each key is pressed and for how long.<\/p>\n


\n