{"id":171843,"date":"2021-02-05T20:37:23","date_gmt":"2021-02-05T17:37:23","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/this-ai-generates-music-from-silent-piano-performances\/"},"modified":"2021-02-05T20:37:23","modified_gmt":"2021-02-05T17:37:23","slug":"this-ai-generates-music-from-silent-piano-performances","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/this-ai-generates-music-from-silent-piano-performances\/","title":{"rendered":"#This AI generates music from silent piano performances"},"content":{"rendered":"

“#This AI generates music from silent piano performances<\/strong>”<\/p>\n

\n Scientists have developed an AI that can generate music from silent piano performances, just by watching the movements of the player\u2019s hands.<\/p>\n

The system, called Audeo, analyzes top-down videos\u00a0of someone tickling the ivories to predict which keys are being pressed in each frame.\u00a0It then produces a transcript<\/a> of the music, which a synthesizer translates into sound.<\/p>\n

The researchers trained and tested the AI on footage\u00a0of<\/span>\u00a0<\/span>pianist Paul Barton<\/a>\u00a0playing tunes by famous\u00a0composers.<\/p>\n

They\u00a0then evaluated the accuracy of the Audeo\u2019s compositions by playing them to music-recognition apps, such as Shazam and SoundHound.<\/p>\n

The apps\u00a0identified the tune 86% of the time\u00a0 \u2014 just 7% less than they recognized the source videos.<\/p>\n

[Read: How much does it cost to buy, own, and run an EV? It\u2019s not as much as you think]<\/em><\/p>\n

Senior study author Eli Shlizerman, an assistant professor at the University of Washington, said<\/a> he was surprised by the quality of the AI\u2019s output:<\/p>\n

To create music that sounds like it could be played in a musical performance was previously believed to be impossible.\u00a0An algorithm needs to figure out the cues, or \u2018features\u2019\u00a0in the video frames that are related to generating music, and it needs to \u2018imagine\u2019 the sound that\u2019s happening in between the video frames. It requires a system that is both precise and imaginative.\u00a0<\/span><\/p>\n<\/blockquote>\n

You can judge its performances for yourself\u00a0in the video below:<\/p>\n

The researchers have also explored using Audeo to change the style of music. Shlizerman said the system could show how music produced by a\u00a0piano sounds when played through a trumpet.<\/p>\n

He hopes the research\u00a0will enable new ways for people to interact with music:<\/p>\n

For example, one future application is that Audeo can be extended to a virtual piano with a camera recording just a person\u2019s hands. Also, by placing a camera on top of a real piano, Audeo could potentially assist in new ways of teaching students how to play.<\/span><\/p>\n<\/blockquote>\n

You can read the full study paper here<\/a>.<\/p>\n

\n Published February 5, 2021 \u2014 17:37 UTC<\/p><\/div>\n