Hear the Sound of Color with Google’s Synesthesia Tool

Some of history’s greatest artists, like Vassily Kandinsky, Duke Ellington, and Mozart, experienced  synesthesia, a neurological condition that allows you to “see” sounds or “hear” colors. Now, a Google Arts and Culture tool called Play a Kandinsky gives you the chance to hear what the artist might have heard while painting one of his most famous works.

Google teamed up with experimental musicians Antonie Bertin and NSDOS to study Vassily Kandinsky’s writings, which detail the artist’s association between shapes, colors, and sound. The team paired its findings with machine learning tools to create Play a Kandinsky, a seven-movement musical game that simulates what Kandinsky might have heard while painting Yellow Red Blue.

Play a Kandinsky starts with a few rudimentary examples of what shape-to-sound or color-to-sound synesthesia might feel like. Then, the game frees you to hover over the Yellow Red Blue canvas and hear a symphony of combined sounds. Some shapes and colors sound soothing, while others sound violent or cacophonous.

Google Arts and Culture developed Play a Kandinsky for its Sounds like Kandinsky preservation project. Other tools in the project include a virtual tour of his Paris studio and an up-close look at another famous Kandinsky painting, Sky Blue.