There is no formula for writing a good melody. One can argue about the ratio of the dimensions of the musical instrument or about the hair in the ear.
Google’s latest doodle pays homage to Johann Sebastian Bach, a 17th-century German composer.
This doodle allows the user to churn their own melodies with just a few clicks. The doodle runs on a machine learning model, which is trained on previous works of Bach.
So when the user selects the notes randomly and the model harmonizes these and plays them out.
Introducing Coconet
The science of music and the auditory sensory experience can be debated till eternity and while we are at it, Google has come up with a machine learning model Coconet which fed itself with more than 300 works of Bach.
This model takes a piece from Bach, randomly erase some notes and then the missing notes are predicted.
The model randomly accepts incomplete notes as input and plays complete scores.
To do this, the model has to:
- Harmonize melodies
- Create smooth transitions
- Rewrite music and
- Compose from scratch
Coconet is not dependent on the chronological order, which other models follow. The user can pick out any part from a music score and the model checks the patterns in the way few notes are selected, harmonize and will develop the complete score.
Musical scores are three dimensional objects; there are four voices and the music for each voice can be represented as a two-dimensional array with time extending horizontally and pitches laid out vertically.
At any point in time, each voice will have one spike in the pitch vector while the rest remain zero.
This pitch vector is used in constructing the probability distribution over a set of pitches for the model to detect which pitch was played and then predict what could be the follow-up the pitch, in case of erased notes.
The model’s output of this probability distribution over pitches is refined by repeated rewrites. All the pitches are sampled(blocked Gibbs sampling) simultaneously and the partially erased nonsensical scores are rewritten until coherence is achieved aka until the ears don’t bleed.
Composers toil for days, sometimes for a lifetime to make masterpieces. They write, they compose, leave out the parts they dislike and repeat. Just like this model, where the erased notes can be filled in with the new harmonised version.
Coconet is a fairly straightforward CNN with batch normalisation. And, is smart enough to accommodate for any deviation from the usual scores as it rewrites repeatedly until it gets the harmonization right.
So, a user can invent their own melodies which align with those of Bach, even if they are oblivious to the process of reading or writing music; Elvis Presley was.
Read more about autoregressive factorisation here
Play more here
