Monday, January 28, 2008

Tune Searching

You know when you remember a bit of melody, but can't place the song? I've got an idea to create a website to accept input of musical notation and find out what the song is and return information, MP3's and sheet music. So all we need is a simple notation input, a database of notation for all songs in history, rights to use the notation, a fast and smart matching algorithm, a web front end, and we've got a music search site!

Input:
  • Maybe make the PC's qwerty keyboard act like a musical keyboard
  • Capture from a midi device
  • Put a musical keyboard on the screen & let them click on it
  • Accept input from microphone and decode the essentials to notation
  • Show their input on screen in some form of notation
  • Allow playback so they can interactively adjust it until it sounds right to them
  • Include some way to adjust the tempo (slider)
  • Include some way to adjust the spacing (timing of each note), like stretching it or a slider
  • Include something to adjust the timbre / instrument they hear on playback

Database:
  • Once the input is in, convert it to some kind of text or binary
  • Encode the database in the same format and search
  • We'll get snippets as input and it'll be timed wrong, out of key and notes will be wrong
  • We'll want closest matches

A brief look for 'music search' returns nothing. All music search is by artist, song name etc, even for sheet music. It seems somebody tried something like what I want years ago at ThemeFinder - I don't understand what input they're asking for. I found out about Abc notation, which stores notation as text, and has software that can play midi and generate musical notation. More looking. 'tunesearch' gives Richard Robinson's Tunebook Search and JC's ABC Tune Match at trillian.mit.edu. They all seem related to ThemeFinder.

Then I found Musipedia. Musipedia is almost what I was planning. It lets you input by keyboard, by drawing notes, tapping in the rythm or by humming, singing or whistling. All the stuff I thought of except the editing. And ugly and kludgy. I tried it. I suck at keyboards and didn't take the time to put in Pachelbel's Canon well enough that I recognized it on playback. It was - almost. Of course it didn't match that to anything I knew. When I whistled it, the closest thing it found that I knew was The Doors' 'People Are Strange'! It didn't find Elvis' 'Hound dog' by me tapping either. I think it's database is limited. The database is like a wiki though - anybody can add more tunes. This is a great idea. Maybe it's search is the problem. I'm disappointed. Musipedia is based on a prize-winning retrieval mechanism by Rainer Typke. I bought his book.

Musipedia included a Google ad from Midomi. At Midomi you to sing into your microphone and it finds a match for what you sing. In contrast to Musipedia, this site is slick, and it works. It helped me get my microphone level correct. When I sang 'I'm Leaving On A Jet Plane' (after doing some talking first) it knew it. It uses more than one rendition of a song to make a match. It encourages people to sing songs that they know and it uses these in its database. It seems to be getting lots of karaoke people.

I don't know if these guys are making money because they're both covered with Google ads. Musipedia could use some polish and Midomi needs instrumental input. So should I persue my idea?

2 comments:

DRB said...

Very cool idea! I think that gaining "rights to use the notation" is going to be the biggest stumbling block. It is too bad because this concept could also be used to find new music! If I am in a certain mood I could just hum for a bit and it could make up a play list for me.

(btw try searching 'hum a tune')
also check out:
http://www.musicovery.com/

Ted said...

Musicovery is cool, thanks!
It can't play any dark calm disco!
For dark, calm gospel it picked Ballin' the Jack!
I looked up 'hum a tune' and found something new - songtapper.com
A few blogs all pointing to midomi or Musipedia
Looks like this is a small market, but Melodis, who own midomi, are looking for some high-priced help, so they see some potential.