Wednesday, September 11, 2013

Retinal Fields for Colours?

When I use a centre-surround field to detect blobs, it does not see apples in a tree
I think this is because I've transformed the input to grey-scale
Because of this red and green are pretty much the same intensity and therefore a red apple with a green background exhibits no contrast
But what if fields compared different colours?
Could contrast the primaries:
  Red-green - this would see apple in tree
  Red-Blue - this would see autumn trees against sky
      Red is dirt, fur, fire, flowers
  Green-blue - this would see trees against sky
Blue is good for shadows, water, sky
Also white and black against primaries:
   Black is night and shadow, also fur, rocks
  Grey is rocks, clouds, shadows, tree trunks, elephants
  White is snow, clouds, reflectance, flowers
This way I'd get more blobs, with specialization, to provide expectations. I'll try it.

Saturday, December 15, 2012

Voice Communications Plugin


Nimajin has created the voice communications add-on for Signalsoft railway dispatching simulations. Previously in the simulations with Dispatcher Centre installed, you could communicate with other players on the network, but only by sending text messages. I guarantee you that not many real dispatchers texted each other in the 1900's! Now, with a headset and the Voice Communications Plugin, you can hear and talk to your fellow dispatchers. And your gameplay is not distracted by all that typing. The sound is clear and the networking does not conflict with any VoIP you may have installed.
Get the Nimajin Voice Communications Plugin soon at http://shop2.signalsoft.nl/index.php
Learn more about Signalsoft railway dispatching simulations at http://www.signalsimulation.com

Monday, February 14, 2011

Script Apps In Smartphones Considered Evil

One of the virtues of Linux - that anybody can write their programs any way they like - can become a detriment in the limited resources of the Android environment over the long term.

Consider SL4A (Scripting Layer for Android). It provides a way for developers to write Android apps in scripting languages like Python, JavaScript and JRuby. If the programmer chooses to write their application in Python, then what happens in the phone? The script is interpreted by Python running in SL4A running in Java in Android. All of the dependent libraries have to be loaded. Look at all that RAM being used! The script gets interpreted, converted to Java API and run on the native OS. That's 3 layers over top of the native OS. Look how slow that executes! If, over time, the user downloads several apps, all written in scripting languages, and runs them, there may be interpreters for Python and Lua and Perl all loaded - a huge demand for RAM and execution time. Will the thing thrash memory? Yeah.

The phone makers will compensate for this with more RAM and faster CPU's. That will mean more energy consumption, less battery life or bigger batteries, more weight and more heat.

From the usability perspective, to support scripted apps, the users have the extra complication of having to download the software dependencies. Because the supporting libraries and interpreters have to be installed, the storage fills up.

For the sake of the user and the ecology, write your apps as close to the metal as you can!

(I won't say browser apps are as bad, because the browser is already in the device)

Wednesday, November 10, 2010

Smartphone traffic on wireless networks will increase 700 percent over the next five years


My title was extracted from this article at VON that points out just how pervasive wireless device networking is becoming and how hard it is for the wireless carriers to provide the service. I am really not surprised. How could you not like having access to all of the Net wherever you are? An article in InformationWeek stated that car sales staff were not happy with WiFi iPads because they couldn't access the dealer database while they out on a test drive with a customer. I guess once you start using a smartphone, how could you stop? It's just natural. You can get TV and movies on your phone or pad from NetFlix for $8 per month! That'll congest the networks more! Still, I don't want to pay Bell $100 per month for 3GB of data. Why so much, when my landline Net costs $30 unlimited? Don't worry, the carriers are struggling through technology changes, but they'll find a way, and even reduce prices - because they can see the huge market.

Monday, September 28, 2009

On Color, Materials and Lighting

First I should say what everyone says - yeah it's been a while. One excuse is I always think it'll take me a long time to write anything here so I put it off. That's because I want to write well, so I spend a lot of time expanding explanations. Maybe I can put more stuff in this blog if I just write my fast way - which is usually point lists.

Reading 'OpenGL SuperBible' by Richard Wright has given me something to say. I'm at the chapter on Color, Materials and Lighting. I think specularity should be a property of the material, not a lighting property. Every surface should have a reflectance specification. Then the amount of normal light that bounces off surface would be calculated.

Thus one would specify 'dull', 'matte', 'bumpy', 'shiny', 'glass' for surface (numerically). This could be specified per-vertex, like colours, and get gradiented like colours are. Or reflectance surface could be like a texture, but each pixel is a reflectance value 0.0-1.0. This would just specify the proportion of light that is reflected - at incidental angle. Or each point of the reflectance surface could be a vector showing which way that point points, so the renderer could calculate where the light reflects to.

You wouldn't need a specular light, or even a diffuse light. The rendering engine could calculate the effects of light bouncing off a shiny surface onto other surfaces for diffuse light. It will take more CPU, but provide more realism.

Because really what you want is to say there's a huge sun millions of miles away, but only its rays provide the primary luminance, everything else is reflected.

Also you want to add diffusion property for volumes, to define what happens to light passing through gas or water or glass. This is internal reflectance.

By the way, I have updated the projects list on the Nimajin web site.

So, it took 10 minutes to get the lighting points up, then another 20 for the picture, introduction, advertising and editing (and another 10 to fix fonts and spacing!).

Monday, April 6, 2009

TimetablEd Now Does Orders And Consists

The Post T railroad simulator timetable editor is gaining new capabilities rapidly! The main panel has been reorganized to show movement orders. This was made possible by compacting the operating days into a new selector that shows the days by their initial letter, in the same way shown by the Stops chart.

The Stops chart can show you trains running on only the same days as the train you're working on, or trains running on any days you select. In the Stops chart you can change the track a train stops on by dragging its bar up and down, or you can change the stop times by dragging left or right. You can change these in the Stops list as well, and there you can also add new stops and delete them.

The train's movement orders are shown in text on the train panel, and can be changed in the Edit Movement Orders window. This windows shows all of the movement orders in the system on the left and the details of the selected order on the right. What's shown on the right depends on the type of order. Shown here is an uncouple order with the convenient graphical decouple point selector just like in the simulator.

I found it easy to get lost in the maze of movement orders, so I added the 'Find' buttons. When you click one of these the editor shows you the order that produced or consumes the item you're looking for, so you can easily follow the trails of the parts of your train.

The train consist editor has been added. You can edit templates that may be used to create many trains that are the same, or create custom consists for unique trains. At the top is shown the consist of the train. These are represented as a list of vehicles in the same way as the uncouple order, but here extra information is included. The bottom line of each vehicle shows the coupler type at each end so its easy to match couplers. In the centre are shown letters for Traction, Electric (or Steam) and Passenger. An L in the top right corners tells you that the vehicle is loaded. You build trains by selecting a vehicle from the vertical list and adding to the consist. For any vehicle selected in the consist or the vehicles list, the editor shows all of its characteristics.

So the editor is nearing completion! Still to do is the timetable printing. And yes, there will be an easy method of scheduling regular trains. My goal is to make the most convenient railway schedule editor. I will consider any suggestions.

Wednesday, March 4, 2009

TimetablEd

Screenshot

In co-operation with SignalSimulation.com / Signalsoft Rail Consultancy Ltd. / Richard Plokhaar, railroad consultant and railway simulation developer, I am developing a new railway schedule editor. The editor makes it easy to set up the trains that will run through the new Signalsoft Post T series of railroad simulators.

While playing the simulator you are the dispatcher who must control the signalling and switching to allow the trains to stop and pass through your area of responsibility on schedule - the challenge is you are using 1950's era technology for communications and control! The simulation platform accurately reproduces train physics and events that effect their travel over your rails. It is useful as a simulation for railroad enthusiasts or as a complete simulation platform for modeling and analyzing real-world railroad traffic.

The timetable editor features:

  • Entry locations & times, exit location
  • Station stop locations & times
  • Arrival, departure & pass-through events
  • Textual & graphical representation & editing for easy conflict resolution
  • Special orders editing (train renumbering, direction switching, coupling & uncoupling of complete consists)
  • Train consist editing: locomotive & car selection
  • Special train & activity designation
  • Timetable printing

There will be a series of simulators, modeling various real-world railway centers. A simulator demo is available now at www.signalsimulation.com/content/category/3/7/7/

More information about the Post T series is available at www.signalsimulation.com

The editor and the first full simulation are in final beta testing stage and will be available spring 2009

Wednesday, November 12, 2008

Who Gets the Money?


I'll post a question just for fun, because everyone loves politics!
My question is: The oil industry makes massive profits - is it better for that money to be controlled by the shareholders, or would it be better if consumer prices were reduced so there were no profits and the money was left in control of the consumers? Who should get the money: investors/shareholders or people who did not participate in the development of the resource, but only consume the product?
It's not like the money does not get spent. Investors do spend the money on other ventures. Maybe they build the Burj Dubai. When consumers spend most of it goes to the local community for food, shelter and services.
The question came from a twisted path. I've been a contributor to Greenpeace for 20 years and I get a quarterly magazine. This month they pleaded to control the Alberta oil sands projects that are using and polluting water and stripping the land and killing birds. I don't agree with them entirely. Oil is important and we have to balance the good with the bad.
My view on water (tailing ponds) is that it always evaporates and recycles. You cannot run out of water and it always cleans itself in the long run. So using water in any amount is not bad.
My view on the land (surface mining) is that nature will reclaim it. It was not clean in the first place - it was full of oil. What they return after extraction is dirt with less oil. I don't see a problem there. Over time nature will blow in sand and soil and it'll be the same as it was. So using land is OK (but only up to the point where the reduction of vegetation threatens the support all of the life on the planet).
Burning gas to provide heat for the extraction process raises the greenhouse gas emissions and I think that is bad.
Anyway the project is huge. Being a tech nerd, I wanted to see how huge it is, so I went to look at it on Google maps (I entered 'Fort McMurray Alberta' - just look for the huge grey patch north of there). You can see the tailing ponds, you can see the huge dump trucks.
I thought I could get better pictures in Wikipedia. I read the Wikipedia articles on oil sands and Athabasca oil sands to learn the history, technology and economics (I love Wikipedia!). The articles say that they are building more mines out there right now, so there will be construction work out there for at least ten years. So I'm thinking, if I really get broke, there is work out there.
While describing the economics, they go into how it costs like $25 to produce a barrel of oil, which they can sell for $60 to $100. I'm thinking: that's an outrageous profit! Is it moral? If they just dropped the selling price to $30, reducing their profit, a huge amount of money would be left for the consumers to spend on other stuff, which would provide jobs for people. As it is, the investors use the profits to capitalize other ventures, which provide jobs for people.
So which is better? That the consumer or the producer gets to control the money… Oh, I won't give an answer. It's a really big question.
I might have posted this because I just read 'Trinity' by Leon Uris, about the English exploitation of Ireland, or maybe not. Who knows how minds work?

Tuesday, July 29, 2008

Centres are Circular Things

If the receptive field centre covers a large area, it will provide a really big output when it's focused on a round object (that fits within the centre).

Centre / surrounds make their biggest output when the centre is stimulated, and the surround is inhibited, like on bright points of light in the dark, like stars at night. Colour versions ought to provide their biggest outputs on round things that contrast with their background, like berries or fruits. Or maybe larger round things like flowers or rocks. Or maybe almost round things like insects or fish. Or maybe distant almost round things like birds in the sky. Then, also, large receptive fields could trigger on partially round things, like rounded corners or fingertips.
So maybe there's not really a curve detector in a brain at all? Maybe curves are detected by partial outputs from the centres of large receptive fields. If it contrasted enough, a semicircle in the centre of a receptive field would cause a positive signal. If the brain knows, 'hey, that positive signal came from a big receptive field', then it could assume that a curved thing is in the field, with its size relative to the receptive field size.

Centres Can Be Corners

Centre/surround receptive fields can detect corners!


Look at the diagrams: corners and angles produce stronger outputs than edges, but weaker than centre dots. If the next layer evaluates its input from receptive fields carefully (i.e. looks for values between edges and dots), it can inform its superiors that a corner is in its field. If that next layer is also getting input from edge detectors, one of these inputs can strengthen its corner detection.

Of course these mid-range values can also indicate some other messy patterns, but that kind of stuff can be neutralized by expectation.

Friday, July 18, 2008

MonoSLAM Port

At the Robot Vision conference, I was impressed by the presentation 'Realtime visualization of monocular data for 3D reconstruction'. This identified interesting features in a scene and used SLAM technology to track and map them in real time. I thought this would be useful for my object and scene measurement system. Such a system requires identifying objects, comparing them to others and building a map of their relationships. SLAM is great for mapping spacial relationships.
I jumped into this and found that Andrew Davison has made his monocular camera SLAM software available as open-source. So now I am adapting this to my machine: 1. building using Windows and VS9, 2. creating camera input for DirectShow, 3. revising the UI for Windows forms using WTL.
The source includes Davison's MonoSLAMGlow application code, his SceneLib, which implements the SLAM algorithm and VW34, the Oxford Active Vision Lab libraries. My plan was to use OpenCV and DirectX as the input and output for my port, but SceneLib is so centred around OpenGL, that it's a lot less trouble to use that instead of DirectX.
The OpenCV demos would not work with my cameras (I have a cheapo USB webcam and a Canon DV Camcorder that connects by FireWire). I tried their suggested alternatives. Only videoInput worked for both my cameras, but I found it uses a lot of CPU, waaay more than DirectShow, so why not just use DirectShow, which works fine? So that's what I'm doing.
VW34 comes with project files for VS7.1, so it wasn't too much trouble getting those built in VS9. The compiler found a few problems that I must yet report back to Oxford. VW34 has a lot of parts. I only used VNL, VW and VWGL. Most of the other parts are Linux UI adapters. VWGL is an OpenGL adapter that on Windows requires GLUT for Windows.
ScenelLib was built on Linux so I made VS project files for it. While porting Scenelib I found something interesting. It appears as though in GCC you can instantiate an array with a size specified by a variable, like this: int Size = fn(X); char Array [Size]; Microsoft C++ won't allow that! I had to change to malloc(). There's something weird in GeomObjects/calibrationtable.h that causes an inexplicable list of errors, so I removed all references to it.
I built my own camera grabber using DirectShow. DirectShow samples are available in the Windows SDK. I made this grabber run in callback mode, store the bitmap and post a message to the application's main thread. This uses 13% of my Core 2 Duo 6400 (2.13 GHz).
So now I've built a WTL application, linked in the libraries and I'm working on getting the OpenGL visualizations working.

Tuesday, July 1, 2008

No Math Required

I don't think the vision problem is mathematical (except to prove that it's accurate or correct). Mathematics can be good for optimizations and specializations and extended functionality. I think simple methods can be used for early vision. The basis should be in the findings of neuroscience. I think the solution at higher stages is extracting the right information from the early stages and representing it in a way that simple pattern matching (using highly interconnected neural networks) will do the job. I think the right information and representation can also come from neuroscience. So that should be my focus. What does neuroscience know?

Tuesday, June 24, 2008

Cloud Robot

I was reading about cloud computing.
So here's an idea:
A robot relying on vision can upload its real-time sensor input to the cloud.
The cloud can extract whatever it likes and present it to humans.
But the cloud will also do all the image processing and download the result (?) back to the robot.
The result could be motor instructions or descriptors for the robot to work with.
So that the robot itself doesn't need a high-powered processor.
All that's needed is real fast wireless link:
640*480*4*60=73,728,000 bytes/sec!
(640x480 pixels, each 4 bytes (RGBI) @ 60 frames/sec)
You could strip that down to 3 bytes/pixel @ 30 frames/sec 640*480*3*30=27,648,000 bytes/sec
OK, 320x240 image = 320*240*3*30=6,912,000 bytes/sec
OK, 256-shade grey scale = 320*240*30=2,304,000 bytes/sec
That's possible!
And that leaves 500 KBs to send stuff back.
Probabaly been done. But maybe not so real time?

Sunday, June 22, 2008

2008 Canadian Robot Vision Conference

This is my report on the Canadian Intelligent Systems Collaborative (AI/GI/CRV/IS) 2008 Conference. This was five conferences in one event. My interest was in the CRV (Computer and Robot Vision) conference held by the Canadian Image Processing and Pattern Recognition Society (CIPPRS).
The buildings in which conference was held were being reconstructed and the noise was distracting. But, with the inconvenience tolerated, the conference was very enlightening for a beginner like me.
The conference was essentially three days, with a keynote each morning and papers or talks throughout each day.
The only keynote I heard was the first, which was by Peter Carbone from Nortel. This was interesting to me (with my history in telecommunications), but off-target for the AI people who were the majority of conference attendees. Mr. Carbone made several predictions that I think will be significant in telecom: widespread broadband wireless penetration by 2010, telephony using SOA with mashup potential, and 100 MBs real-time encryption capability.
Following the keynote the CAIAC Precarn Intelligent Systems Challenge was announced. This offers a $10K prize to the student submitting the best method of detecting ships meeting at sea using satellite and radar tracking data. I think the poor data makes this a challenging problem.

The highlight of the conference for me was a talk by Dr. Steven Zucker from Yale. He was the only presenter who seemed interested in doing AI and computer vision to emulate biology, as I am. I think artificial systems should understand what they're working on; be a part of their world, as biological systems are. A biological goal Zucker identified is guiding animal movement, such as monkeys jumping to tree branches. This objective is the same as Arathorn's example of goats jumping to rocky ledges. Zucker's talk was mainly on stereo vision. He confirmed my assessment that Canny edge detectors suck. He implemented a nice curve detector based on tangents. He showed how he used spacial and orientation disparity to get a better matching of the image pair. A nice point was about self-referential calibration: a system that can move can identify its own parts (e.g. in a mirror) as the ones that move when it moves them.
I missed the talk by Dr. James Crowley, to my regret. I gathered that his points included that intelligence requires embodiment and autonomy. This confirms my subscription to the philosophy of Spinoza, who states that the mind is the entire body. Any organism's mental reality would not be what it is without all of the sensory input and motor feedback provided by the body.
The talk by Dr. Greg Dudek about his AQUA robot was interesting because of the focus and completeness of the project. It's another very specialized machine, although you can program its actions by a visual language. Apparently they discarded a visual system that recognized human hand gestures.

I attended all of the CRV paper presentations. These seemed to be arranged in ascending order of complexity and accomplishment. I was surprised that I could understand much of the work. Some of the papers were not amazing to me at all. Some were incremental improvements on previous work. Most were applications of existing work. This may be a survey of the state of the art, or it may just be a sampling of people who are trying to get attention (who didn't go to other conferences). I'm not going to summarize all of the papers - just give criticisms of the ones I found useful.
'An Efficient Region-Based Background Subtraction Technique' and 'Ray-based Color Image Segmentation' presented image segmentation optimizations based in iterative deduction. This is a good and intuitive idea and I think I can implement it using layers of neural networks. The ray-based segmentation idea was clever, but had problems finding all segments and was slow. I still don't know if colour-based segmentation is natural.
The methods used in 'A Cue to Shading - Elongations Near Intensity Maxima' to differentiate shadows from textures got me confused, but Gipsman's point that knowledge of shading detection is still primitive surprised me - I think analysis of shading would be fundamental to determining shape and orientation of 3D objects. I agree with her that feedback from higher layers will be essential. But I think the feedback will loop: the shape of the shadow will help in recognizing the object and the shape of the object will help in recognizing the shadow.
'Fast Normal Map Acquisition Using an LCD Screen Emitting Gradient Patterns' presents an innovative method for lighting objects to get 3D information. An interesting point is their use of the polarized LCD light and a filter to remove specular reflection. I found later that the human eye can differentiate linear from non-linearly polarized light (see Haidinger's Brush). Perhaps the brain can use this information in determining where the light is really coming from?
'Realtime visualization of monocular data for 3D reconstruction' was a treat for me because it relates so well to my planned measurement with a camera project. To me, this paper is like an instruction book on how to model 3D space from a single camera. I must look into its Simultaneous Localization and Mapping (SLAM) methods and other tricks. Monocular is cheap, stereo is more accurate? Again, the system doesn't have a clue what its looking at, but it may be a good start for a more complex system. I must analyze to more depth.
'Object Class Recognition using Quadrangles' is a general-purpose implementation of edge-based object recognition which also considers colour uniform regions. On top of this the authors implemented a structural descriptor (the paper describes quadrangles only, but the speaker described use of ellipse descriptors in their newer work) and template-based spacial relationship matching system much simpler than that used by Sinisa Todorovic's self-learning segment-based system (but less capable). 'Geometrical Primitives for the Classification of Images Containing Structural Cartographic Objects' is another system base on edge/ region/ structural descriptor, but focused on the single problem of finding roads, bridges and such in satellite imagery. The software seems to be more capable, handling higher level primitives such as blobs, polygons, arcs and junctions. It uses AdaBoost binary classification. Results were good except for detecting bridges. I suggest looking for the bridge shadows.
Most of the motion tracking papers used very simple recognition techniques or did not describe them. '3D Human Motion Tracking Using Dynamic Probabilistic Latent Semantic Analysis' presents a highly mathematical approach that seems to be another form of template matching. It works well but it will take quite an effort for me to understand it. 'Visual-Model Based Spatial Tracking in the Presence of Occlusions' presents a pre-processing trick to mask occlusions from a template/visual-model based 3D tracking system. While the system is highly performant, using the GPU, it is highly specialized to a single object. 'Automatically Detecting and Tracking People Walking Through Transparent Door with Vision' tracks Harris corners through time. It can be taught to subtract expected movements from new ones, by simple geometric trajectory comparison. This can serve many applications, but the use of just corners means it can't tell you what is moving through the scene. But could specializations like this be used as keys to brain behaviour? e.g. does the brain just use the moving corners of a door to perceive it? 'Invariant Classification of Gait Types' classifies body movements by comparison to a database of shape contexts derived from template silhouettes. This is an efficient and accurate method used in handwriting recognition, and I think I'll look into it more, because the bin concept applied to pattern matching lends itself to implementation using neural networks.
'Active Vision for Door Localization and Door Opening using Playbot' is another specialization - for doorframes and handles. Its advance is active vision. The robot solves its position geometrically after it detects the door by using a pre-programmed door size - meaning it will only work with one size of door. The active vision part is that the robot takes pictures at multiple angles and multiple positions and solves its position using the camera angles and the detected door edges and corners, then calculates a move to a new position. 'Automatic Pyramidal Intensity-based Laser Scan Matcher for 3D Modeling of Large Scale Unstructured Environments' tackles an incredibly hard problem of mosaicing adjacent spherical laser images without feature, location or rotation information by matching their overlapping depth values. This is useful for other mosaicing problems, but I don't think its complicated methods will be required in most computer vision applications which will have less interval between images and can rely on feature detection and sense of place. '6D Vision Goes Fisheye for Intersection Assistance' shows that using fisheye lenses provides a wider angle of view with only small hits on their low processing time requirements and relatively wide accuracy requirements in a real-time stereo mobile object tracking application. 'Challenges of Vision for Real-Time Sensor Based Control' explains how additional sensor input to an extended Kalman filter can be used to supplement poor video data caused by bad camera angles.

One thing I brought away from this conference is that although there is a large amount of existing work and many new efforts in the computer vision field, the presented applications are not trying to understand or duplicate biology. They're using mathematical methods to solve specific problems. Well perhaps the concepts can be implemented in neural networks. And the solutions are so specific! I guess it'll be a long time until there is general purpose vision. And not surprisingly so, because that will require general purpose concept representation. Too bad I didn't hear the AI papers too.
Since this was my first academic conference, I learned that what to look for in papers is what is new, or what can be adapted to my purpose. Attending has motivated me to get an IEEE membership so I can access more research papers. Poster presentations seem pretty valueless to me. Either they don't present enough information or I am forced to stand while reading an entire paper.
Another thing is that there is a lot of existing technology out there that can be used to solve problems. A counterpoint to this and a kind of semi-corollary to the first point is that a lot of the existing technology is highly focused, inaccurate and slow, so there is still a lot of research and development needed.
From a business point of view, I got no leads on paying work. Some people at the conference believe that contracting in this field can be viable. But I think I'll have to prove I'm capable by example before anyone will hire me. Since most researchers only solve special cases, another opportunity is to complete a project to make it useful in lots of situations.

Sunday, April 13, 2008

Web Site Update

Nimajin website
I've been busy. Not making any money, but busy. When the jobs aren't pouring in it's time to concentrate on marketing. So I'm prospecting and networking. And I've updated the Nimajin website. It looks much more professional now. I've added a resume and portfolio so people can learn more about me. I'm not real happy with the site yet - it concentrates on my past and not my future. So it'll change again.
The future is not real clear to me now. I like to be associated with science and communications. My intention to concentrate on media processing and content recognition is still attractive to me, but I'm not finding much interest from others. I think it's mostly because of my lack of communication. I'll work on that. Some friends say I've got to jump on the web services bandwagon to make a living. Improving people's advertising or bookkeeping is very useful, but not as incredibly fascinating to me as making a machine able to draw a floor plan of my house from photographs or making it able to fly me through town from surveillance camera input. I know what it'll take to do these and I'll keep working on them, but it'd be great to have some sponsorship so I could afford some more time to work on them.
I think this happens to a lot of people. There's not enough commercial value to our ambitions of making machines more intelligent, so over time we have to abandon our efforts and go where the money is. I'm still holding on!
That isn't really where I wanted to go with this post, but heck I'll post anything. My goal is to perkily point out how great the new web site is and how ready I am to go an extra mile, learn more new technology and build anything you can think of. What I really love is making people happy.

Monday, January 28, 2008

Tune Searching

You know when you remember a bit of melody, but can't place the song? I've got an idea to create a website to accept input of musical notation and find out what the song is and return information, MP3's and sheet music. So all we need is a simple notation input, a database of notation for all songs in history, rights to use the notation, a fast and smart matching algorithm, a web front end, and we've got a music search site!

Input:
  • Maybe make the PC's qwerty keyboard act like a musical keyboard
  • Capture from a midi device
  • Put a musical keyboard on the screen & let them click on it
  • Accept input from microphone and decode the essentials to notation
  • Show their input on screen in some form of notation
  • Allow playback so they can interactively adjust it until it sounds right to them
  • Include some way to adjust the tempo (slider)
  • Include some way to adjust the spacing (timing of each note), like stretching it or a slider
  • Include something to adjust the timbre / instrument they hear on playback

Database:
  • Once the input is in, convert it to some kind of text or binary
  • Encode the database in the same format and search
  • We'll get snippets as input and it'll be timed wrong, out of key and notes will be wrong
  • We'll want closest matches

A brief look for 'music search' returns nothing. All music search is by artist, song name etc, even for sheet music. It seems somebody tried something like what I want years ago at ThemeFinder - I don't understand what input they're asking for. I found out about Abc notation, which stores notation as text, and has software that can play midi and generate musical notation. More looking. 'tunesearch' gives Richard Robinson's Tunebook Search and JC's ABC Tune Match at trillian.mit.edu. They all seem related to ThemeFinder.

Then I found Musipedia. Musipedia is almost what I was planning. It lets you input by keyboard, by drawing notes, tapping in the rythm or by humming, singing or whistling. All the stuff I thought of except the editing. And ugly and kludgy. I tried it. I suck at keyboards and didn't take the time to put in Pachelbel's Canon well enough that I recognized it on playback. It was - almost. Of course it didn't match that to anything I knew. When I whistled it, the closest thing it found that I knew was The Doors' 'People Are Strange'! It didn't find Elvis' 'Hound dog' by me tapping either. I think it's database is limited. The database is like a wiki though - anybody can add more tunes. This is a great idea. Maybe it's search is the problem. I'm disappointed. Musipedia is based on a prize-winning retrieval mechanism by Rainer Typke. I bought his book.

Musipedia included a Google ad from Midomi. At Midomi you to sing into your microphone and it finds a match for what you sing. In contrast to Musipedia, this site is slick, and it works. It helped me get my microphone level correct. When I sang 'I'm Leaving On A Jet Plane' (after doing some talking first) it knew it. It uses more than one rendition of a song to make a match. It encourages people to sing songs that they know and it uses these in its database. It seems to be getting lots of karaoke people.

I don't know if these guys are making money because they're both covered with Google ads. Musipedia could use some polish and Midomi needs instrumental input. So should I persue my idea?

Thursday, January 17, 2008

The Future is Coming!

Software UI is coming to look and act more like Star Trek control consoles.

The iPhone's UI looks like Star Trek graphics. Check out the black background and brightly coloured icons, all behind a glossy glass surface.
Office 2007 is starting to look like that and all of the WPF examples too.


But the really big deal is that Star Trek consoles are touch screens. They are operated by tapping on them. The iPhone operates by tapping, pinching, dragging, flicking, etc. The Microsoft Surface is so like a Star Trek console. It's a big, bright touch console. We are there!


I'm real happy about this. I've always loved the way the consoles looked in Star Trek. I want to be flicking chunks of code around in one of those one day. It makes me wonder whether all of the computer developers love Star Trek consoles too, or if the Star Trek set designers just correctly imagined the future.

Tuesday, January 15, 2008

Vista Spooler.xml

I was working away on my less than year old Vista PC with a 250 Gig hard drive when Windows tells me the disk is getting full! Windows is all like 'disk cleanup' and 'remove some programs'. Serious! Was it the VS2008 I just installed? No.
I used Silurian DiskSpaceChart to find out where all my disk was going. That's a typical disk usage pie chart thing that puts itself in the right menu. You can drill down through the large folders to find the large file. It's OK. It was the first thing I found for Vista.
So what I found was something writing continuously to C:/Windows/system32/spool/spooler.xml, at like 300 MB per minute! I tried to find out who was writing the file. Resource Monitor helpfully identified 'system'. Thanks. I didn't have SysInternals Process Monitor installed, and I didn't want to try while the machine was so sick, so I couldn't get any details on who was doing all that writing.
I rebooted into safe mode and deleted the file.
When I booted back the file was 33k and stayed that way. I think the file is a printer log and the problem is either my HP printer drivers for the 3390 or MS XPS. I recall running WireShark and seeing every machine with those 3390 drivers polling the printer status over the LAN every millisecond.

Sunday, December 16, 2007

Why Straight Lines?

Why should a visual recognition system expect straight lines? The arrangement of photoreceptors in the retina has no straight lines. Very little in nature that may be seen has straight lines. Primitive people do not build angular objects. Yet, I think straightness came from building things. What could be the first straight thing? Baskets? Arrows? Lines drawn in clay? Not even knife edges were straight. Maybe the horizon - out over the ocean. There is no survival value in building a recognition system within an animal upon a framework of straight lines. Except maybe for orientation to the horizon.
Why did Hubel and Wiesel concentrate on straight lines?
The things that are important to recognize are curved. In all directions. Food like berries, fruits, grasses and animals are all curved. The path upon which you walk. The trees over your head.
So can a recognition system be built upon collections of curves?

Monday, December 10, 2007

Artificial Intelligence: Answer the Questions

The point of AI is to get answers to questions.

In music:

  • What is that instrument?
  • Who is that artist?
  • What else have they done?
  • What is similar?
  • Has this been done before?
  • How is that sound made?
  • What are the words?
  • What is its name?
  • Where was this performed?

In images and video:

  • What is that thing?
  • Who made it?
  • What else have they made?
  • What is similar?
  • Where can I get one?
  • Who is that person?
  • What is it made from?
  • How old is it?
  • Who made it?
  • Can you show me more details?
  • How did they do that?

About things:

  • What are the parts?
  • How much does it cost?
  • Where can I get one?
  • How does it work?
  • Who made it?
  • Who's working on its development?

Then there are the 'what if' scenarios: Substitute this for that - what happens?

AI is so complex and deep that it is very easy to get lost in the implementation details. The details need to get worked out, but the point is to answer questions.