Friday, July 18, 2008

MonoSLAM Port

At the Robot Vision conference, I was impressed by the presentation 'Realtime visualization of monocular data for 3D reconstruction'. This identified interesting features in a scene and used SLAM technology to track and map them in real time. I thought this would be useful for my object and scene measurement system. Such a system requires identifying objects, comparing them to others and building a map of their relationships. SLAM is great for mapping spacial relationships.
I jumped into this and found that Andrew Davison has made his monocular camera SLAM software available as open-source. So now I am adapting this to my machine: 1. building using Windows and VS9, 2. creating camera input for DirectShow, 3. revising the UI for Windows forms using WTL.
The source includes Davison's MonoSLAMGlow application code, his SceneLib, which implements the SLAM algorithm and VW34, the Oxford Active Vision Lab libraries. My plan was to use OpenCV and DirectX as the input and output for my port, but SceneLib is so centred around OpenGL, that it's a lot less trouble to use that instead of DirectX.
The OpenCV demos would not work with my cameras (I have a cheapo USB webcam and a Canon DV Camcorder that connects by FireWire). I tried their suggested alternatives. Only videoInput worked for both my cameras, but I found it uses a lot of CPU, waaay more than DirectShow, so why not just use DirectShow, which works fine? So that's what I'm doing.
VW34 comes with project files for VS7.1, so it wasn't too much trouble getting those built in VS9. The compiler found a few problems that I must yet report back to Oxford. VW34 has a lot of parts. I only used VNL, VW and VWGL. Most of the other parts are Linux UI adapters. VWGL is an OpenGL adapter that on Windows requires GLUT for Windows.
ScenelLib was built on Linux so I made VS project files for it. While porting Scenelib I found something interesting. It appears as though in GCC you can instantiate an array with a size specified by a variable, like this: int Size = fn(X); char Array [Size]; Microsoft C++ won't allow that! I had to change to malloc(). There's something weird in GeomObjects/calibrationtable.h that causes an inexplicable list of errors, so I removed all references to it.
I built my own camera grabber using DirectShow. DirectShow samples are available in the Windows SDK. I made this grabber run in callback mode, store the bitmap and post a message to the application's main thread. This uses 13% of my Core 2 Duo 6400 (2.13 GHz).
So now I've built a WTL application, linked in the libraries and I'm working on getting the OpenGL visualizations working.

7 comments:

Unknown said...

Hey Ted,

What you are doing is really amazing. Its only highly experienced programmers like you who can accomplish porting this.

I always wanted to use monoslam on windows using a USB cam. But i am an engineering student and dont have the know how on how to do this. Thank you for working on this port

How much have you achieved so far with porting it?

Best of Luck

Unknown said...

btw these pages might be of your interest. these guys have also worked on porting monoslam glow.

http://xuning-cn.blogspot.com/2007/11/monoslam-on-windows-howto.html


http://streebgreebling.blogspot.com/2006/06/monoslam-for-windows.html

gottingen said...

Hi, did you finish the porting? I have the same problem as many people I need a windows and USB version, I have already run the C# version and looked at xuning's web page for the porting operation, the missing and non trivial piece seems to be how to get the USB input in combination with the original Davison (not C#) code. Have you any code or ideas to share for this? Thanks

Ted said...

My MonoSLAM port is not complete. I've been too busy trying to pay the bills to work on it. It does run, and it has video capture using DirectX. Input works for USB and Firewire (DV). The UI is implemented with WTL and looks just like Davison's (output is OpenGL). What's not working is the tracking. My first goal is for it just follow features seen by a stationary camera. It's having problems and crashing while managing features to track. I don't understand it yet. Maybe I'm missing something or just operating it incorrectly. Modifications to the tracking code are minimal. It's a VS2008 project. I have the code available for anyone who emails me. If you solve the tracking problems please let me know and I'll update the code. It's GPL.

Anonymous said...

Hello

im stuck at the point where im not sure how to get the camera calibration parameters for my usb camera.
do you have any idea/suggestions?

many thanks

Ted said...

Nope, sorry, never got into calibration.

chang said...

Hi Ted,
How is your code working now? I am working on a robot with only monocular camera. I am using windows & VS2008. It would really be helpful if monoslam can work well. I read Davison's papers and understand the mathematical models. Would you mind provide me the code and I can look into the tracking problem, if lucky, get it solved:)