It is pretty cool technology but I think you're going to find that the processing time is huge. I looked at some research going on using opencv along these lines and you were looking at processing a 1 minute video in about 2h of processing. I'm sure these guys have managed to optimize it but I think it has to be in the 1:1 range to be feasible.
I do think the future will be gopro type cams with GPS and gyro for this type of processing. By cutting out all the processing to figure out where the camera is in 3d space it would definitely cut processing time down significantly.