Video as a Panorama Source

There have been some exciting developments in deriving three dimensional geometry of a space from collections of photos (popularized recently by Microsoft researchers), and tools for stitching photos into panoramas are old news. But I'd like to see a tool that stiches video into a panorama in real time (but also producing a final still image). It should account for any panning, rotation or zooming of the camera.

One approach would be to continuously deform the image in various ways: shifting the x/y position, changing the size, perspective and skew deform on both axes, and rotation. Once the deformations are known, it becomes a search problem for the minimum difference between the deformed image and where it overlaps with the previous image. Alternatively, x/y/z position (z being zoom) and all three degrees of tilt could be used — which only takes 6 dimensions instead of 8.

Update (10/22/07): This was first done in 1993 by Steve Mann.