Posted by Nikola Bozinovic, VP Engineering

The Technology Behind FixMyMovie

Today, we'll shed some light on what happens behind the scenes of FixMyMovie.com. The service is easy to use: you upload your movie and we fix it. But if you're interested, here's a little more information on what actually happens to your video when you send it to us.

As you probably know, every video is really a series of rapid snapshots.

Most consumer cameras capture between 5 and 30 frames every second. When you take even a short video, you end up with a large number of frames. This "abundance" of frames is the reason why we perceive movies as something "live." But it's also the main reason why your video can be improved. Simply put, we recover information from multiple frames of video.

When you upload a movie to www.fixmymovie.com (or e-mail it to upload@fixmymovie.com ), our servers analyze how the neighboring frames in your clip are related to one another. This is called "motion estimation." But unlike the relatively simplistic (block-based) motion estimation used in video compression, our algorithms estimate motion to a much higher accuracy, as they are relying on the more natural motion representation of the actual objects and scene of the video. Take a look at the details in this bookshelf clip, and observe what is visible in the four video frames:

Original input frames	Enhanced output

You can see that each of the original video frames (left) offers a slightly different view of the same object (you may want to look carefully to confirm that there is a difference between them). In this clip, different views are caused by camera shake. Accurate measurements of the relative movement between frames allow us to combine these low-resolution frames into a single high-resolution image.

For example, take a look at the letter S (highlighted in the first frame). You can guess that the same letter S is visible all other frames, but you probably wouldn't want to bet on it. The clear shape of the letter is revealed only after the information from all low-res frames is combined together (right). We do this for every frame of video, one by one, and then put them together in a high-res video. This is the result that you can see and download from FixMyMovie.com.

The main power of the technology behind FixMyMovie.com is that it can quickly and reliably identify the same objects in different frames of video. The more frames you combine in the process, the better the final result is (on FixMyMovie.com we use between 7 and 11 frames to generate each output frame).

In addition to combining multiple frames of video, we deploy many other techniques to make your video look better. These include automatic lighting correction (especially important when fixing dark videos), sharpening of blurry clips (correcting for poor camera lenses), and video deblocking (improving heavily-compressed videos). We'll write about these in the future so stay tuned, and keep fixing your movies!

If you have questions about how FixMyMovie.com works, send them to us, and we'll respond in the next newsletter.

Thanks,
Nikola Bozinovic
VP of Engineering