Google Research Blog: "Pan and Scan" for Mobile Video

"Pan and scan" is a technique long used to fit wider movie images onto an analog NTSC TV screen. Engineers at Google say they now can do the same thing for video to be displayed on a mobile screen, rather than a larger PC screen.

Videos come in different sizes, resolutions and aspect ratios, but the device used for playback, may it be your TV, mobile phone, or laptop, only has a fixed resolution and form factor, Google engineers say.

As a result, you cannot watch your favorite old show that came in 4:3 on your new 16:9 HDTV without having black bars on the side, referred to as letterboxing.

Likewise, widescreen movies and user-videos uploaded on YouTube are shot using various cameras with wide-ranging formats, so they do not fit completely on the screen. As an alternative to letterboxing, several devices try to upscale the content uniformly, which either changes the aspect ratio, making everything look stretched out, or simply crop the frame, thereby discarding any content that cannot fit the screen after scaling.

Google Research, together with collaborators from Georgia Tech, says it has developed an algorithm that resizes (or retargets) videos to fit the form factor of a given device without cropping, stretching or letterboxing.

At some point, this is going to be important for providers of videos that will be viewed on mobile devices, and the business ecosystem (advertising, marketing, transactions) that will grow up around mobile video.

Official Google Research Blog: Discontinuous Seam Carving for Video Retargeting