Images from current CMOS cameras exhibit complex geometric distortions due to "rolling shutter" as an inherent result of the design of CMOS sensor chips.
While software can attempt to remove the effect of rolling shutter distortion from images, there are several unavoidable limitations:
- It introduces a resampling of the image, softening it.
- It introduces unavoidable image artifacts, especially around the edges of objects that are moving independently or at a greatly different distance from the background.
Given that rolling shutter distortion will be present, this white paper describes how to integrate computer generated imagery (CGI) with rolling shutter footage, based on existing workflow for lens distortion processing. This workflow requires appropriate software algorithms, which we will describe, for
- Match-moving (available in SynthEyes 1208 and later) and
- Rendering
With this workflow, the source imagery, with rolling shutter distortion present, is match-moved to produce a match-move that corresponds to an idealized camera with no rolling shutter. New CGI elements are created using a renderer that simulates the rolling shutter of the original camera. These CGI elements can then be composited directly with the original unmodified source imagery. The final delivered footage is therefore as close as possible to the original. Using the appropriate software, rolling shutter footage can be handled in existing lens distortion handling workflows with few changes. A major point of this white paper is to advise renderer developers of the need to be able to simulate rolling shutter, and provide information on what is required.