Plenocap aims to study and enhance plenoptic capture systems, develop computational methods and algorithms, and investigate their effect on future immersive video communication services.

Introduction

Computational photography is a fast growing multi-disciplinary research area building on fields such as applied optics, signal processing, and computer graphics.

A plenoptic capturing system acquires a multidimensional (commonly 4D or 5D) image representation of a given scene, contrary to the 2D image captured by a conventional camera. These two factors combined allows for a plenoptic capture and computational photography platform that opens up the possibility to synthetically produce novel images. In addition, it provides means to infer properties from the acquired high-dimensional dataset that are impossible to achieve using traditional imaging.

Examples of fundamental image and light properties that can be computed after the time of capture are (but not limited to) refocusing, change of viewpoint, 3D depth extraction, occlusion removal, separation of direct and global light, instant measurement of a surface’s reflectance function.

Motivation

Current plenoptic cameras are mainly derivatives of the beginning of 1900 and incremental improvements of the design proposed in 1992 . In brief, a lenslet array is added to a conventional camera thereby multiplexing spatial and angular information such that it is captured by a two-dimensional planar image sensor. Moreover, these previously proposed designs also share a common property when capturing the incoming light field: “in a single snapshot”. While providing an exteded range of possible applications compared to conventional camera systems these plenoptic cameras are somewhat limited by the constraints imposed by the optical properties of the 2D camera system on which they are built.

Purpose

The work planned for this project will waive the boundary conditions stemming from conventional two-dimensional cameras, in favor of addressing the plenoptic capture problem from the perspective of capturing a multi-dimensional data set as efficiently as possible given the requirements of a plenoptic application, not a 2D equivalent.

This relaxation of constraints significantly increases the potential of novel and distinguished results and opens up a new research branch within the community that can be pursued with both analytical and empirical research tools. New multidimensional data sets require analysis of their characteristics (using simulation and experiment), which in turn requires defining evaluation metrics that are capable of addressing important plenoptic signal and system properties.