Multimedia Representation and Compression

Data compression is a ubiquitous aspect of modern computing, and particularly important when dealing with visual data as this type of data largely dominates global IP traffic. Visual data has evolved drastically away from traditional 2D images and video.

Current sensor technologies capture reality in unprecedented detail, leading to a high variety in modalities, such as HDR content, 360° video, hyperspectral imaging, point clouds (e.g., from 3D scanners), or light field data (e.g., from plenoptic cameras). Besides this, there is also a larger variety in display/visualization technologies, such as auto-stereoscopic screens, multi-view screens, HMDs, immersive 3D projection systems, or even holographic displays. The result is that one no longer just renders the signal that was captured (which is the case with 2D video), and that visual data needs to be processed between capturing and rendering, something that is often coined “computational imaging”.

These evolutions pose several challenges on the representation, compression, storage, and transmission of emerging visual data. Our research team focuses on:

  • Dimensionality. Whereas traditional video data has 3 dimensions, the full plenoptic function has 7. Current (traditional) compression techniques do not scale with dimensionality and are unable to exploit the redundancies at hand. Even for multi-view video (only 4D), current video coding techniques are inadequate. Radically new techniques are necessary in order to achieve an efficient sparse representation for high-dimensional visual data.
  • Interactivity. When data volumes increase beyond the physical limits of rendering systems/hardware (e.g., memory, computing power), interactivity becomes problematic. As such, appropriate (scalable) data representations are required that offer features such as regions of interest, random access, and level of detail. On top of that, networked environments pose additional challenges related to bandwidth and low-delay processing.

 

More specifically, our current research efforts focus on the following topics:

  • Real-time and ultra low-delay video coding and transcoding for state-of-the-art codecs such as H.264/AVC and HEVC. We have developed a generic and extensible vide coding framework to facility various real-time and low-delay video applications, incl. transcoding, multi-stream generation, watermarking, encryption, video analysis. Much of the processing is done in the compressed domain. In this context, we also work on complexity-constrained encoding.
  • A generalized coding approach for multi-dimensional visual data (hyperspectral, plenoptic, light field) based on machine learning. The current coding system uses Steered Mixture-of-Experts Regression (SMoE), and has been successfully applied for image and video data.
  • A wavelet-based scalable compression system for irregular 3D triangle meshes that allows random access to regions of interest and spatially varying levels of detail (both resolution and quality). The compression performance is state of the art in the low-to-mid range of bit rates.
  • Genomic data compression inspired by video coding techniques. Current results outperform the state of the art in compression while, at the same time, allowing data streaming and random access.

 

Staff

Peter Lambert, Glenn Van Wallendael.

Researchers

Vasileios Avramelos, Johan De Praeter, Gaétan Deglorie, Jonas El Sayeh Khalil, Tom Paridaens, Luong Pham Van, Ignace Saenen, Jelle Van Campen, Niels Van Kets, Ruben Verhack, Thijs Vermeir.

Projects

  • ICON V-FORCE: Video - 4K Composition and efficient streaming
  • ICON HD2R: Highly Delightful Dynamic Range for Next Generation Digital Cinema and Television
  • ICON SELVIE: Scalable, Efficient, and Low-delay Video Interaction during Events. This project led to the start-up initiative crowdvid.io.

Key publications

Real-time interactive regions of interest for personalized video streaming
Real-time interactive regions of interest for personalized video streaming

 

Video coding framework allowing multi-stream encoding
Video coding framework allowing multi-stream encoding

 

Scalable mesh compression with feature preservation at low rates
Scalable mesh compression with feature preservation at low rates