Spatio-temporal Enrichment and Analysis

As information changes over time and space, and contextualization (personalization and localization) becomes more and more important, we need spatio-temporal extraction, enrichment, querying and mapping tools to support end users in optimizing their (meta)data for future exploration. However, the current metadata scope of all kind of multimedia collections is too narrow and too high-level to allow easy and adequate exploration in time and space, i.e., they only marginally support spatio-temporal analysis; only limited amount of data is currently queryable in time and space; and visualization/modeling tools focusing on these features are still rare.

IDLab focuses on fundamental and applied research to improve the spatio-temporal (meta)data quality and querying process, and to optimize mapping, finding and analysing data related to a specific place and time. Furthermore, we develop methodologies to optimize the metadata management/filtering and provide tools for geo-temporal similarity detection. The latter technologies will allow cross-media analysis in time and space, facilitating scientific interpretation of data in a broader sense.

The majority of our current building blocks for extraction, enrichment, filtering and mapping of geo-temporal metadata are based on textual analyses/natural language processing (e.g., named entity recognition (NER) of geo-locations / timestamps) and computer vision techniques (e.g. deep learning based image clustering/scene recognition with convolutional neural networks (CNN). However, other research domains participating in the overall spatio-temporal data enrichment process (such as multimodal sensing) are being investigated  and will be further explored in future research/projects.

A wide range of applications (linked to current projects/proposals) can benefit from our spatio-temporal enrichment and analysis tools. Currently, we focus on the following domains: Sports and Health (sport analytics in collaboration with VICTORIS partners - www.victoris.be), Media and Culture (collection valorization), Safety and Security (video fire analysis and traffic control), and Industry applications (condition monitoring).

Subtopics: spatio-temporal annotation, metadata enrichment/quality analysis, data mining, multi-modal sensing, time/place mapping. 

Staff

Sofie Van Hoecke, Steven Verstockt, Erik Mannens.

Researchers

Kevin Baker, Florian Vandecasteele, Olivier Janssens, Baptist Vandersmissen, Gerald Haesendonck, Dieter De Witte, Ruben Taelman, Pieter Colpaert, Miel Vander Sande.

Projects

  • IWT O&O SPOTSHOP: On-the-spot e-commerce for television

Key publications

Spatio-temporal object tagging for second-screen shopping of TV content (IWT O&O Spotshop)
Spatio-temporal object tagging for second-screen shopping of TV content (IWT O&O Spotshop)

 

Spatio-temporal activity analysis: weather and location profiling of MTB loggings.
Spatio-temporal activity analysis: weather and location profiling of MTB loggings.

 

Time-Location-Context (TLC) linking of users/devices (wearables) with multimedia content.
Time-Location-Context (TLC) linking of users/devices (wearables) with multimedia content.