Senbay Kit

Overview

Synchronizing video and sensor data is essential for context-aware applications, but existing methods require complex infrastructure. Senbay Kit is a platform that embeds collected sensor data into video frames using animated two-dimensional barcodes via real-time video processing on smartphones.

Features

• Sensor-federated video: Embeds sensor data as animated QR codes directly into video frames during recording on smartphones, enabling easy sharing through popular video sharing services
• Multi-platform: Supports iOS, Android, macOS, Unix-based OS, and web browsers
• Senbay Studio: A free macOS app that decodes and visualizes Senbay video in real time, displaying up to 15 types of sensor data simultaneously
• SenbayKit-Browser: A JavaScript library that decodes sensor data from Senbay video directly in a web browser, including live screen-capture decoding
• Data integration: Seamlessly combines video and sensor data into a single video stream for context-aware applications, integrated data recording, and sensor-data-based video editing

Results

• Demonstrated real-time embedding and extraction of sensor data in video frames on consumer smartphones
• Enabled sharing of "sensor-federated" video through existing video sharing platforms without additional infrastructure

Significance

Senbay provides a lightweight, infrastructure-free solution for integrating sensor data with video. By leveraging animated QR codes and existing video sharing platforms, it enables researchers and developers to easily capture, share, and restream synchronized multi-sensor video data using only smartphones.