Ubiquitous Computing and Sensing
Sensing is becoming part of the spaces we live and work in, from wearables on the body to ambient sensors in homes and offices. Our goal is to make these systems understand people's activities and context in ways that are accurate, robust, and respectful of privacy. A recurring theme in our work is moving away from cameras and raw audio toward privacy-preserving modalities such as radar, lidar, thermal, IMU, and filtered audio, while still achieving rich recognition. For example, VAX uses existing video and audio models to bootstrap privacy-sensitive sensors without manual labeling, and OrganicHAR lets activity categories emerge from what the sensors themselves can reliably detect, using vision-language models only at key moments. Kirigami filters speech out of always-on audio sensing, TAO infers higher-level context from daily activity patterns, and IMUCoCo enables flexible on-body IMU placement for pose estimation and activity recognition. We are broadly interested in how sensing systems can be deployed in-situ, maintained over time, and adapted to new people and environments with minimal effort.
Projects
Lets privacy-preserving sensors discover the activities they can actually recognize, calling vision-language models only at key moments.
Pose estimation and activity recognition from IMUs placed anywhere on the body, such as a pocket, an armband, or a brooch.
A lightweight on-device filter that strips speech from audio features, defeating fine-tuned speech recognizers while preserving activity recognition accuracy.
An analytics engine that merges data from different classroom sensing systems into one schema to answer pedagogical research questions.
Attributes classroom camera observations to individual students within and across sessions, without identifiable data, enabling per-student learning analytics.
A hybrid ontology and temporal-clustering system that turns detected daily activities into higher-level contexts, reaching 87% accuracy on real-world datasets.
Uses off-the-shelf audio and video activity models to label data for privacy-preserving home sensors, then removes the camera and microphone.
A storyboard Speed Dating study of how college STEM students judge classroom sensing systems that turn their data into feedback for instructors.
Meta reinforcement learning lets newly deployed batteryless sensors borrow experience from other locations, detecting up to 66% more events in low light.
An ML system that trains, tunes, and serves personalized IoT models, re-training them as sensors and compute resources change.
A camera-based 3D digital twin of a classroom that estimates every student's and instructor's head gaze without any worn sensors.
Reinforcement learning that tunes how often battery-free, energy-harvesting building sensors sample, keeping 60 deployed nodes running 99.9% of the time.
A deployable system that uses classroom cameras to extract audio and visual features of instructor and student behavior across many classrooms.
Predicts which activity recognition models can be safely reused after sensors are replaced or deployments expand, saving up to 53% of retraining effort.
A battery-free BLE sensor node that harvests indoor light into a super-capacitor and adapts its sensing rate to run perpetually.