EyeFi: Fast Human Identification Through Vision and WiFi-based Trajectory Matching

We develop a novel system called EyeFi that fuses WiFi and camera data to provide long-term identification of people without requiring facial recognition and installation of multiple WiFi units. Using a student-teacher model, a neural network is trained to estimate motion trajectory of people by leveraging fine-grained WiFi signals (CSI) generated from their smartphones. By performing cross-modal trajectory matching, individuals are identified using the WiFi MAC addresses of their smartphones. EyeFi improves WiFi CSI based AoA estimation accuracy by more than 30% and offers 3,800 times computational speed over the state-of-the-art solution. [Paper] [Video] [Dataset]
Occupancy Patten Modeling and Prediction for Effective HVAC Control

With long-term occupant-count data collected from a commercial building, we develop models to capture occupancy dynamics and predict future occupancy patterns. We design an adaptive model predictive controller to minimize inefficient control actions according to different types of misprediction and occupancy states. Our control design provides probabilistic guarantees on the control cost under occupancy prediction uncertainty. Our evaluation shows that the proposed solution reduces energy consumption by 29.5% while improving the average weighted occupant comfort by 86.7% in Predicted Mean Vote (PMV) over the fixed schedule strategy.
Inferring Thermal Comfort Using Body Shape Information

Can we use body shape to determine thermal comfort of individuals? In order to test this hypothesis, we performed a year-long experiment, where we recruited 77 participants and put each of them in a thermally controlled conference room for three hours and recorded their subjective responses regarding thermal comfort at different temperature ranging from 60°F to 80°F. Using the collected data, we build a thermal comfort model and find that our model can infer thermal comfort of individuals with 60 % accuracy when body shape information is taken into account. [Paper] [Dataset]
Long-Term Occupancy Estimation in a Commercial Space: An Empirical Study

Understanding occupancy patterns in a building is very useful to improve energy efficiency of the building and occupant comfort. In this work, we leverage FORK (see below) to collect occupancy count from an 11,000 square foot commercial space for nine months. We analyze the collected data and describe key findings that provide deep insights about how a commercial space is used and serve as a guideline to formulate novel and efficient control strategies. [Paper] [Dataset]
ODDS: Real-Time Object Detection using Depth Sensors on Embedded GPUs

In this work, we develop a prototype system called ODDS (Object Detector using a Depth Sensor) that detects various types of objects including backpacks, boxes, laptops, phones, umbrellas, cups, nerf guns, and humans in smart buildings using only depth sensors (no RGB cameras) in real-time on embedded GPUs (Jetson TX1, TK1). It shows that despite of low resolution of a depth sensor and noisy depth frames, objects can be detected accurately (80% mAP) though they are a few feet away from the depth sensor. [Paper] [Video]
FORK: Fine-grained Occupancy Estimation using Depth Sensors on ARM Platforms

In this work, we develop a prototype system called FORK (Fine grained Occupancy estimatoR using Kinect) to explore the potential of using depth sensors to detect, estimate, identify, and track occupants in buildings on a cheaper and low power ARM processor in real-time instead of using a powerful computer like XBOX or an Intel Core i7 processor. Unlike other camera based approaches, FORK is much less privacy invasive (even if the sensor is compromised). [Paper] [Video]
OCIR: Occupancy Counting using IR array sensors

In this work, we develop a system called OCIR to estimate occupancy using a very low resolution (8x8 pixels) IR-array sensor. The sensor is very tiny and it is very easy to mount at the doorways. We also show that it can be used to sense skin temperature to improve thermal comfort of the occupants. [Paper] [Video]

Previous Projects

FailureSense: Detecting Sensor Failure using Electrical Appliances in Homes

Our proposed solution, FailureSense, uses a novel idea of using electrical appliances to detect various types of sensor failure in a home, e.g., fail-stop, obstructed-view, and moved-location failures. The intuition is whenever someone turns on electrical appliances in a home, there will be a positive correlation between the appliance activation event and the firing of the motion sensor that is covering that area. Since the movements of the residents are constrained by the floorplan of the house, with sufficient training data, we learn all the regular intervals between sensor firing and appliance activation and report a failure when we see a significant deviation from the regular behavior. [Paper]
DepSys: Dependency-aware Integration of Systems in Homes

We designed a sensing and actuation infrastructure for smart homes named DepSys that treats each system as an app and enables running many apps from different domains including energy, health, security, and entertainment. App developers specify dependency information as meta-data within their apps and their apps are put in an app store. Users can choose and install apps from the app store that will control home appliances. DepSys uses the meta-data to detect and resolve conflicts at app installation time and at run-time. We have developed a working prototype of DepSys upon HomeOS platform. [Paper]
EyePhy: Detecting Dependencies in CPS Apps due to Human-in-the-loop

As app based paradigms are becoming popular, millions of apps are developed from many domains including energy, health, security, and entertainment. Many of these apps are CPS apps. In addition to sensing, communication, and computation, they perform interventions to control human physiological parameters, which can cause dependency problems as multiple interventions of multiple apps can increase or decrease each others effects, some of which can be harmful to the user. We present EyePhy, a system that detects dependencies across interventions by having a closer eye on the physiological parameters of the human in the loop and offers personalized dependency analysis for the user. [Paper]
Predicting Passenger Demand for Taxicabs

In this work, we use a 450GB dataset containing 6 months of GPS records of 14000 taxicabs to predict passenger demand for taxicabs, which poses a big data challenge. To address this challenge, we define a novel parameter called pickup pattern to quantify taxicab operational similarity that not only reduces the amount of data to be analyzed, but achieves better inference accuracy as it allows us to consider only data of similar days instead of naively considering all the historical data. [Paper]
Reducing Energy Waste for Computers by Human-in-the-Loop Control

Our user study with 20 human subjects suggests distractions as a major source of energy waste on computers. However, current common practice for detecting distraction is very coarse-grained. It is usually based on a fixed timer that initiates the sleep mode of the computer after several minutes of inactivity. In this project, we collect empirical data to understand human behavior regarding distractions in computer usage and develop a human-in-the-loop control that can put workstations to sleep by early detection of distraction using several novel strategies, including adaptive timeout interval, where we learn and adapt the timeout interval of the individuals, and multilevel sensing, where we turn on the webcam and track the gaze of the users. [Paper]
Reliable Communication and Latency Bound Generation in Wireless Networks

Many real-time applications require reliable transmission within latency bounds. Achieving this goal is very difficult because of link burstiness and interference. Based on significant empirical evidence of 21 days and over 3,600,000 packets transmission per link, we propose a scheduling algorithm that produces latency bounds of the real-time periodic streams and accounts for both link bursts and interference. The solution is achieved through the definition of a new metric Bmax that characterizes links by their maximum burst length, and by choosing a novel least-burst-route that minimizes the sum of worst case burst lengths over all links in the route. [Paper]