Master's Thesis / Graduation Project in efficient on-device Machine Learning for AR applications at Snap Inc. You will explore how to combine modern deep learning with event-based and embedded processors to push the limits of what AR glasses can do on-device.
## What you'll do
Project focuses on addressing how conventional frame-based pipelines and large neural networks are too slow and power-hungry for always-on, real-time AR. By exploiting temporal and spatial sparsity through event-based sensing and processing, the goal is to:
- Turn always-on perception into something that fits within strict power budgets
- Push more intelligence closer to the sensor, reducing latency and data movement
- Co-design models and systems built for edge hardware
As a thesis student, you will:
- Design and prototype ML models tailored to AR use cases under embedded constraints (e.g., event-based vision models, lightweight CNNs/Vision Transformers, or hybrid frame+event pipelines)
- Set up datasets and baselines relevant to AR tasks and define evaluation metrics across accuracy, latency, memory usage, and energy
- Implement and train models in PyTorch, including data pipelines, training loops, and evaluation scripts
- Explore efficiency techniques such as sparsity, pruning, quantization (PTQ/QAT), or event-based representations
- Profile models under embedded-like conditions using simulators, profiling tools, or edge accelerators
- Communicate findings through ablation studies, thesis report, and reproducible codebase
## Expected Outcomes
- Demonstrate proof-of-concepts on AR hardware (e.g., Spectacles)
- Deliver measurable improvements in runtime performance, efficiency, and adaptability
- Provide insights into model-system co-design for low-power, on-device ML
- Contribute to ML frameworks, tooling, or deployment strategies for embedded AR systems
- Produce high-quality thesis report with reproducible code and results