A camera can record every second of a person at work. Understanding it is the hard part: what their hands did, what they meant to do, what they noticed and what they missed. This is a research role. You will design the architectures that turn real recordings into that understanding: world models that predict how a task unfolds, and mental models of the people in it.
What we care about
- Representations, not pixels. Predict in a learned space, and let go of what cannot be predicted.
- Long horizons. Real work lasts hours, not clips, so memory has to hold across a whole day.
- People in the scene. What a person sees, intends and believes is part of the state, and it is rarely in the frame.
- Evidence over intuition. Ablations, held-out tasks, and every number stated with its limits.
The work
- Design and train world models on video and motion of people at work: predict what happens next, given what people do.
- Build mental models: infer a person’s intent, attention and beliefs from what cameras, eye trackers and sensors record.
- Work on memory for long recordings: what a model should keep from hour one to act well in hour six.
- Learn across views and streams at once: fixed cameras, eye-tracking glasses, wearables, depth and motion.
- Design the evaluations: tasks, probes and benchmarks drawn from real homes and workplaces, checked against what people actually believed.
- Read widely, reproduce what matters, and write up what you find as technical reports and papers.
- Tell us what the models need next, and help shape how we record it.
You bring
- An MS or PhD in machine learning, computer vision or robotics (preferred).
- Two or more years of machine learning research or research engineering on video or other real-world sensor data.
- Strong Python and PyTorch: you have trained large models across many GPUs, and fixed them when they did not learn.
- Self-supervised learning on video: masked, contrastive or predictive objectives.
- Transformers and sequence models, and the mathematics behind them.
- Papers, a thesis or open-source work we can read, and clear technical writing.
Good to have
- World models, latent dynamics or model-based reinforcement learning.
- Belief-state estimation, theory of mind, or models of human attention.
- Planning, imitation or reinforcement learning for robots.
- Video-language models or long-video understanding.
- Egocentric or hand-object datasets, such as Ego4D or EPIC-KITCHENS.