Robotics and AI Institute (RAI), Cambridge, mA - Reinforcement learning intern
Supervisors: Thomas Weng and Erica Lin
Data curation for fully offline model-based RL. Uncertainty weighting and out-of-distribution guidance for online model-based RL. Paper submission in progress for CoRL 2026
Tufts University, human-robot interaction - Computer Science. PhD Candidate
Human-robot interaction
Supervising Faculty: Elaine Schaertl Short
Agent-Centric Human Demonstrations Train World Models
Human demonstrations are extremely effective at bootstrapping world-model based reinforcement learning systems (MBRL). We demonstrate this effect in two simulated environments and compare imitation learning methods to DreamerV3 without and without demonstrations. Published at the Reinforcement Learning Conference. Ongoing follow-up work investigates how MBRL and imitation learning algorithms learn from human versus AI demonstrations. 2024 Paper.
Contingency Detection
Contingency detection between a human participant and multiple agents in a simulated environment. An online human-subjects study platform was implemented for remote studies. In this environment multiple agents interact with toys and a participant. Temporal pattern analysis was conducted on extracted features to determine which agent held the participants attention. Short Paper. Study Environment.
Ongoing work uses mutual information as a proxy for contingency between agent and human behavior distributions. Full paper pending.
Socially Assistive Robotics
Robot arm trajectory modified by simple human feedback for socially assistive robotics. Preset trajectory represented by Dynamic Motion Primitives (DMPs). Trajectory was modified by adjusting DMP basis functions in line with binary human feedback.
Reinforcement Learning
Supervising Faculty: Jivko Sinapov
Hierarchical Reinforcement Learning multi-step tasks. Sub-policies and a Top-level DQN were trained to collect materials and make a goal object in a multi-step Minecraft task. Ongoing work uses soft-actor critic sub-policies to adapt to novel environmental changes in either the state, transitions or rewards.
Human-criticized reinforcement learning for modifying a simulated arm trajectory. Goal achievement with a TAMER-like algorithm to learn stylistic flare without sacrificing goal achievement. Tiled state representation. Python.
Anomaly Detection
Supervising Faculty: Matthias Scheutz
Improving task performance of a multi-agent team with anomaly detection. Task performance of a two-agent was statistically analyzed to detect non-directly observable equipment failures. Task success rates could be improved through role reallocation if systematic anomalies were detected. Full Paper.Multi-Agent Anomaly and Novelty Detection in Science Data for Swarm Spacecraft
NASA Goddard space flight center - research intern
Supervisor: Evana Gizzi
Multi-Agent Anomaly and Novelty Detection in Science Data for Swarm Spacecraft
Detect anomalous sensors or novel environmental features using constellations of satellites operating in distant, unexplored contexts assuming the absence of in-domain training sets. Work done on NASA’s Research in Artificial Intelligence for Spacecraft Resilience (RAISR) group. Poster accepted at SmallSat 2023.
National Robotics Engineering Center, Carnegie Mellon, Pittsburgh, Pa. Summer ‘13
Orange Grove Laser Analysis
Analyzed 3D laser scans of 150 acre orange groves to extract features about the tree canopies. Performed regression analysis on extracted features and analyzed correlation to orange yield. Visualizing results. Matlab. Paper.
Deep Learning Neural Net Configuration
Implemented artificial neural network package to test deep learning performance on obstacle recognition in farm environment. The ANN was fed labeled data of images with and without farm obstacles. Different configurations of layers and nodes were tested. Matlab.
Colby College, Waterville, Me. Summer ’10
Humanoid Robot Vision and Interaction
Developed interactive games for a small humanoid robot to play with children. The work was done as part of an NSF grant aimed at increasing young children’s exposure to STEM. Simon Says and Red Light Green Light were implemented using motion and face detection software. Work was presented to NSF advisory/funding board. C.