Autonomous inspection missions in unknown environments require robots to tackle two interconnected challenges simultaneously: mapping unexplored space efficiently, and gathering detailed observations of discovered objects of interest. For example, a UAV searching a disaster scene or industrial facility must decide in real time whether to continue exploring new areas or stop to inspect a suspicious object it has just detected. Making this decision well is critical, because spending too long inspecting every uncertain detection wastes mission time, while ignoring potential anomalies defeats the purpose of the mission. more...

Contact: Please send CV and transcript to blumh@uni-bonn.de

Robots often need to reason about objects stored in drawers, cabinets, shelves, or boxes, where objects may be hidden, cluttered, or only briefly visible. In this project, we want to build a probabilistic dynamic scene graph from egocentric or robot-centric observations to represent objects, storage regions, observations, and relations such as “inside,” “on,” or “seen when opened.”

The goal is to evaluate whether graph-based memory with confidence scores can answer object-location queries, such as “Where is the bowl?”, more reliably than single-frame vision-language model predictions.

Keywords: dynamic scene graphs, probabilistic reasoning, hidden objects, egocentric video, VLMs, object-location reasoning.

Related references: Articulated 3D Scene Graphs; FunGraph; Seeing the Unseen; SEEK.

Requirements

Experience with Python and PyTorch. Knowledge of graph-based representations or basic 3D geometry will help, but is not strictly required.

Contact

Women, as well as other students who identify under the FLINTA* umbrella, are particularly encouraged to apply!

Please send your CV and transcript to gsikarog@uni-bonn.de

Recent advances in 2D/3D foundation models have enabled 4D spatial AI systems that can detect, ground, track, and build rich scene graphs from live perception. However, these systems treat every session independently and run the same models at the same intensity regardless of the scene — wasting compute in easy settings and failing in hard ones, while discarding all spatial knowledge between sessions. This thesis builds an adaptive lifelong 4D spatial AI system that solves both problems through a single unified mechanism: a memory-aware perceptual controller that dynamically adjusts what gets processed based on what is already known, while continuously compressing and storing new observations into a persistent hierarchical memory. Thus system perceives efficiently because it remembers, and remembers efficiently because it perceives adaptively — improving over time rather than resetting every session. more ...

Contact: Please send CV and transcript to rmohiudd@uni-bonn.de

Vision-Language-Action (VLA) models like π₀ and OpenVLA promise general-purpose robot control from language instructions — but they need task-specific data to be useful. This thesis takes a hands-on, end-to-end approach: you will set up an XLeRobot — a dual-arm mobile manipulator [1] — build a corresponding simulation environment, design mobile manipulation tasks that specifically require spatial memory, collect demonstration data via teleoperation in both real and simulated settings, and fine-tune state-of-the-art VLA models on your collected dataset. The result: a complete pipeline from hardware to data to policy, and a benchmark comparing how well current foundation models handle memory-dependent mobile manipulation. more ...

Contact: Please send CV and transcript to rmohiudd@uni-bonn.de

As VLMs grow more capable, robotics research has focused primarily on direct applications, e.g. VLAs and spatial reasoning. However, VLMs are now proficient at 'computer use' and coding, interacting with standard interfaces like web browsers. We propose to leverage this capability for robotics by enabling VLMs to use the tools roboticists use daily, such as RViz for navigation.

Robots have access to large amounts of data that they can collect from their deployment environments. We want to tap into this resource to optimize foundation models such as DINO to work optimally in these deployment environments, and to leverage the scale of long-term deployment to improve them for downstream applications such as object identification and tracking.

Requirements

Experience with pytorch and python

Contact

Please send your CV and transcript to blumh@uni-bonn.de

Multimodal Large Language Models (MLLM) have pushed the applicability of scene understanding in robotics to new limits. They allow to directly link natural language instructions to robotic scene understanding. Sometimes however, MLLMs trained on internet data have troubles to understand more domain-specific language queries, such as "bring me the 9er wrench" or "pick all the plants that are not beta vulgaris". This project builds up on prior work that developed a mechanism to adapt open-vocabulary methods to new words and visual appearances (OpenDAS). Currently, the method is impractical as it requires a lot of densely annotated images from the target domain. We want to develop mechanisms that allow to do such adaptation in a self-supervised way, e.g. by letting the robot look at the same object from multiple viewpoints and enforcing consistency of representation.

Requirements

Experience with python and pytorch.

Contact

Please send your CV and transcript to blumh@uni-bonn.de

Object search is the problem of letting a robot find an object of interest. For this, the robot has to explore the environment it is placed into until the object is found. To explore an environment, current robotic methods use geometrical sensing, i.e. stereo cameras, LiDAR sensors or similar, such that they can create a 3D reconstruction of the environment which also has a clear distinction of 'known & occupied', 'known & unoccupied' and 'unknown' regions of space.

The problem of the classic geometric sensing approach is that it has no knowledge of e.g. doors, drawers, or other functional and dynamic elements. These however are easy to detect from images. We therefore want to extend prior object search methods such as https://naoki.io/portfolio/vlfm with an algorithm that can also search through drawers and cabinets. The project will require you to train your own detector network to detect possible locations of an object, and then implement a robot planning algorithm that explores all the detected locations.

Requirements

experience with python, pytorch, ideally with open3d

Contact

please send your CV and transcript to blumh@uni-bonn.de