PhD Position: Toward Trustworthy and Adaptive Agentic AI Systems

The project aims to develop a deeper understanding of the underlying mechanisms of agentic AI systems and to leverage these insights to design novel approaches that enhance their trustworthiness and interpretability.

Examples of topics that could be explored in this PhD project:

Planning and Reasoning Ability

An agentic AI system must transform high-level instructions into a sequence of executable actions. This process inherently involves complex reasoning and planning challenges. As tasks become more realistic and open-ended, these challenges grow significantly in scale and complexity. Existing approaches often rely on predefined topological structures, such as trees or graphs, to model interactions among agents. However, such static structures are insufficient to capture the dynamic and evolving nature of real-world multi-agent systems over time. Furthermore, upstream tasks must provide sufficient information to downstream components to enable effective

coordination, while simultaneously preventing unintended information leakage, including privacy-sensitive data. A comprehensive understanding of the underlying interaction mechanisms in agentic systems remains largely unexplored.

Safety

Agentic AI systems are increasingly deployed to act on our behalf in real-world environments. This implies that they may gain access to sensitive personal data and interact directly with physical systems. In such settings, an error is no longer merely an incorrect response—it may lead to tangible consequences, such as damaging property or even causing harm to humans. Compared to foundation models that primarily generate text or predictions, agentic AI systems operate through embodied actions and collaboration with external tools, environments, and other agents. This expanded scope of interaction significantly enlarges the attack surface and introduces new safety vulnerabilities. A common mitigation strategy today is to introduce multiple filtering layers and perform repeated safety checks before producing outputs. However, these approaches often incur substantial computational overhead and primarily focus on response-level filtering, rather than enforcing action-level constraints or ensuring safety throughout the decision-making process.

Continual Learning.

Continual learning in agentic AI requires an explicit and well-designed memory management mechanism that enables agents to progressively accumulate knowledge and skills over time. As agentic systems are deployed to handle an increasing number and diversity of tasks, several fundamental challenges arise: How can agents effectively identify and retain critical knowledge while filtering out redundant or low-value information? How can they accurately detect outdated knowledge and update it without disrupting previously acquired competencies? Addressing these questions is essential for building adaptive, scalable, and long-lived agentic AI systems capable of sustained autonomous operation.

The goal is to advance the fundamental understanding of agentic AI systems by generating high-impact research contributions suitable for publication in leading machine learning and AI venues.

Employed PhD candidates are expected to spend 10% of their working hours on teaching and/or supervising candidates.