Toward Controllable and Trustworthy LLM Reasoning: From Failure Mapping to Cognition-inspired Control and Real-world Impact

Abstract Large Language Models (LLMs) have advanced rapidly and raised the bar for what AI is expected to do. However, accompanied with such progress is a stronger consensus that these models consistently fail in out-of-distribution reasoning, especially on tasks that require abstraction, transfer, or long-horizon planning. While acceptable for most consumer use, these issues prevent AI from being safely deployed in high-stakes settings (e.g., healthcare), where stakeholders cannot trust AI models that exhibit uncontrollable and unpredictable failures. In this talk, I will discuss our work and insights on how to make LLM reasoning controllable and trustworthy, by 1) understanding the mechanisms of LLM reasoning and predicting when LLM will fail; 2) improving model reasoning and generalization based on such insights; and 3) moving towards trustworthy AI applications through such improvements, and identifying new problems to form a healthy positive-feedback loop.