Diversity Meets Relevancy: Multi-Agent Knowledge Probing for Industry 4.0 Applications

Abstract Industrial data scientists require deep domain understanding to model asset conditions effectively, yet traditional sources such as Subject Matter Experts (SMEs) and Failure Modes and Effects Analysis (FMEA) documents are often unavailable or incomplete. We present a deployed Multi-Agent System (MAS) that leverages Large Language Models (LLMs) to automatically generate and refine domain-relevant questions, improving modeling decisions across industrial projects. The system addresses two key challenges—ensuring linguistic diversity and maintaining high relevance—by combining established information diversity metrics with a grounded relevancy classifier. We evaluate its effectiveness through diversity benchmarks, compare against direct prompting and AutoAgents baselines, knowledge coverage on downstream FMEA tasks, and controlled user studies. Deployed in real-world projects, the MAS has improved multiple stages of the CRISP-DM methodology, resulting in measurable savings in cost and man-hours.