Master Thesis on Agentic LLM for Tool Evaluation

Safe Intelligence — this forms the core brand of the Fraunhofer Institute for Cognitive Systems IKS . Connected cognitive systems drive innovation in many sectors, including mobility, healthcare, and automation in industry. Disruptive technologies such as artificial intelligence and quantum computing play a key role here. Fraunhofer IKS is conducting research to ensure that these applications are reliable and verifiably safe. We consider resilience and intelligence to be part of the same process.

What you will do

Advancements in agent-based AI approaches are gaining significant attention in industry due to their wide-ranging applications. This master thesis aims to explore innovative methods for evaluating and classifying engineering tools by leveraging these advanced methodologies. By focusing on tool assessment through agentic AI, we strive to minimize the risk of systematic faults in safety-critical systems.

Responsabilities:

• Conduction of a literature review focusing on methodologies related to tool evaluation
• Design and implementation of a framework using agent-based AI approaches
• Performation and evaluation of experiments
• Collaboration with safety experts to gather insights and validate your framework
• Building a web-based platform to present the results
• Creation of documentation for the experimental process and the developed platform.

What you bring to the table

• Master's student at a university in Computer Science, Electrical Engineering, or a related field
• Strong programming skills, particularly in Python
• Experience with AI technologies, including LLMs, RAGs, and Agents (e.g., LangChain)
• Experience with training and fine-tuning AI models (e.g., using SLURM)
• Competency in developing graphical user interfaces (GUI)
• Strong analytical thinking and attention to detail
• Ability to conduct independent research and translate findings into practical implementations.

Bonus Skills (Plus but Not Mandatory):

• Participation on AI projects showcased on GitHub
• Familiarity with web development frameworks like React, Django, or Flask
• Knowledge of ISO 26262 and familiarity with safety-critical systems.

What you can expect

• Modern research environment with close connections to industry
• Hybrid working environment, combining remote and on-site collaboration
• Collaborative and innovative team atmosphere.

We value and promote the diversity of our employees' skills and therefore welcome all applications - regardless of age, gender, nationality, ethnic and social origin, religion, ideology, disability, sexual orientation and identity. Severely disabled persons are given preference in the event of equal suitability. Remuneration according to the general works agreement for employing assistant staff.

With its focus on developing key technologies that are vital for the future and enabling the commercial utilization of this work by business and industry, Fraunhofer plays a central role in the innovation process. As a pioneer and catalyst for groundbreaking developments and scientific excellence, Fraunhofer helps shape society now and in the future.

Interested? Apply online now. We look forward to getting to know you!

Additional questions will be answered gladly by:

Dr. Núria Mata: [email protected]
Dr. Reiner Birkl: [email protected]

If you have further questions, please contact our colleagues from HR: [email protected]

Fraunhofer Institute for Cognitive Systems IKS

Requisition Number: 79421Application Deadline: