Master Thesis: Evaluation of an automated feedback loop for generating structured text code

The Fraunhofer institute for Cognitive Systems IKS is specialized on applied research in the innovative areas of artificial intelligence, cognitive systems and intelligent software architectures for autonomous systems. Our focus lies on safety-critical applications in the fields of automation, mobility and health. We develop reliable software technologies with a benefit for humans. For example, we conduct an extensive safety analysis and calidation of perception algorithms in automotive. Through our work, we lay the foundation for a reliable digital future.

What you will do

Generative AI opens up new possibilities for the automatic creation of structured text (ST) for PLCs. Although LLMs can already generate functional ST code from natural language requirements, the quality of the result is often insufficient. An automated feedback loop that analyzes the code, converts errors into appropriate correction feedback, and feeds it back to the LLM promises significant improvements in this area. The aim of this master's thesis is to systematically evaluate the functionality of such an automated feedback loop for the first time.

Tasks:

• Systematic literature analysis on feedback mechanisms and quality metrics for LLM-based code generation.
• Identification of relevant metrics and tests (static/dynamic) that best reflect the code quality and functional safety of ST.
• Determination of the optimal format for transferring the metrics and test results to the LLM.
• Development of a prototype software module that automatically generates feedback in the defined format from the selected metrics and tests.
• Quantitative and qualitative evaluation of the feedback process by measuring the code improvements achieved through the feedback and comparing them with a baseline.

What you bring to the table

• Currently pursuing a degree in engineering, with a focus on informatics or AI
• Good knowledge in programming with python and on software development
• Knowledge on LLMs and langchain is a plus
• Analytical and structured mindset
• Good knowledge of the English and German language (B2).

What you can expect

• Participation in a dynamic team with innovative task areas and nice supervisors.
• Practical approach to your studies
• Excelent support for your thesis
• Collaboration on practical research projects.

We value and promote the diversity of our employees' skills and therefore welcome all applications - regardless of age, gender, nationality, ethnic and social origin, religion, ideology, disability, sexual orientation and identity. Severely disabled persons are given preference in the event of equal suitability.

With its focus on developing key technologies that are vital for the future and enabling the commercial utilization of this work by business and industry, Fraunhofer plays a central role in the innovation process. As a pioneer and catalyst for groundbreaking developments and scientific excellence, Fraunhofer helps shape society now and in the future.

Interested? Apply online now with your motivation letter, cv and a current transcript of records . We look forward to getting to know you!

Additional questions will be answered gladly by
René Beck per eMail: [email protected]
If you got further questions, please contact our colleagues from HR via eMail: [email protected]

Fraunhofer Institute for Cognitive Systems IKS

Requisition Number: 80650Application Deadline: