Sebastian Schäfer

Motivation

Recent advances in artificial intelligence have shifted the paradigm from automated systems to collaborative, human-in-the-loop pipelines in which users actively guide and iteratively refine AI outputs. A vivid example is interactive segmentation in computer vision: when processing image data such as architectural floorplans, users iteratively provide feedback — clicks, sketches, or corrections — to steer a model toward precise object delineation. This interplay raises fundamental questions about how interaction should be designed, how human input is best incorporated, and how the quality and efficiency of collaboration can be measured and improved.

Objectives

The overarching aim of this research topic is to investigate the design, evaluation, and optimization of interactive segmentation services — pipelines in which human feedback and AI inference are tightly coupled in an iterative loop. Concrete application contexts include the processing of structured visual data such as architectural floorplans, where users guide segmentation models through clicks, sketches, or corrections. Depending on your specific interests and skill set, a suitable research question and methodological approach will be developed together. Possible angles include the design of interaction mechanisms, the evaluation of human-AI task performance, or user behavior and cognitive aspects of interactive CV services.

Profile

• Interest in interdisciplinary research of human interaction with AI systems
• Technical skills: Basic understanding of CV technological foundations, programming skills (depending on approach)
• Self-driven and open learning attitude and curiosity
• English skills

Contact

We offer an exciting research topic with strong relevance to both academia and practice, close supervision, and the opportunity to develop theoretical, methodological, and practical skills. If you are interested, please send a current transcript of records, a short CV, and a brief motivation (2–3 sentences) to Sebastian Schäfer (sebastian.schaefer2@kit.edu).

Literature

• Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P.N., Inkpen, K., Teevan, J., Kikin-Gil, R. and Horvitz, E. (2019) 'Guidelines for Human-AI Interaction', in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. New York: Association for Computing Machinery, pp. 1–13.
• Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.-Y., Dollar, P. and Girshick, R. (2023) 'Segment Anything', in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 4015–4026.
• Liu, Q., Xu, Z., Bertasius, G. and Niethammer, M. (2023) 'SimpleClick: Interactive Image Segmentation with Simple Vision Transformers', in 2023 IEEE/CVF International Conference on Computer Vision (ICCV). Paris: IEEE, pp. 22233–22243.
• Jakubik, J., Hemmer, P., Vössing, M., Blumenstiel, B., Bartos, A. and Mohr, K. (2022) 'Designing a Human-in-the-Loop System for Object Detection in Floor Plans', in Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), pp. 12524–12530.