Sebastian Baltes

Software Engineering
& Empirical Research

This page provides an overview of my previous and ongoing research projects, arranged along seven main research areas. Many projects we work on focus on software analytics, that is, processing, analyzing, and visualizing software engineering data to monitor, govern, and improve software development processes and tools. We are further interested in interdisciplinary research and methodological aspects of empirical software engineering. We are convinced that thoroughly analyzing and understanding the state-of-practice is an essential first step towards improving how software is being developed. Our vision is that software engineering becomes more evidence-based, which is only possible if academic researchers provide actionable insights on topics that are relevant to practitioners. You can find more information on my personal notion of empirical software engineering in my research statement and in my inaugural lecture at the University of Bayreuth, which is available online.

Research Areas

Data-driven decision making in software engineering

Projects in this research area include empirical studies and tool prototypes built to support data-driven decision making in software projects. A central theme is test flakiness: Across a series of studies conducted at SAP HANA, we analyzed the impact of timeouts, studied how test and environmental complexity relate to flakiness, and, more recently, investigated how LLM-generated tests and test code features predict flakiness. Other work in this area includes a tailored visualization for company-wide software service dependencies used to guide service deprecation, and our global Pandemic Programming study providing data-driven recommendations to support developers during forced remote work.

Developer experience and tool support

Developer experience encompasses how the interplay of people (e.g., developers or other stakeholders) and the environment (e.g., processes, tools, culture) positively or negatively affects activities along the software development lifecycle. This broad area touches “micro” aspects such as developer tooling, including AI assistants, but also “macro” aspects such as documentation ecosystems and organizational practices. Recent work includes a critical review of developers’ trust in AI assistants and a study on context engineering for AI agents in open-source software. Related aspects are developer productivity, wellbeing, and satisfaction, all of which are difficult to accurately describe and operationalize. Other papers in this area presented novel tool prototypes, for example, to better integrate cost transparency in cloud applications, support automated query reformulation, or link documentation and source code.

Human-centric software engineering

Software is developed with and for a wide range of stakeholders, and understanding how human factors shape software development, and how development practices affect people, is central to this research area. Our work spans a broad range of topics: We critically reviewed developers’ trust in AI assistants, contributed to the Copenhagen Manifesto on human-centered generative AI in software engineering, and helped develop the Software Infrastructure Attitude Scale (SIAS) for measuring professionals’ attitudes toward technical infrastructure. We have also studied job satisfaction and turnover intentions of software professionals, examined cognitive capability and personality as predictors of coding performance, and investigated diversity and inclusion in software engineering, including ageism and gender bias in the industry.

Mining software repositories

Projects in this research area analyze data from software repositories, including version control systems, issue tracking systems, and developer Q&A forums, with the goal of identifying patterns or deriving actionable recommendations. Recent work includes studying context engineering for AI agents in open-source software and applying information-theoretic methods to detect unusual source code changes. Earlier work studied links in commit messages, developer use of GitHub Discussions, search behavior on Stack Overflow, code duplication, and attribution practices for reused code snippets. Several of those studies are based on SOTorrent, a dataset for studying the evolution of Stack Overflow posts that we maintained between 2017 and 2020 (see also the SOTorrent Project Page).

Meta-scientific issues in software engineering research

As empirical software engineering matures as a discipline, meta-scientific questions about how to conduct and communicate empirical research become increasingly important. An early focus was sampling: We studied the effectiveness and ethical implications of different strategies for sampling software developers, and later published a critical review of sampling in software engineering research. We also addressed secondary research, discussing that literature reviews in software engineering are still quite rudimentary compared to other disciplines. Our paper The Silent Scientist examines why software engineering research often fails to reach practitioners. More recently, we investigated the challenges software engineering researchers face when trying to publish interdisciplinary work, and we are developing evaluation guidelines for empirical studies involving LLMs (see also the project website).

Behavioral studies of software developers

Traces of software developers’ behavior can be found in repositories and artifacts (see Mining software repositories), but behavior can also be studied through lab and field studies, online surveys, and interviews. Recent work in this area includes studies on how cognitive capability and personality influence problem-solving in coding tasks, large-scale surveys on job satisfaction and turnover intentions of software professionals, the development of the SIAS questionnaire instrument for measuring attitudes toward software infrastructure, and an analysis of how code comments affect the perceived helpfulness of Stack Overflow posts. Earlier work studied how developers debug performance bugs in a pair programming setting, how they use sketches and diagrams in their daily work, and how they reference documentation resources.

Interdisciplinary research

Interdisciplinary research in software engineering can mean applying concepts from other disciplines such as psychology or information theory to software engineering problems. However, it can also mean working with researchers from other disciplines on problems rooted in software engineering or in another discipline. It can also mean conducting a study focusing on software engineering, the results of which are then picked up in other disciplines as well.