Type systems of programming languages are a much discussed topic of software engineering. There are many voices arguing towards static as well as dynamic type systems, although their actual impact on software development is rarely evaluated using rigorous scientific methods. In the context of this work, a controlled experiment with 36 participants was conducted which tried to compare the performance of software developers using a static and a dynamic type system for the same tasks using an undocumented API. The two programming languages used were Java and Groovy. The experiment and its results are analyzed and discussed in this thesis. Its main hypothesis was that a static type system speeds up the time developers need to solve programming tasks in an undocumented API. The main results of the experiment speak strongly in favor of this hypothesis, because the static type system seems to have a significantly positive impact on the development time.
Table of Contents
1. Introduction
2. Motivation & Background
2.1 Motivation
2.2 Maintenance and Debugging
2.2.1 Maintenance in a Nutshell
2.2.2 Debugging in a Nutshell
2.3 Documentation and APIs
2.3.1 Documentation of Software Systems
2.3.2 APIs and Application of their Design Principles in General Programming
2.4 Type Systems
2.5 Empirical Research in Software Engineering
2.5.1 On Empirical Research
2.5.2 Controlled Experiments
2.5.3 Current State of Empirical Research in Software Engineering
3. Related Work
3.1 Gannon (1977)
3.2 Prechelt and Tichy (1998)
3.3 Daly, Sazawal and Foster (2009)
3.4 Hanenberg (2010)
3.5 Steinberg, Mayer, Stuchlik and Hanenberg - A running Experiment series
3.5.1 Steinberg (2011)
3.5.2 Mayer (2011)
3.5.3 Stuchlik and Hanenberg (2011)
4. The Experiment
4.1 The Research Question
4.2 Experiment Overview
4.2.1 Initial Considerations
4.2.2 Further Considerations: Studies on Using Students as Subjects
4.2.3 Design of the Experiment
4.3 Questionnaire
4.4 Hard- and Software Environment
4.4.1 Environment
4.4.2 Programming Languages
4.5 Workspace Applications and Tasks
4.5.1 The Java Application - A Labyrinth Game
4.5.2 The Groovy Application - A simple Mail Viewer
4.5.3 Important Changes made to both Parts
4.5.4 The Tasks
4.6 Experiment Implementation
5. Threats to Validity
5.1 Internal Validity
5.2 External Validity
6. Analysis and Results
6.1 General Descriptive Statistics
6.2 Statistical Tests and Analysis
6.2.1 Within-Subject Analysis on the complete data
6.2.2 Analysis for residual effects between the two Participant Groups
6.2.3 Within-Subject Analysis on the two Participant Groups
6.2.4 Exploratory Analysis of the Results based on Participants’ Performance
6.2.5 Hypotheses and Task based Analysis
7. Summary and Discussion
7.1 Final Remarks
7.2 Result Summary
7.3 Discussion
8. Conclusion
Research Objectives and Core Themes
The primary objective of this thesis is to empirically investigate whether static type systems enhance developer performance during software maintenance tasks, specifically when working with undocumented APIs. The study aims to clarify the ongoing debate regarding the practical advantages of static versus dynamic type systems by conducting a controlled experiment, thereby narrowing the gap between theoretical speculation and empirical evidence in software engineering.
- Impact of static versus dynamic type systems on developer performance.
- Role of API documentation and variable naming in maintenance tasks.
- Influence of task complexity (number of types) on development time.
- Distinction between semantic errors and latent type errors in programming.
- Methodological rigor in empirical software engineering experiments.
Excerpt from the Book
4.5.4.7 Tasks 7 and 16 – Stack size 2 and branch size 3
Tasks 7 and 16 were tasks which contained latent type errors. As explained above, this means that the point at which a wrong type is passed as a parameter is at a different part of the program stack. In this case, the participants were given a simulated interaction with the application that created the wrong application state, along with some consistency check logic which was supposed to call the code which would use the wrong state and result in a type error during runtime for Groovy, or a compile error for Java.
Because the Groovy part is the most interesting for these type of tasks, all explanations will be done using the Groovy application. The code in Listing 4-18 shows the simulated interaction given to the participants in the task. The code essentially just calls the TrapFactory class to create a new trap, and then passes this instance to a new TrappedLevelField. This field is then queried in a consistency check which gets all messages of the trap on the field and checks whether they contain a specific word.
Summary of Chapters
1. Introduction: Presents the motivation for the study, highlighting the controversial nature of static versus dynamic type systems and the scarcity of empirical research on their impact on software maintenance.
2. Motivation & Background: Discusses core concepts including software maintenance, debugging, documentation, APIs, and the theoretical underpinnings of different type systems, alongside an overview of empirical research methods.
3. Related Work: Reviews existing literature and previous empirical studies regarding type systems, emphasizing the need for rigorous experimental designs in software engineering.
4. The Experiment: Details the design, research questions, hypotheses, software environment, and specific task categories utilized in the controlled experiment.
5. Threats to Validity: Analyzes potential limitations of the experiment, such as learning effects, participant variance, and the use of students as subjects, to ensure critical interpretation of the results.
6. Analysis and Results: Provides a comprehensive statistical evaluation of the gathered data, including descriptive statistics, within-subject analyses, and hypothesis testing.
7. Summary and Discussion: Synthesizes the main findings, discusses the validity of the experimental approach, and reflects on the implications of the results for software engineering practices.
8. Conclusion: Summarizes the study’s contributions, concluding that the results support the hypothesis that static type systems can positively impact developer performance in specific maintenance scenarios.
Keywords
Static type systems, dynamic type systems, software maintenance, developer performance, controlled experiment, empirical software engineering, undocumented API, debugging, latent type errors, semantic errors, programming languages, Java, Groovy, task complexity, experimental design.
Frequently Asked Questions
What is the core focus of this empirical study?
The study investigates the impact of static versus dynamic type systems on software developer performance, specifically during maintenance tasks involving undocumented APIs.
Which programming languages were compared in this research?
The experiment used Java to represent a statically typed language and Groovy to represent a dynamically typed language.
What is the central hypothesis of the experiment?
The primary hypothesis is that a static type system reduces development time when developers perform tasks using an undocumented API.
What research methodology was employed?
A controlled experiment with a repeated-measures design was conducted, involving 36 participants who performed various programming tasks.
What specific types of errors were the participants tasked with fixing?
Participants dealt with three task categories: type identification tasks, semantic errors, and latent type errors.
How is developer performance measured in this study?
Performance is primarily measured as the time required for a developer to successfully complete a given programming task.
How did the study address potential learning effects?
The study used a repeated-measures design and two participant groups (Java-first and Groovy-first) to detect and account for learning and carry-over effects.
What were the main conclusions regarding the hypotheses?
The main hypothesis regarding the benefit of static type systems was supported, while the additional hypotheses concerning error distance and semantic error fixing times were rejected due to inconclusive or contradictory data likely caused by learning effects.
Why are the participants primarily students?
The use of students is a common practice in software engineering research to manage resources and maintain experimental control, though the study discusses the implications of this choice on external validity.
- Arbeit zitieren
- Master of Science Sebastian Kleinschmager (Autor:in), 2011, Can static type systems speed up programming? An experimental evaluation of static and dynamic type systems, München, GRIN Verlag, https://www.hausarbeiten.de/document/199362
