Advancing Software Testing: Mutation Testing in Actor Concurrency and Empirical Insights into Machine Learning Test Practices

Abstract

Software testing, especially in large-scale projects, is becoming increasingly important in real-world and mission-critical software. This requires the tests to be highly effective at finding bugs that occur in the real world. However, developers find designing such effective tests challenging. My research aims to tackle this challenge by advancing software testing in two key domains: actor concurrency and machine learning that has been widely used across web, mobile, and desktop applications and adopted at scale by companies like Google and Facebook. This dissertation presents µAkka, a framework for mutation testing of Akka actor concurrency using real actor bugs. The research analyzes 186 real Akka bugs, designs 32 mutation operators, and implements them in an Eclipse plugin. µAkka generates 11.7k mutants of 10 GitHub applications and runs 7.9k tests. The evaluation compares results to PIT with 26.2k mutants. µAkka mutants are higher quality, cover more bugs, and tests are less effective in detecting them. Additionally, this dissertation includes a large-scale study on a set of real-world ML test cases in GitHub to understand their topics, popularities, difficulties, correlations, and historical changes. The study curates a set of 2,525 ML projects and their 136,463 test cases from GitHub; uses topic modeling to group these test cases into ML test topics; groups similar topics into an ML test topic hierarchy; discusses these ML topics using sample test cases; analyze the popularity and difficulty of these topics; and study the correlation and historical trends over the past 10 years. Together, these efforts address two critical domains: actor concurrency and machine learning. In actor concurrency, this work introduce a mutation testing framework designed to measure and strengthen test suites. In machine learning, we provide empirical insights into testing practices and gaps through a large-scale analysis. Both contributions advance understanding of how software is tested in areas of growing importance.