Scaling and Adapting Automated Testing for Broader Educational Impact

Table of Links

Abstract and Introduction

Related Work

Project Prerequisites

Project Implementation

Insight and Reflections

Conclusions, Acknowledgement and References

5 INSIGHTS AND REFLECTIONS

In this section, we reflect upon what we learned, what worked well for us, what challenges we faced, and how a project such as this could be adapted and scaled in the future. We share our outlook that exposure to version control and automated software testing early in a student’s computer science education can provide a strengthened ability to work through errors, find and resolve bugs, and write cleaner code.

5.1 Adaption, Future Work, and Scalability

This project and the automated tests we developed applied specifically to the scientific computing applications developed in connection with our lab. We wanted to give our volunteer students an opportunity to gain practical experience and meaningfully contribute to these scientific computing applications while also being introduced to new concepts and learning new skills. The project websites provided a good place to start because the students could immediately begin learning the relevant software engineering concepts without requiring prerequisite domain science knowledge in physics (JETSCAPE [1]) or chemical engineering (GOMC [2]).

We see value in generalizing this content and scaling it to benefit other students for other applications. The repository management and automated testing concepts can apply broadly to any informational websites from local small businesses to a student’s own promotional website showcasing work to prospective employers.

A challenge we faced with respect to developing this project is that these scientific computing applications and other research projects active in our lab would require prerequisite domain science knowledge, knowledge of algorithms and data structures, or knowledge of parallel programming and high-performance computing. A project to develop these automated tests was something we could achieve without overwhelming background requirements. The XML reader was chosen to unit test because it was a part of the application that could be understood without domain science knowledge. However, implementing unit tests to validate example input files is not, in our view, the most pedagogical use case for unit testing. For a future endeavor, it may be more appropriate to introduce unit testing to validate the correctness of a function itself rather than the validity of an input file.

We faced an additional challenge in that, in our view, we were unable to provide an overview of the project at the outset. Much of this project’s development was experimental and we weren’t sure how long it would take to complete each task. We also didn’t have upfront knowledge about which of the available utilities in the GitHub Actions Marketplace would work for us; therefore, the project involved researching the available tools and encountering trial and error as we progressed. While this provided exposure to the experimental environment of a laboratory, the students might have felt better prepared if a more complete overview of the project’s scope, schedule, and tools were given at the outset. An introductory handout of commands and concepts could have provided the students with a more actionable roadmap, study guide, and measure of their progress throughout the project.

5.2 Advantages of Early Learning

Software engineering concepts such as version control, unit testing, and continuous integration often aren’t introduced until several semesters into an undergraduate student’s studies and after multiple programming courses. Having worked with many students at the undergraduate level, it is our observation that students often resort to saving multiple versions of their code under different file names, creating a disorganized working environment that could be improved with a brief introduction to version control. Practicing version control early in one’s programming studies could lead students to developing fluency in this important skill well before graduation or internships [9]. A knowledge of software testing – especially unit testing – could lead beginning programmers to think more cleanly about program design, and a requirement to write unit tests could lead beginning students to write more succinct and modular functions as opposed to the lengthy and sprawling functions often observed in beginning students’ code.