Software Engineering for Computational Science and Engineering on Supercomputers
Authors: David Bernholdt (Oak Ridge National Laboratory)
Abstract: Software engineering (SWE) for computational science and engineering (CSE) is challenging, with more sophisticated, higher fidelity simulation of larger and more complex problems involving larger data volumes, more domains, and more researchers. Targeting high-end computers multiplies these challenges. We invest a great deal in creating these codes, but we rarely talk about that experience. Instead we focus on the results.
Our goal is to raise awareness of SWE for CSE on supercomputers as a major challenge and to begin the development of an international “community of practice” to continue these important discussions outside of annual workshops and other “traditional” venues.
Long Description: The engineering of software for computational science and engineering is an area that gets little attention in our community. We celebrate the big machines, the scientific discoveries they enable when driven by sophisticated software, and the cleverness and creativity of the software itself. The opportunities are fewer to talk about how that software was designed, the successes and failures of the development processes used, testing and verification strategies that maximize confidence in the code while minimizing the use of expensive resources, and many other aspects of the entire lifecycle of a CSE application, including portability, sustainability, and overall productivity. Additionally, the majority of the developers of such software have little or no formal training in software engineering or computer science.
At the same time, the landscape is changing for the computer hardware architecture. With the end of Dennard scaling, the pace of change and level of diversity in the architectural arena have increased dramatically, and the drive to exascale will exacerbate the situation. CSE software developers who are already facing scientific demands for increasing size, fidelity, and sophistication in their modeling and simulation capabilities with the associated increases in data volume and complexity, developed by larger, more multidisciplinary and geographically dispersed teams, now also have to contend with these architectural changes. The increasing use of cloud and “big data” infrastructures (hardware and software) for CSE raises additional SWE challenges.
We believe that the situation has the makings of a serious Software Crisis in CSE using HPC, which we ignore at our own expense in scientific productivity and opportunity. The good news is that a growing number of organizations are paying more attention to this problem. But the work is not widely shared, and uptake of good practices takes place in a fragmented fashion. We believe that the next step in the process is a concerted effort to increase awareness and sharing of work on SWE for HPC CSE across the community, with the aim to foster good practice that will see software being fit to power CSE through the next era of computing.
The goal of this BOF is to bring together people people concerned about this topic to raise awareness of existing activities, and discuss how we can expand and improve on them, and share the results. We are forming a grassroots organization, the CSE Software Forum (cse-software.org), which we plan to use as a venue to continue the discussions and interactions begun at this BOF, and to complement “traditional” venues for the academic discussion of SWE for CSE, such as conferences and workshops.
The SC Conference Series provides an ideal venue for this kind of discussion. A large fraction of the attendees are CSE practitioners, or researchers who support such activities. There is also an SC16 workshop closely associated with this topic. We expect a high level of interest.
We are working to recruit an additional organizer from Europe to help ensure strong international participation.
Birds of a Feather Index