Test sheets

Software testing is an important aspect of modern software development. Testing is performed to ensure that a product or component meets the requirements of all stakeholders. Just as the requirements themselves tests can vary widely in their nature. Some tests check whether executing certain code paths results in a correct state or answer while others may check whether a certain code path delivers its results in a specified amount of time.

However writing and evaluating tests is often not much different from programming itself. Tests are usually written in a formal programming language such as Java/JUnit. This necessitates that in order to create a test and understand its results knowledge of a formal programming language is required. This is a significant barrier of entry for stakeholders without a background in IT even though they may be interested in the tests themselves. Even without deep IT knowledge a stakeholder may still be interested in how well a product performs with regard to her requirements.

Visual test representations such as the UML Testing Profile try to lower the barrier of entry into testing. However most of the time these visualizations are only partial descriptions of the tests and so do not contain all the desired information for evaluation.

The Software Engineering group has started to develop a new representation for tests called Test Sheets. The goal is to create a way to define tests which combines the power and completeness of formal programming language with a representation that is easy to understand and work with even for people with little IT knowledge. This is achieved by representing a test in tabular form as a spreadsheet. Rows in a Test Sheet represent operations being executed while the columns represent input parameters and expected results. The actual content of a cell can be made dependent on other cells by addressing them via their location. This works in a way similar to existing spreadsheet software such as Microsoft Excel. After executing a test the cells for each expected result is colored according to the result of the test. A successful test causes cells to become green while failed tests are indicated by red cells.

Details about Test Sheets: