Quality Assurance

The development of our software follows a mixture of  Scrum and Rational Team Concert (IBM) as  structured and methodical approach. Coordination of activites of our employees is enabled by project management via our issue tracking system (Mantis) and the version control system Git. Both are integrated into an Eclipse-based IDE. Our efficient and agile software development model requires regression tests for each build (short tests) and after pushing changes to central repository (medium size tests). Long running tests are executed automatically with our nightly builds. This allows us to detect bugs and incompatibilities between versions or platforms early.

Quality assurance of our software is assisted by the following tools:

• Static code analysis with Cppcheck und scan-build (Clang)
• Dynamic code analysis with Valgrind (Linux) and HEAPCHK (Host)
• Custom-tailored heap memory monitoring for the detection of buffer overflows, double frees and similar
• Custom testing framework for all platforms; allows easy and efficient development of function, module, component and regression tests
• Code coverage analyse is done with Coco of Froglogic as integrated part of our testing framework
• The graphical user interfaces (GUI)  on Unix and Windows systems are testet with with Squish von Froglogic
• The ISPF panels on z/OS are tested with x3270 unsing Phyton-Scripts (py3270) who are executed as z/OS test over our FLCL remote  on a central linux server

We plan to turn our testing framework (FLTF) into a dedicated open source project at some point. Complex tests are easily and quickly written in C code to get maximum code coverage with few lines of code. The tests can be executed fully automated. Test results are written to HTML files so that all tests on all platforms and for every developer can be accessed from the intranet via web browser. If all tests have been successful, the FLTF offers button for publishing the current version of the product.

Before committing changes, every developer must build these on Linux, Windows, USS and z/OS and all regression tests must pass successfully. After pushing changes to the central Git repository, further tests are performed. If a problem is detected, the developer in question receives an e-mail and its changes are not merged into the main development branch until the problem has been resolved.

In addition, we carry out cross-platform tests to ensure interoperability between platforms. For example, output generated on a Linux system is tested on a z/OS system for correct readability and vice versa. We also verify that cryptographic hardware - such as PKCS # 11 tokens and IBM CCA devices - is compatible with our software implementations of crypto algorithms. For standard data formats, we also test for interoperability with standard tools such as GnuPG, gzip and 7-Zip.

Like the version number that each FLAM release has, each individual component has a component-specific version number. In support cases, this helps us to determine the exact software version of the customer installation to be able to offer help quickly. It also makes inconsistent installations easier to recognize.