Agile best practices recommend overlapping development and testing cycles. This helps identify and resolve software defects early, while minimizing scenarios where developers are simultaneously trying to fix old bugs and build new features.
Overlapping development and testing cycles also promotes the incremental delivery of working code. This process allows new features and bug fixes to be rapidly promoted, while allowing the Customer to review new features as they are developed, and make decisions about when specific features are ultimately deployed to Production.
Partnet’s software development process is enhanced by the incorporation of continuous delivery methodologies. Continuous delivery is a collection of techniques designed to improve and automate software delivery. These techniques include:
- Continuous delivery pipeline
- Automated testing
- Mainline development
- Continuous integration
Continuous delivery allows software to be developed to a high standard, easily packaged, and rapidly deployed to the various environments. Additionally, new features and bug fixes can be rapidly promoted to the Production environment with minimal risk and overhead.
Our developers use a continuous delivery pipeline to implement continuous delivery principles.
The deployment pipeline is like an automated assembly line which the code passes through to ensure it is ready for Production. This assembly line includes:
- Building the software
- Running unit tests
- Building the application deployment
- Running automated User-Interface (UI) tests
This method allows functional software to be rapidly promoted to non-Production environments for Customer inspection; or promoted directly to Production in cases where a feature or bug fix is immediately required. For projects employing the Scrum process, this cycle repeats within each sprint until the Product Owner signifies all requirements have been met and the feature is Production-ready
Automated testing requires close interaction between software developers and test engineers. The following process is employed within each development cycle (Scrum or Kanban):
- Test engineers write test outline(s) for each development task.
- Developers and test engineers determine what portions of the test outline will be automated and how, e.g., unit testing, automated testing.
- Developers write code to satisfy the user story acceptance criteria.
- Developers implement associated unit tests; test engineers write automated tests in parallel.
Automated tests are run during the development pipeline, several times a day. If any tests fail, a ticket is created in Partnet’s issue-tracking system and assigned to the developer for immediate resolution.
Mainline Development and Continuous Integration
Adoption of automated build, deployment, and testing practices allows our developers to employ a mainline development model. Mainline development allows multiple release versions to be concurrently developed within the same code base. This is accomplished by building feature toggles into the software. Feature toggles act like light switches that allow functionality to be turned on or off in a code deployment. For example, a feature toggle could be used to “turn off” functionality still under development but anticipated for a future release version. Partnet uses this mainline development approach to build features multiple Go-Live phase requirements simultaneously. Mainline development also supports the continuous integration of code. This means that each software developer frequently commits their code changes to the shared code repository. This promotes early detection of code collisions that can occur when two or more modules of code are integrated, while producing fewer collisions overall.
Working in a mainline development model also allows our developers to:
- Operate more efficiently, with more flexibility.
- Work on multiple features and versions simultaneously with fewer code collisions.
- Eliminate complexity and risk associated with maintaining multiple versions of the same code base for different releases, which also reduces overall costs.
- Continuously integrate changes without breaking existing functionality.
Testing and quality assurance activities are performed in parallel with software development in order to ensure user stories, or other increments of work, are developed and tested as expediently as possible. This process begins with preparation of a test outline for each user story or problem report assigned to the current development cycle. The test outline determines:
- How testing efforts will be coordinated
- Where automated testing can be performed
- What the testing priorities are based on the following factors:
- Likelihood of failure (risk)
- Importance to the customer/business
- Expected load/usage
Partnet’s automated testing process draws on Behavior Driven Development (BDD) methodologies. BDD defines techniques for automated testing by writing behavioral specifications in a structured, machine-readable syntax.
Each automated test consists of two parts:
- Story file: describes how a unit of software is expected to behave in order to satisfy the requirements. The story file includes:
- Reference to the originating requirement, or user story.
- A list of preconditions that must be true in order to execute the test.
- One or more scenarios to be executed and their expected outcomes.
- Step definitions: blocks of code, or test scripts, that execute the test scenarios in the associated story file.
Manual and Exploratory Testing
Manual test cases are prepared for any areas of the code that cannot be automated. Each test case documents the parameters and scripted procedures required to satisfy all conditions of the test. All test cases are peer reviewed by the test engineering and development teams to verify and validate:
- Requirements have been covered
- Test priority is correct
- Accuracy of the test
- Thoroughness of the test coverage
- Possible impact to other functional areas
- Performance and security considerations
Manual testing also includes unscripted exploratory tests. Exploratory testing is a technique emphasizing simultaneous test design and execution, and allows Test Engineers to identify behavioral corner or edge cases in the software. Corner and edge cases describe possible scenarios or operations that may occur outside a standard use case. Exploratory tests can then be used to create more formal tests around any identified corner or edge cases.
Test Engineers run the suite of automated tests after each deployment to the non-Production environment. If any tests fail, a ticket is created in Partnet’s issue-tracking system and assigned to the Software Developer for immediate resolution.
Load and Stress Testing
Partnet load tests all new software features where heavy usage is likely to affect functional performance. Test engineers determine which features need to be load tested during the Solution
Design stage as part of user-story review.
Partnet performs load testing using Apache JMeter—a desktop application designed specifically to load test functional behavior and measure application performance by simulating user interactions in a web browser. The tool has built-in reporting features that measure throughput, response times, and success rates for client requests.
Load tests can be created to measure performance for a variety of services and protocols, including:
- Hypertext Transfer Protocol (HTTP/HTTPS)
- File Transfer Protocol (FTP/SFTP)
- Database connectivity (JDBC)
- Web services
- Simple Object Access Protocol (SOAP)
- Representational State Transfer (REST)
- Lightweight Directory Access Protocol (LDAP)
- Messaging (JMS)
- Email (SMTP)
Test Engineers work with Configuration Managers and Production Support staff to determine
baseline criteria for testing. This includes the number of users to simulate, peak usage conditions
for stress testing, and performance benchmarks.
Partnet has a number of government customers who require 508 compliancy in their on-line products. To ensure Federal Section 508 Compliance standards are met, our suite of tests employs a combination of accessibility tools, including:
- JAWS (Job Access With Speech)
- Total Validator
- Firefox Accessibility Extension
Partnet Test Engineers use these tools on both government and commercial systems to ensure accessibility compliance. Section 508 checklists are used when gathering application requirements.
In accordance with Agile best practices, Partnet demonstrates the working software at the conclusion of each sprint. In addition, the Customer can test the software in a non-Production environment at any time. This will facilitate user feedback earlier in the development cycle and provide our developers with more time to resolve any identified issues or defects. The sooner issues are found, the easier and more cost-efficient they are to correct.
At the conclusion of the software development and testing activities, Partnet convenes a Technical Requirements Review (TRR) with the Customer and other essential stakeholders. The TRR will be conducted as a structured walkthrough of the high-level development requirements—demonstrating how each requirement is supported in the application and how they can be tested. Draft test cases are delivered to the Customer at this time. Partnet also delivers draft release notes, which contain a description of each feature and/or Problem Reports in the release.
After the Testing Phase is completed, we are ready to move on to Part 5 of the Software Development Life Cycle – Release Planning and Deployment. Part 5 of our blog series: The Importance of a defined Software Development Life cycle will cover Software Release Planning and Deployment.
I hope that you have enjoyed Part 4 of our Blog Series, the Importance of a Defined Software Development Life Cycle. If you have missed any parts of the series, check them out at partnet.com.