DEVOPS QUALITY ENGINEERING
A leading cloud technology provider required a robust test engineering process to ensure the quality, performance, and security of its systems. These systems are integral to migrating new customers.
The recent accelerated adoption of cloud services increased the necessity for comprehensive and automated testing targeting the provider’s code and systems. Any concealed risks need to be identified and mitigated to avoid costly production failures or security issues.
In this case study, we look at how Valorem Reply’s quality engineering team has helped to ensure seamless digital transformation for the customers adopting cloud platforms with proactive identification of functional, API, and security issues and test effort cost reduction.
CHALLENGES
- Multiple inter-dependent systems have 150+ APIs with different versions that frequently change, making it a strenuous effort to maintain test suites
- Dependency on legacy applications, out of which some have vulnerable API endpoints prone to security threats
- Hotfix releases for Security bug fixes need a much faster turnaround time
- The same set of API endpoints and corresponding UI tests require repetitive testing across multiple environments. Manually testing them poses quite a challenge as they are tedious
SOLUTION
- Hybrid BDD framework
- UI automation for multiple applications using inbuilt BDD UI framework and tools like Specflow, Selenium Web driver, C# and MSTest
- API automation tests cover functionality and security such as parameter fuzzing, authorization tests, cross customer data validation, etc.
- Advanced logging and redaction
- Daily recurring unit tests to ensure the framework robustness
- Azure CI/CD integration and nightly execution of automation tests in multiple environments
- Custom tools to generate API headers for testing and analyze test coverage
OUTCOME
- More than 6099 automated tests (including 1202-unit tests)
- Daily average savings of approximately 770 hours of the manual testing effort
- Based on our new custom BDD approach,
- Average lines of code per test reduced from 21.3 to 10.8 (50% reduction)
- The number of tests automated per workday increased from 1.8 to 21.7 (12x upsurge)
- Around 173 security bugs identified and reported through automated tests