Engineering systems at Reward Gateway.
I led quality engineering for a complex employee engagement platform serving thousands of clients and millions of users. Hands-on work covered AI evaluation, delivery infrastructure and incident analysis.
- Role
- Head of Quality Engineering
- Team
- Grew the quality engineering function
- Organisation
- Supported more than 200 people in product and engineering
The evidence behind an AI release
Reward Gateway’s first customer-facing AI assistant answered employees’ questions from their own company’s content, on web, iOS and Android. I built the test suite to assess release readiness, wrote the gap analysis and risk register for the production go/no-go, and supplied the QA test report for the panel.
The test suite covered:
- Security. Authentication, tenant isolation, personal data and prompt injection, including oversized and mixed-language attempts to get past guardrails.
- Answer quality. Required and prohibited content, retrieval relevance, grounding, citation accuracy and consistency.
- Running costs and resilience. Latency, streaming, failure handling, load, data ingest and token cost.
- After launch. Versioned datasets and regression gates supported ongoing evaluation. Test failures and issues identified through Datadog monitoring informed new regression tests.
Understanding the cost of production incidents
A Python pipeline combined incident, documentation and monitoring data to assess prevention opportunities and develop a business case for improvement.
Test infrastructure for reliable releases
I built the test framework and CI integration supporting end-to-end tests, most of which were written by my team. Tests ran at pull-request, release and nightly stages. The infrastructure remained in use through two changes of ownership during my tenure, including the Edenred acquisition.
- Fast feedback. Suites ran in parallel by product area, using repeatable environments with predefined test data. Reports combined test results with application-error checks.
- Introducing shared test frameworks. I introduced shared test frameworks and trained teams on them. Quality owned the frameworks; backend engineers owned API tests covering end-to-end journeys across services.
- Localisation. Missing translations caught automatically in CI.
- Capacity. Representative workloads, controlled scaling and order-fulfilment checks.
- Traceability. Test cases linked to releases, with scope agreed before build.
Scaling quality as the team grew
Grew the quality engineering function across London and Sofia and developed an operating model for shared engineering responsibility for delivery quality.
- Operating model. Strategic vision, RACI and target operating model, backed by shared tools, standards and coaching.
- Shared metrics. Datadog quality metrics provided a shared view of delivery quality.
- Progression. A technical career framework for quality engineers.
- New routes in. Colleagues from outside engineering completed a quality engineering intern programme covering manual testing, test management and automation, and were recommended for engineering roles.
Accessibility evidence for enterprise procurement
Automated accessibility checks covered web, iOS and Android. I jointly scoped a WCAG 2.2 AA audit and VPAT report, including the user journeys assessed, to support enterprise procurement requirements.
AI testing and accessibility evaluations informed supplier recommendations, documenting limitations.