The Problem
A company has thousands of automated tests and wants to reduce execution time by distributing tests across multiple machines. Design the test execution engine.
Requirements
- • Accept test execution requests
- • Queue and prioritize test jobs
- • Distribute tests across execution workers
- • Support parallel execution
- • Track test status
- • Retry failed tests when appropriate
- • Handle worker failures
- • Collect execution results
- • Scale workers dynamically
High-Level Architecture
Architecture Components
API Gateway — receives execution requests.
Test Scheduler — validates requests and creates execution jobs.
Message Queue — stores pending test jobs.
Execution Workers — consume jobs and execute tests.
Worker Registry — tracks available workers and their capabilities.
Result Collector — receives test results and execution metadata.
Test Database — stores execution history and status.
Reporting Service — converts execution data into dashboards and reports.
Monitoring Layer — tracks worker health, queue depth, execution time and failures.
Senior QA Perspective
“I would treat the execution engine as a distributed system rather than simply adding more Selenium machines. The scheduler should decouple test requests from execution through a queue. Workers should be independently scalable and preferably stateless. The design should account for worker failure, duplicate execution, retries, queue backpressure and observability. The goal is not only faster execution but predictable and reliable execution at scale.”
Interviewer Follow-ups
- • What happens if an execution worker crashes?
- • How would you prevent duplicate test execution?
- • How would you prioritize smoke tests over regression tests?
- • How would you handle queue overload?
- • How would you autoscale workers?
- • What metrics would you monitor?
- • How would you retry failed tests safely?