How the test automation companies are scored

The ranking scores twelve test automation service providers on eight dimensions. The dimensions and weights were fixed before any vendor data was collected. They put the most weight on what a buyer of automation work keeps after the contract: documented automation cases, ownership of the test code, and how the suite is run and maintained in CI.

Each vendor is scored from 0 to 100 on every dimension, using only public evidence. A score describes what can be verified, not what a vendor may do in practice without publishing it. The scoring notes for each vendor cite the data field and the source behind every score.

How the final scores were set

Each vendor was scored three times, in three separate runs that applied the same rules and did not see each other's results. The score on each dimension is the median of the three runs. Where two vendors still tie on every dimension, the order follows the average of the three runs, dimension by dimension in the order of their weights. The notes for each run are kept with the ranking data.

How evidence is read

The same rules were applied to every vendor. They clarify how public evidence is read; they do not change the weights or the level descriptions.

1. A review count or rating read from a search result of the platform's own page counts the same as one read from the page itself.
2. A platform profile with no reviews counts as no platform for the review dimension. Its rate band or minimum project still counts for pricing and engagement.
3. Review totals are the sum of review counts on platforms that have reviews. The average rating is weighted by review count.
4. A metric is a measured result of automation work, such as coverage, run time, release cadence, defect numbers or effort saved. A count of tests or scripts on its own is not a metric.
5. A measurable outcome for a named client follows the same definition.
6. Only organisational certifications count as certifications: ISO standards, SOC 2, CMMI, TMMi, TISAX and Cyber Essentials Plus. Staff qualifications, partner programmes and compliance claims such as HIPAA or GDPR do not. Certificates held by another company do not count. A certificate is verified only when it appears on the issuer's or registry's own site.
7. A minimum project listed on Clutch counts as a stated minimum engagement. A time range such as "one to two weeks" is read at its upper end.
8. A rate band from a review platform counts as third-party pricing. Package or usage prices on the vendor's own site count as published prices.
9. A vendor gets the highest level whose conditions are all met. Partly met levels are noted in the scoring notes but do not raise the score.
10. A general statement that the client owns deliverables is read as general ownership. A statement that the client owns the test code, or that tests live in the client's repository, is read as test-code ownership. A statement about avoiding lock-in is not an ownership statement.
11. A case counts as an automation case only if it describes automated functional, regression, API or mobile tests. Load-testing and AI triage cases are not counted.
12. Engagement models are counted as distinct commercial models, such as dedicated team, managed service, project-based work and staff augmentation. Billing terms and delivery locations are not models, and a pilot counts as free only when the vendor says so.

Scoring scales

Each dimension is scored against the levels below. A vendor gets the highest level whose conditions its public evidence fully meets; there are no scores between levels. How evidence is read is set out in the interpretation rules above.

Framework depth and documented cases · 20%

Documented automation cases with coverage or regression-time metrics; breadth across Playwright, Cypress, Selenium, Appium and API frameworks.

Score	What the public evidence shows
0	No public automation case and no named frameworks.
25	Frameworks are named on the site, but there is no public automation case, or cases have no metric.
50	At least one public automation case with a metric (coverage, regression time, release cadence or defect escape), or three or more cases without metrics; frameworks cover web or mobile, not both.
75	Three or more public automation cases, at least two with metrics; frameworks named across web (Playwright, Cypress or Selenium) and at least one of mobile (Appium or native) or API.
100	Five or more public automation cases with metrics across at least two industries; web, mobile and API frameworks all named, with cases that show at least two of them in use.

Test code ownership and handover · 16%

Documented policy that the client owns the code, standard frameworks, no proprietary runner lock-in, documented handover.

Score	What the public evidence shows
0	Nothing public on who owns test code or what it runs on.
25	Automation is built on open-source frameworks, but nothing public on code ownership or handover; or delivery depends on a proprietary runner with no stated export path.
50	Open-source frameworks plus a general public statement that the client owns deliverables or IP; no described handover.
75	Public statement that tests live in the client's repository or that the client owns the test code, on open-source frameworks, with no proprietary runner required.
100	The 75 level plus a documented handover or exit process (documentation, knowledge transfer, training of the in-house team) published on the site.

CI/CD integration and maintenance · 14%

Documented CI integration practice, flaky-test handling, maintenance model after delivery.

Score	What the public evidence shows
0	No public statement on CI integration or maintenance.
25	CI/CD integration is mentioned as a service without named tools or practice.
50	Named CI tools (for example Jenkins, GitHub Actions, GitLab CI, Azure DevOps) or a described pipeline integration practice.
75	Named CI tools plus one of: a described flaky-test practice, or a described maintenance model after delivery.
100	Named CI tools, a described flaky-test practice, and a described maintenance model (who fixes broken tests, in what time, under what plan), with at least one case showing it.

Client review quality and volume · 12%

Verified reviews on Clutch, G2, GoodFirms, weighted to automation engagements.

Score	What the public evidence shows
0	No profile with reviews on Clutch, G2 or GoodFirms.
25	Fewer than 10 verified reviews in total, or average rating below 4.5.
50	10-29 verified reviews in total with average rating 4.5 or higher.
75	30-59 verified reviews in total with average rating 4.7 or higher, on at least one platform.
100	60 or more verified reviews in total with average rating 4.8 or higher, on at least two platforms.

Named clients and published outcomes · 12%

Named clients with measurable automation outcomes in public cases.

Score	What the public evidence shows
0	No named clients in public sources.
25	Client logos or names only, without outcomes.
50	One or two named clients with a measurable outcome in a public case.
75	Three or four named clients with measurable outcomes, at least one about automation.
100	Five or more named clients with measurable outcomes, at least three about automation.

Security and delivery governance · 10%

Verified certifications, data handling, environments.

Score	What the public evidence shows
0	No certification and no public data-handling statement.
25	A public data-handling or NDA statement, no certification.
50	One relevant certification (ISO 9001, ISO 27001, SOC 2, CMMI) stated with a source.
75	ISO 27001 or SOC 2 plus one more certification, or a verifiable certificate from the issuer's registry.
100	ISO 27001 or SOC 2 verified at the issuer, plus a second certification and a published data-handling or environment policy (test data, access, client-controlled environments).

Engagement flexibility and onboarding · 9%

Models offered, documented time to start, minimum contract.

Score	What the public evidence shows
0	No public information on engagement models.
25	One engagement model described.
50	Two or more models (for example dedicated team, managed service, project-based, staff augmentation).
75	Two or more models plus either a stated time to start or a stated minimum engagement.
100	Three or more models, a stated time to start of 14 days or less, and a stated minimum engagement or free pilot.

Pricing transparency · 7%

Published rates or ranges and minimum project size.

Score	What the public evidence shows
0	No public price information anywhere.
25	Only a third-party band (for example the Clutch hourly-rate band) or only a minimum project size.
50	Third-party hourly band and minimum project size both public.
75	Rates or package prices published on the vendor's own site.
100	Rates or package prices on the vendor's site, plus minimum project size and what the price includes.

Totals and ties

The total is the weighted sum of the eight scores divided by 100, rounded to one decimal. It is computed at build time from the dimension scores; no total is entered by hand. Equal totals are resolved by the score on the first dimension, then the second, and so on in the order above. If vendors still tie, the average of the three scoring runs decides, dimension by dimension.

Scores by dimension