Case Study · E-Commerce

Fabletics: Moving Robot Framework Tests to Playwright with an AI Agent

API and UI test automation for Fabletics and Savage X in nine regions, with an engineer reviewing every AI-written test.

engagement / fableticsOngoing
Client
Fabletics, activewear e-commerce
Brands
Fabletics, Savage X
Regions
NA, UK, DE, FR, ES, NL, DK, SE, EU
Engagement
4+ years, ongoing
Team
Senior QA automation lead (our founder), 2 automation test engineers
Stack
PlaywrightPlaywright MCPPactumJSTypeScriptJenkinsGitHub ActionsClaude CodeGitHub Copilotqa-core-heal
Every AI-Written Test Reviewed by an Engineer
1,000+Automated Tests
450+Tests Moved to Playwright
9Regions
2Brands on One Platform

The Result

What Changed for Fabletics

  • Releases with no blocker or critical bugs. None has reached production: a critical bug found in testing is fixed before the build is delivered. Some releases ship with no known bugs; when minor bugs get through, it is typically 3 to 5 in a release, each logged and tracked. Regression runs in Jenkins caught the three bugs below before release, so support agents and merchandisers never met them.
  • 1,000+ automated tests in the release pipelines. Smoke and regression run in Jenkins and GitHub Actions.
  • 450+ tests moved without a manual rewrite. The AI migration agent put API and UI tests in one language, TypeScript, with an engineer reviewing every converted test.
  • A partnership of 4+ years, still running.

Overview

Four Internal Tools, Two Brands, One Migration

Fabletics sells activewear online across North America and Europe, and Savage X, a lingerie brand, runs on the same platform. SprintSynergy has worked on e-commerce test automation for Fabletics for more than four years, and today covers both brands. We write and maintain 400+ API tests in PactumJS and 650+ UI tests for the internal tools behind the stores. Smoke and regression pipelines in Jenkins and GitHub Actions run them. To move the UI tests from Robot Framework to Playwright, we built an AI agent that converts them. It moved 450+ tests, and an engineer reviewed each one before it joined the suite.

Sample Conversion, Simplified

The Problem

Risk Behind the Shopping Pages

Two brands share one platform. Every change has to work for each brand, in each region and in more than one environment.

Much of the risk sits behind the shopping pages. Support agents look up members and act for them. Merchandisers create products and pick their images. Marketing teams change what the stores show. A bug in any of these internal tools reaches shoppers without touching the storefront code.

The UI tests were written in Robot Framework. We proposed moving them to Playwright, mainly because it works better with AI agents, and Fabletics approved. That left 450+ tests to move without rewriting each one by hand.

What We Tested

Two Suites, One Language

API Suite: 400+ Tests in PactumJS and TypeScript

The API tests cover member accounts, membership, products, cart, search, promotions and marketing content.

UI Suite: 650+ Playwright Tests Across Four Internal Tools

Customer Service Console
The tool support agents use to help members, across both brands, including region-specific VIP compensation tests.
420+
Product Media Tool
Controls the photos and videos shown for each product.
110+
Product Catalog System
Where products are set up before they go on sale.
60+
Content Management System
Where marketing edits store content.
50+

Environment variables choose the brand, region and environment for each run, so one suite can target either brand and each region.

How We Built It

AI Agents, With an Engineer on Every Test

  1. An AI Agent for the MigrationWe built a conversion agent with GitHub Copilot and Claude. It reads a Robot Framework test, writes the same test in Playwright with TypeScript and checks the result in a live browser. Every converted test then goes through a pull request, and an engineer reviews it before it is merged. More than 450 tests moved this way.
  2. AI Agents for New and Failing TestsThe team works with AI agents in Claude Code, connected to the official Playwright MCP server, which lets an agent drive a live browser. The agents explore the app and draft test plans, write tests from real browser steps, and investigate and fix failing tests. The same rule applies: no AI-written test joins the suite until an engineer has reviewed it.
  3. Page Object ModelEach screen in each tool is a page object. When a screen changes, we update one page object instead of every test that uses it.
  4. Self-Healing LocatorsThe Playwright suite uses qa-core-heal, the open-source self-healing locator package built by our founder.
  5. Smoke and Regression PipelinesSmoke and regression run as separate pipelines in Jenkins and GitHub Actions. The 650+ UI tests run across 8 Jenkins pipelines in parallel, and a full run takes about 20 to 25 minutes.
  6. TrainingWe train the client's manual QA engineers in test automation.

What We Found

Caught Before Release

Three bugs our regression runs in Jenkins caught before release.

Customer Service Console · Both BrandsRegression

Order Cancellation

Support agents could not cancel an order with the status Placed, although Placed orders can be cancelled once a reason is given. Had it gone live, agents could not have cancelled eligible orders for customers.

Customer Service Console · Savage XRegression

Returns

Finalizing a new return failed with an error saying the customer ID was missing, although the order already had one. Had it gone live, agents could not have completed returns for the affected orders, and customers would have waited longer.

Product Media Tool · FableticsRegression

Product Publishing

After a product was edited, the publish window stayed stuck on screen and blocked any further action on that product. Had it gone live, product updates would have stalled.

In Place Now

Running Today

  • 1,000+ automated tests across the API and four internal tools, for two brands in nine regions.
  • 450+ UI tests moved from Robot Framework to Playwright by an AI agent, each reviewed by an engineer.
  • API and UI tests in one language, TypeScript.
  • Smoke and regression pipelines in Jenkins and GitHub Actions: 650+ UI tests across 8 Jenkins pipelines, a full run in about 20 to 25 minutes.
  • A SprintSynergy team of three across several Fabletics teams: our founder as senior QA automation lead, and two automation test engineers.
  • Manual QA engineers learning automation alongside us.

Questions

Frequently Asked Questions

Yes. We built an AI agent with GitHub Copilot and Claude that converts a Robot Framework test into a Playwright test in TypeScript. An engineer reviews every converted test in a pull request before it is merged. More than 450 tests moved this way.

Moving a Test Suite to Playwright?

Book a free 30-minute QA strategy call. A written QA plan follows within 48 hours.

Book a Free QA Strategy Call