How Self-Healing Test Locators Work (And When To Refuse)

A developer renames an id from inputField to inputFeld and back again during a refactor. Nothing about the page looks different. The next morning a dozen tests are red, and none of them are red for an interesting reason.

That is most test maintenance. Not finding bugs, just repairing references to elements that still exist and still work.

Self-healing test automation is supposed to solve this. In practice, most of it makes the problem worse in a quieter way. This post explains how a locator can be repaired safely, where the line sits, and why the most valuable thing a healing tool does is refuse.

Our founder built this approach into qa-core-heal, a free open source package for Playwright that our engineers use on client projects. It is MIT licensed and runs locally with no model call and no API key.

Why most self-healing cannot be trusted

The usual design uses the test result as the oracle. The tool swaps in a new selector, re-runs, and if the test goes green it keeps the change.

That sounds reasonable until you look at what a typical test asserts. Plenty of tests assert that a page loaded, that an element is visible, that a count is greater than zero. A tool that is allowed to try selectors until the test passes will eventually find a selector that satisfies a weak assertion while pointing at the wrong element.

The test is now green, the suite is now lying, and nobody will look at it again because green tests do not get reviewed. A red test costs you an hour. A wrong heal costs you the confidence of the whole suite.

So the oracle cannot be the test result. It has to be evidence about the element.

What a locator failure actually is

Before anything can be healed, the failure has to be identified as a locator failure at all. A test can go red for four different reasons:

• the locator no longer resolves
• the assertion is wrong or out of date
• the application is genuinely broken
• the environment failed, such as a timeout or a DNS error

Only the first one is a candidate for healing. The other three need a human, and a tool that rewrites code in response to a real application bug is actively dangerous.

This is why the run comes first. The tests are executed, the failures are read from the Playwright report and trace, and each one is classified before any repair is considered. An assertion failure is reported and left alone. An application failure is reported and left alone.

The evidence bar

Once a failure is confirmed as a locator failure, the page is scanned as it actually was at the moment of the failure, not as it is on a fresh load. State matters. An element inside a closed dropdown does not exist on a fresh page.

Every candidate element on that page is collected with its identifying properties: id, accessible name, role, label, title, placeholder, and text. The broken locator is then compared against that pool, and a repair is only written when three conditions hold at once.

1. One close match. The similarity to a single candidate has to be high enough to be an obvious repair, such as a one character typo, not a plausible guess between two options.
2. The same kind of element. If the broken locator was a button, the candidate has to be a button. A heading whose text happens to match is not a button, and this rule catches more bad proposals than any other.
3. Uniqueness on the page. If the proposed locator would match two elements, it is not a repair. It is a new flaky test.

If all three hold, the locator is rewritten in the source file and the test is re-run. If any one of them fails, nothing is written.

That last sentence is the whole design. Most tools optimise for how many failures they can fix. This one optimises for never being wrong, and accepts a lower fix rate as the price.

Three verdicts, and only one of them touches your code

Heal. The evidence is near certain. The locator is rewritten, every identical occurrence of it inside that failing test is rewritten with it, and the test is re-run to confirm.

Refuse. The evidence is thin. Instead of a guess, you get a named reason: which element came closest, what score it reached, and what is missing. If the closest candidate is an icon only button with no accessible name, the message can say so and suggest adding an aria-label. The refusal is a code review comment, delivered at the moment the information is useful.

Revert. The heal was applied, the test was re-run, and it is still red. The change is undone and the file is restored. The report also distinguishes between the two possible causes: the heal was wrong, or the heal was right and the test fails for a different reason such as a stale assertion.

Nothing is left half applied. That guarantee is what makes the tool safe to run in CI.

What it refuses on purpose

These are not gaps. They are decisions, and they hold on every run.

• Generated ids. A locator pointing at a GUID that changes on every load cannot be repaired, because the next value is unknowable. The fix is a stable test id in the application.
• Indexed XPath. A path like the third div inside the second section carries no identity. Matching it to anything would be coincidence.
• Nameless elements. If the target has no accessible name, no label, and no stable id, there is nothing to match on. You get a message explaining what to add.
• Reordered words. Changing "Submit order" to "Order submitted" is not a typo. Word order carries meaning, and treating the two as equivalent opens a whole class of wrong matches.
• Anything less specific than what broke. A repair never replaces a precise locator with a vaguer one, even if the vaguer one would pass.

What it does not do yet

Honest limits, because a tool that hides them is a tool you will stop trusting after the first surprise.

• Elements inside iframes are not healed today. The refusal message points out that the element may live in an embedded frame, and frame support is on the roadmap.
• Intent is never recovered. If a button genuinely changed meaning, the test should fail, and it will.
• Playwright only. Cypress, Selenium and Appium versions are on the shelf until there is real demand for them.
• It is pre 1.0. The command line surface and the JSON output are stable in practice, but the semantic version commitment comes with 1.0.

What we measured

The package carries its own test harness of deliberately broken locators, currently 72 cases that must heal, 32 that must be refused, and 10 that must be left untouched, alongside 189 unit tests. A change that flips any pinned refusal into a heal does not ship.

For the last release, the published build was then run against a public practice site, first by the author and then by SprintSynergy engineers who did not write the tool, across 21 acceptance runs covering typo ids, missing accessible names, shadow DOM, multiple occurrences in one test, and deliberate revert cases.

The result was 21 acceptance runs and zero wrong heals. Every failure landed on the safe side, meaning a refusal or a revert rather than a bad rewrite. Three of those runs produced findings that were recorded rather than hidden, and they went straight into the work list for the next release.

Try it

npm install qa-core-heal
npx qa-core-heal

It runs your existing Playwright suite, reports what it healed, what it refused and why, and leaves your repository clean if anything did not verify. There is no account, no key, and no data leaving your machine. The source is on GitHub and the full release detail is on the qa-core-heal page.

FAQ

Is self-healing test automation safe to run in CI?

It depends entirely on whether the tool can undo itself. A tool that rewrites source and leaves the change in place when verification fails will corrupt your suite over time. Look for a documented revert behaviour and a non zero exit code before you let anything run unattended.

Does self-healing hide real bugs?

It can, if healing is applied to every failure. It should not be, because a locator failure and an application failure are different events. Any tool you use should classify the failure first and only touch the locator category.

Does it need an LLM or an API key?

Not this one. The matching is deterministic string and structure comparison against elements read from the failure page, so the same input always produces the same verdict and nothing is sent anywhere.

What happens when it cannot repair a locator?

It refuses and explains. The message names the closest candidate, the score it reached, and the missing piece, such as an element with no accessible name. That output is usually more useful than a repair, because it tells you what to fix in the application.

Will this remove the need for a QA engineer?

No. It removes one specific chore, which is repairing references to elements that still work. Deciding what to test, what an assertion should prove, and whether a failure matters is the actual job.

Closing

Self-healing is not a magic trick. It is a narrow, well defined repair with a hard evidence bar and an undo. Everything outside that bar should come back as a clear explanation, not a guess.

If your suite spends more time being repaired than it spends finding problems, tell us what it looks like and we will point you at the part worth fixing first.