Screenshot Testing with Chromatic

Is this bubble chart correct?

Answering this question with unit tests requires breaking it down into several smaller questions:

• Is the method responsible for calculating bubble size working as expected?
• Does VictoryScatter render the correct number of Point components?
• Does each Point component receive the correct props?
• Does each Point component render the correct SVG path?
  • Is the path circular?
  • Is the path centered around the correct coordinate?
  • Is the path styled correctly?
• Are the axes correctly positioned?
• Do the axes display correct tick labels for the given data? And so on...

No matter how many unit test you write for a given component, one nagging, unanswerable question always remains- What have I forgotten to test?

Snapshot testing can help answer this question, but its answers are often unhelpful, particularly when testing the output of SVG-heavy components. Diffs can be verbose, and border on unreadable, especially when components render complex paths. Worse, these test can be extremely brittle, with changes in things like attribute order causing spurious test failures.

Screenshot testing is a perfect match for an SVG-based charting library like Victory. Visual diffs make changes apparent and unmistakable. Victory relies on Storybook and Chromatic for automated screenshot testing. We maintain a library of stories that describe Victory components both in isolation, and composed in complex charts. Chromatic builds this library of stories with every pull request, and compares it against a set of accepted stories on the master branch. Any differences trigger a notification on the pull request:

The Chromatic dashboard gives an overview of everything that has changed.

And, best of all, Chromatic highlights visual differences in individual stories. In the bubble chart below, each bubble is slightly too large. The difference is very slight, but instantly apparent when highlighted in bright green.

Screenshot testing with Chromatic has been an extremely useful and confidence-inspiring addition to Victory's test suite. Chromatic also lends itself to collaboration and open source projects. Anyone can explore Victory's library of visual tests and sumbit pull requests to add more.