From f2372d1cc0cd62d973464fc66301be28166c4bcd Mon Sep 17 00:00:00 2001 From: Yann Braga Date: Wed, 8 Nov 2023 19:15:08 +0100 Subject: [PATCH] Add storyshots migration guides --- MIGRATION.md | 21 ++ MIGRATION.portable-stories.md | 365 ++++++++++++++++++++++++++++++++++ MIGRATION.test-runner.md | 207 +++++++++++++++++++ 3 files changed, 593 insertions(+) create mode 100644 MIGRATION.md create mode 100644 MIGRATION.portable-stories.md create mode 100644 MIGRATION.test-runner.md diff --git a/MIGRATION.md b/MIGRATION.md new file mode 100644 index 00000000..979b990b --- /dev/null +++ b/MIGRATION.md @@ -0,0 +1,21 @@ +# `@storybook/addon-storyshots` Migration Guide + +`@storybook/addon-storyshots` was replaced by the [Storybook test-runner](https://storybook.js.org/docs/react/writing-tests/test-runner) in 2021, due to storyshots being a performance and maintenance problem for Storybook. As Storybook 8 moves forward with the `storyStoreV7` and its on-demand architecture, `@storybook/addon-storyshots` will become incompatible, and you'll have to migrate from it. This migration guide will aid you in that process. + +Below you will find two options to migrate. We recommend migrating to and using the Storybook test-runner, but you can decide for yourself which path to choose, by following the guides below. + +## Option 1 - Portable stories + +Portable stories are utilities from Storybook that assist in converting stories from a story file into renderable elements that can be reused in your Node tests with JSDOM with tools like [Jest](https://jestjs.io/) or [Vitest](https://vitest.dev/). This is the closest you will get from storyshots, but with the caveat that you will face similar challenges, given that the tests still run in Node. + +If your project uses React, React Native (without the [react-native-web addon](https://storybook.js.org/addons/%2540storybook/addon-react-native-web)) or Vue3, and you use storyshots extensively with complex mocking mechanisms and snapshot serializers, this migration will be the most seamless to you. + +Follow the [migration steps to portable stories here](./MIGRATION.portable-stories.md). + +## Option 2 - Storybook test-runner + +The Storybook test-runner turns all of your stories into executable tests, powered by [Jest](https://jestjs.io/) and [Playwright](https://playwright.dev/). It's powerful and provides multi-browser testing, and you can achieve many things with it such as smoke testing, DOM snapshot testing, Accessibility testing, Visual Regression testing and more. + +The test-runner supports any official Storybook framework, and is compatible with community frameworks (support may vary). If you use Storybook for React Native, you can use the test-runner as long as you set up the [react-native-web addon](https://storybook.js.org/addons/%2540storybook/addon-react-native-web) in your project. + +Follow the [migration steps to test-runner here](./MIGRATION.test-runner.md). diff --git a/MIGRATION.portable-stories.md b/MIGRATION.portable-stories.md new file mode 100644 index 00000000..b4181ce5 --- /dev/null +++ b/MIGRATION.portable-stories.md @@ -0,0 +1,365 @@ +# Migration Guide: From `@storybook/addon-storyshots` to portable stories + +## Table of Contents + +- [Migration Guide: From `@storybook/addon-storyshots` to portable stories](#migration-guide-from-storybookaddon-storyshots-to-portable-stories) + - [Table of Contents](#table-of-contents) + - [Pre-requisites](#pre-requisites) + - [What are portable stories?](#what-are-portable-stories) + - [What will you achieve at the end of this migration?](#what-will-you-achieve-at-the-end-of-this-migration) + - [Getting started](#getting-started) + - [1 - Disable your existing storyshots test](#1---disable-your-existing-storyshots-test) + - [2 - Import project level annotations from Storybook](#2---import-project-level-annotations-from-storybook) + - [3 - Use the portable stories recipe](#3---use-the-portable-stories-recipe) + - [Vitest](#vitest) + - [Jest](#jest) + - [4 - (Optional) extend your testing recipe](#4---optional-extend-your-testing-recipe) + - [5 - Remove storyshots from your project](#5---remove-storyshots-from-your-project) + - [6 - Provide feedback](#6---provide-feedback) + +## Pre-requisites + +Before you begin the migration process, ensure that you have: + +- [ ] A Storybook project with `@storybook/react` or `@storybook/vue3`. +- [ ] A working Storybook setup with version 7. +- [ ] Familiarity with your current Storybook and its testing setup. + +> **Note** +> If you are using a different renderer for your project, such as Angular or Svelte, this migration is not possible for you. Please refer to the [test-runner migration](./MIGRATION.md) instead. + +## What are portable stories? + +Storybook provides a `composeStories` utility that assists in converting stories from a story file into renderable elements that can be reused in your Node tests with JSDOM. It also makes sure to apply all their necessary decorators, args, etc so that your component can render correctly. We call this portable stories. + +Currently, the only available renderers that provide this functionality are React and Vue3. We have plans to implement this for other renderers in the near future. + +## What will you achieve at the end of this migration? + +Portable stories will provide you the closest experience possible with storyshots. You will still have a single test file in node, which runs in a JSDOM environment, that render all of your stories and snapshots them. However, you will still face the same challenges you did with storyshots: + +- You are not testing against a real browser. +- You will have to mock many browser utilities (e.g. canvas, window APIs, etc). +- Your debugging experience will not be as good, given you can't access the browser as part of your tests. + +You could consider migrating to the [test-runner](./MIGRATION.md) instead, which is more powerful, runs against a real browser with Playwright, provides multi-browser support, and more. + +## Getting started + +The first thing you have to do is to disable your storyshots tests. You can keep it while doing the migration, as it might be helpful in the process, but your ultimate goal is to remove `@storybook/addon-storyshots`. + +### 1 - Disable your existing storyshots test + +Rename your `storybook.test.ts` (or whatever your storyshots test is called) to `storybook.test.ts.old`. This will disable the test from being detected, and allow you to create an updated test file with the same name. + +### 2 - Import project level annotations from Storybook + +If you need project level annotations such as decorators, styles, or anything that is applied to your stories via your `.storybook/preview` file, you will have to add the following code to your test setup file. Please refer to the documentation from [Jest](https://jestjs.io/docs/configuration#setupfilesafterenv-array) or [Vitest](https://vitest.dev/config/#setupfiles) on setup files. + +```ts +// your-setup-file.js +import * as projectAnnotations from './.storybook/preview'; +import { setProjectAnnotations } from '@storybook/react'; + +// apply the global annotations from Storybook preview file +setProjectAnnotations(projectAnnotations); +``` + +If you are using the new recommended format in your preview file, which is to have a single default export for all the configuration, you should adjust that slightly: + +```diff +- import * as projectAnnotations from './.storybook/preview' ++ import projectAnnotations from './.storybook/preview' +``` + +### 3 - Use the portable stories recipe + +Then, create a `storybook.test.ts` file, and depending on your tool of choice, follow the recipes below. + +- [Vitest](#vitest) +- [Jest](#jest) + +#### Vitest + +This recipe will do the following: + +1. Import all story files based on a glob pattern +2. Iterate over these files and use `composeStories` on each of their modules, resulting in a list of renderable components from each story +3. Iterave over the stories, render them and snapshot them + +Fill in your `storybook.test.ts` file with the following recipe. Please read the code comments to understand + +```ts +// @vitest-environment jsdom +import { describe, expect, test } from 'vitest'; +import { render } from '@testing-library/react'; +import { composeStories } from '@storybook/react'; +import type { Meta, StoryFn } from '@storybook/react'; + +type StoryFile = { + default: Meta; + [name: string]: StoryFn | Meta; +}; + +const compose = (entry: StoryFile): ReturnType> => { + try { + return composeStories(entry); + } catch (e) { + throw new Error( + `There was an issue composing stories for the module: ${JSON.stringify(entry)}, ${e}` + ); + } +}; + +function getAllStoryFiles() { + // Place the glob you want to match your stories files + const storyFiles = Object.entries( + import.meta.glob('./stories/**/*.(stories|story).@(js|jsx|mjs|ts|tsx)', { + eager: true, + }) + ); + + return storyFiles.map(([filePath, storyFile]) => { + const storyDir = path.dirname(filePath); + const componentName = path.basename(filePath).replace(/\.(stories|story)\.[^/.]+$/, ''); + return { filePath, storyFile, componentName, storyDir }; + }); +} + +// recreate similar options to storyshots, place your configuration below +const options = { + suite: 'Storybook Tests', + storyKindRegex: /^.*?DontTest$/, + storyNameRegex: /UNSET/, + snapshotsDirName: '__snapshots__', + snapshotExtension: '.storyshot', +}; + +describe(options.suite, () => { + getAllStoryFiles().forEach(({ storyFile, componentName, storyDir }) => { + const meta = storyFile.default; + const title = meta.title || componentName; + + if (options.storyKindRegex.test(title) || meta.parameters?.storyshots?.disable) { + // skip component tests entirely if they are disabled + return; + } + + describe(title, () => { + const stories = Object.entries(compose(storyFile)) + .map(([name, story]) => ({ name, story })) + .filter(({ name, story }) => { + // Create your own logic to filter stories here if you like. + // This is recreating the default behavior of storyshots. + return !options.storyNameRegex?.test(name) && !story.parameters.storyshots?.disable; + }); + + if (stories.length <= 0) { + throw new Error( + `No stories found for this module: ${title}. Make sure there is at least one valid story for this module, without a disable parameter, or add parameters.storyshots.disable in the default export of this file.` + ); + } + + stories.forEach(({ name, story }) => { + // Instead of not running the test, you can create logic to skip it instead, so it's shown as skipped in the test results. + const testFn = story.parameters.storyshots?.skip ? test.skip : test; + + testFn(name, async () => { + const mounted = render(story()); + // add a slightly delay to allow a couple render cycles to complete, resulting in a more stable snapshot. + await new Promise((resolve) => setTimeout(resolve, 1)); + + expect(mounted.container).toMatchSnapshot(); + }); + }); + }); + }); +}); +``` + +The snapshots will all be aggregated in a single `storybook.test.ts.snap` file. If you had storyshots configured with multisnapshots, you should change the above recipe a little to use `toMatchFileSnapshot` from vitest: + +```ts +// ...everything else + +describe(options.suite, () => { + // 👇 add storyDir in the arguments list + getAllStoryFiles().forEach(({ filePath, storyFile, storyDir }) => { + // ...existing code + describe(title, () => { + // ...existing code + stories.forEach(({ name, story }) => { + // ...existing code + testFn(name, async () => { + // ...existing code + + // 👇 define the path to save the snapshot to: + const snapshotPath = path.join( + storyDir, + options.snapshotsDirName, + `${componentName}${options.snapshotExtension}` + ); + expect(mounted.container).toMatchFileSnapshot(snapshotPath); + }); + }); + }); + }); +}); +``` + +This will result in separate snapshot files per story, located near their stories file e.g.: + +``` +components/Button/Button.stories.ts +components/Button/__snapshots__/Primary.storyshot +components/Button/__snapshots__/Secondary.storyshot +// ... +``` + +#### Jest + +This recipe will do the following: + +1. Import all story files based on a glob pattern +2. Iterate over these files and use `composeStories` on each of their modules, resulting in a list of renderable components from each story +3. Iterave over the stories, render them and snapshot them + +Fill in your of your `storybook.test.ts` file with the following recipe: + +```ts +// storybook.test.ts +import path from 'path'; +import * as glob from 'glob'; +import { describe, test, expect } from '@jest/globals'; +import { render } from '@testing-library/react'; +import { composeStories } from '@storybook/react'; +import type { Meta, StoryFn } from '@storybook/react'; + +type StoryFile = { + default: Meta; + [name: string]: StoryFn | Meta; +}; + +const compose = (entry: StoryFile): ReturnType> => { + try { + return composeStories(entry); + } catch (e) { + throw new Error( + `There was an issue composing stories for the module: ${JSON.stringify(entry)}, ${e}` + ); + } +}; + +function getAllStoryFiles() { + // Place the glob you want to match your stories files + const storyFiles = glob.sync( + path.join(__dirname, 'stories/**/*.(stories|story).@(js|jsx|mjs|ts|tsx)') + ); + + return storyFiles.map((filePath) => { + const storyFile = require(filePath); + return { filePath, storyFile }; + }); +} + +// recreate similar options to storyshots, place your configuration below +const options = { + suite: 'Storybook Tests', + storyKindRegex: /^.*?DontTest$/, + storyNameRegex: /UNSET/, + snapshotsDirName: '__snapshots__', + snapshotExtension: '.storyshot', +}; + +describe(options.suite, () => { + getAllStoryFiles().forEach(({ storyFile, componentName }) => { + const meta = storyFile.default; + const title = meta.title || componentName; + + if (options.storyKindRegex.test(title) || meta.parameters?.storyshots?.disable) { + return; + } + + describe(title, () => { + const stories = Object.entries(compose(storyFile)) + .map(([name, story]) => ({ name, story })) + .filter(({ name, story }) => { + // Create your own logic to filter stories here if you like. + // This is recreating the default behavior of storyshots. + return !options.storyNameRegex.test(name) && !story.parameters.storyshots?.disable; + }); + + if (stories.length <= 0) { + throw new Error( + `No stories found for this module: ${title}. Make sure there is at least one valid story for this module, without a disable parameter, or add parameters.storyshots.disable in the default export of this file.` + ); + } + + stories.forEach(({ name, story }) => { + // Instead of not running the test, you can create logic to skip it instead, so it's shown as skipped in the test results. + const testFn = story.parameters.storyshots?.skip ? test.skip : test; + + testFn(name, async () => { + const mounted = render(story()); + // add a slightly delay to allow a couple render cycles to complete, resulting in a more stable snapshot. + await new Promise((resolve) => setTimeout(resolve, 1)); + expect(mounted.container).toMatchSnapshot(); + }); + }); + }); + }); +}); +``` + +The snapshots will all be aggregated in a single `__snapshots__/storybook.test.ts.snap` file. If you had storyshots configured with multisnapshots, you can change the above recipe a little by using `jest-specific-snapshot` (you will have to install this dependency): + +```ts +// 👇 augment expect with jest-specific-snapshot +import 'jest-specific-snapshot'; +// ...everything else + +describe(options.suite, () => { + // 👇 add storyDir in the arguments list + getAllStoryFiles().forEach(({ filePath, storyFile, storyDir }) => { + // ...existing code + describe(title, () => { + // ...existing code + stories.forEach(({ name, story }) => { + // ...existing code + testFn(name, async () => { + // ...existing code + + // 👇 define the path to save the snapshot to: + const snapshotPath = path.join( + storyDir, + options.snapshotsDirName, + `${componentName}${options.snapshotExtension}` + ); + expect(mounted.container).toMatchSpecificSnapshot(snapshotPath); + }); + }); + }); + }); +}); +``` + +This will result in separate snapshot files per component, located near their stories file e.g.: + +``` +components/__snapshots__/Button.stories.storyshot +components/__snapshots__/Header.stories.storyshot +components/__snapshots__/Page.stories.storyshot +// ... +``` + +### 4 - (Optional) extend your testing recipe + +The aforementioned recipes will only get you so far, depending on how you used storyshots. If you used it for image snapshot testing, acessibility testing, or other scenarios, you can extend the recipe to suit your needs. You can also consider using [the Storybook test-runner](https://github.com/storybookjs/test-runner), which provides solutions for such use cases as well. + +### 5 - Remove storyshots from your project + +Once you make sure that the portable stories solution suits you, make sure to remove your old storyshots test file and uninstall `@storybook/addon-storyshots`. + +### 6 - Provide feedback + +We are looking for feedback on your experience, and would really appreciate if you filled [this form](some-google-form-here) to help us shape our tooling in the right direction. Thank you so much! diff --git a/MIGRATION.test-runner.md b/MIGRATION.test-runner.md new file mode 100644 index 00000000..21d75a61 --- /dev/null +++ b/MIGRATION.test-runner.md @@ -0,0 +1,207 @@ +# Migration Guide: From `@storybook/addon-storyshots` to `@storybook/test-runner` + +## Table of Contents + +- [Migration Guide: From `@storybook/addon-storyshots` to `@storybook/test-runner`](#migration-guide-from-storybookaddon-storyshots-to-storybooktest-runner) + - [Table of Contents](#table-of-contents) + - [Pre-requisites](#pre-requisites) + - [What is the Storybook Test Runner?](#what-is-the-storybook-test-runner) + - [Migration Steps](#migration-steps) + - [Replacing `@storybook/addon-storyshots` with `@storybook/test-runner`:](#replacing-storybookaddon-storyshots-with-storybooktest-runner) + - [Migrating storyshots features](#migrating-storyshots-features) + - [Smoke testing](#smoke-testing) + - [Accessibility testing](#accessibility-testing) + - [Image snapshot testing](#image-snapshot-testing) + - [DOM Snapshot testing](#dom-snapshot-testing) + - [Troubleshooting](#troubleshooting) + - [Handling unexpected failing tests](#handling-unexpected-failing-tests) + - [Snapshot path differences](#snapshot-path-differences) + - [HTML Snapshots Formatting](#html-snapshots-formatting) + - [Provide feedback](#provide-feedback) + +## Pre-requisites + +Before you begin the migration process, ensure that you have: + +- [ ] A working Storybook setup with version 7. +- [ ] Familiarity with your current Storybook and its testing setup. + +> **Note** +> If you are coming from a highly complex storyshots setup, which includes snapshot serializers, tons of mocking, etc. and end up hitting a few bumps in this migration, you might consider checking the [portable stories](./MIGRATION.portable-stories.md) migration. + +## What is the Storybook Test Runner? + +The [Storybook test-runner](https://storybook.js.org/docs/react/writing-tests/test-runner) turns all of your stories into executable tests, powered by [Jest](https://jestjs.io/)and [Playwright](https://playwright.dev/). It's powerful and provides multi-browser testing, and you can achieve many things with it such as smoke testing, DOM snapshot testing, Accessibility testing, Visual Regression testing and more. + +Check [this video](https://www.youtube.com/watch?v%253DwEa6W8uUGSA) for a quick look on the test-runner. + +## Migration Steps + +### Replacing `@storybook/addon-storyshots` with `@storybook/test-runner`: + +First, remove the `@storybook/addon-storyshots` dependency and add the `@storybook/test-runner`: + +```sh +yarn remove @storybook/addon-storyshots +yarn add --save-dev @storybook/test-runner +``` + +Then, update your `package.json` scripts to include a `test-storybook` command: + +```json +{ + "scripts": { + "test-storybook": "test-storybook" + } +} +``` + +Now, run test the setup by running Storybook and the test-runner in separate terminals: + +```sh +# Terminal 1 +yarn storybook +``` + +```sh +# Terminal 2 +yarn test-storybook +``` + +Check the results to ensure that tests are running as expected. + +### Migrating storyshots features + +Storyshots was quite flexible and could be used for different purposes. Below you will find different recipes based on your needs. If you were not using storyshots that extensively, you can benefit from following the recipes and improve your testing experience within Storybook. + +#### Smoke testing + +Storyshots provided a `renderOnly` utility to just render the story and not check the output at all, which is useful as a low-effort way of smoke testing your components and ensure they do not error. + +The test-runner does smoke testing by default, so if you used storyshots with `renderOnly`, you don't have to configure anything extra with the test-runner. The test-runner will also assert the [play function](https://storybook.js.org/docs/react/writing-stories/play-function) of your stories, providing you a better experience and more confidence. + +#### Accessibility testing + +If you used [`@storybook/addon-storyshots-puppeteer`](https://storybook.js.org/addons/@storybook/addon-storyshots-puppeteer)'s `axeTest` utility to test the accessibility of your components, you can use the following recipe to achieve a similar experience with the test-runner: https://github.com/storybookjs/test-runner#accessibility-testing + +#### Image snapshot testing + +If you used [`@storybook/addon-storyshots-puppeteer`](https://storybook.js.org/addons/@storybook/addon-storyshots-puppeteer)'s `imageSnapshot` utility to run visual regression tests of your components, you can use the following recipe to achieve a similar experience with the test-runner: https://github.com/storybookjs/test-runner#image-snapshot + +#### DOM Snapshot testing + +If you used storyshots default functionality for DOM snapshot testing, you can use the following recipe to achieve a similar experience with the test-runner: https://github.com/storybookjs/test-runner#dom-snapshot-html + +### Troubleshooting + +#### Handling unexpected failing tests + +If tests that passed in storyshots fail in the test-runner, it could be because there are uncaught errors in the browser which were not detected correctly in storyshots. The test-runner treats them as failure. If this is the case, use this as an opportunity to review and fix these issues. If these errors are actually intentional (e.g. your story tests an error), then you can tell the test-runner to ignore this particular story instead by defining patterns to ignore via the `testPathIgnorePatterns` configuration. (TODO: Improve this once skipping stories is simpler in the test-runner) + +#### Snapshot path differences + +Snapshot paths and names generated by `@storybook/test-runner` differ from those by `@storybook/addon-storyshots`. You'll need to configure the test-runner to align the naming convention. + +To configure the test-runner, use its `--eject` command: + +```sh +yarn test-storybook --eject +``` + +This command will generate a `test-runner-jest.config.js` file which you can use to configure Jest. +Update the file to use a custom snapshotResolver like so: + +```ts +// ./test-runner-jest.config.js +import { getJestConfig } from '@storybook/test-runner'; + +const defaultConfig = getJestConfig(); + +const config = { + // The default configuration comes from @storybook/test-runner + ...defaultConfig, + snapshotResolver: './snapshot-resolver.js', +}; + +export default config; +``` + +Now create a `snapshot-resolver.js` file to implement a custom snapshot resolver: + +```ts +// ./snapshot-resolver.js +import path from 'path'; + +export default { + resolveSnapshotPath: (testPath) => { + const fileName = path.basename(testPath); + const fileNameWithoutExtension = fileName.replace(/\.[^/.]+$/, ''); + const modifiedFileName = `${fileNameWithoutExtension}.storyshot`; + + // make Jest generate snapshots in a path like __snapshots__/Button.storyshot + return path.join(path.dirname(testPath), '__snapshots__', modifiedFileName); + }, + resolveTestPath: (snapshotFilePath, snapshotExtension) => + path.basename(snapshotFilePath, snapshotExtension), + testPathForConsistencyCheck: 'example.storyshot', +}; +``` + +#### HTML Snapshots Formatting + +The test-runner uses `jest-serializer-html` for HTML snapshots which might have slightly different formatting than your existing snapshots. + +Additionally, you might have elements that contain random or hashed properties which might cause your snapshot tests to fail every time they run. For instance, Emotion class names, or Angular ng attributes. You can circumvent this issue by configuring the test-runner to use a custom snapshot serializer. + +To configure the test-runner, use its `--eject` command: + +```sh +yarn test-storybook --eject +``` + +This command will generate a `test-runner-jest.config.js` file which you can use to configure Jest. +Update the file to use a custom snapshotSerializer like so: + +```ts +// ./test-runner-jest.config.js +import { getJestConfig } from '@storybook/test-runner'; + +const defaultConfig = getJestConfig(); + +const config = { + ...defaultConfig, + snapshotSerializers: [ + // use your own serializer to preprocess the HTML before it's passed onto the test-runner + './snapshot-serializer.js', + ...defaultConfig.snapshotSerializers, + ], +}; + +export default config; +``` + +Now create a `snapshot-serializer.js` file to implement a custom snapshot serializer: + +```tsx +// ./snapshot-serializer.js +const jestSerializerHtml = require('jest-serializer-html'); // available as dependency of test-runner + +const DYNAMIC_ID_PATTERN = /"react-aria-\d+(\.\d+)?"/g; + +module.exports = { + // this will be called once expect(SomeHTMLElement).toMatchSnapshot() is called from the test-runner + serialize(val) { + // from + // to + const withFixedIds = val.replace(DYNAMIC_ID_PATTERN, 'mocked_id'); + return jestSerializerHtml.print(withFixedIds); + }, + test(val) { + return jestSerializerHtml.test(val); + }, +}; +``` + +### Provide feedback + +We are looking for feedback on your experience, and would really appreciate if you filled [this form](some-google-form-here) to help us shape our tooling in the right direction. Thank you so much!