Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e tests for examples #516

Open
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

macintoshhelper
Copy link
Contributor

@macintoshhelper macintoshhelper commented May 10, 2020

This PR introduces a new e2e test folder that runs the code from all examples at once.

To avoid instantiating each folder's npm packages, I'm installing every npm package that is needed for specific examples, and aliasing them with Webpack, e.g. chroma-js, etc. This does currently mean that a postinstall script to install the packages in __tests__/examples is required though.

npm run test:e2e appears to be broken since the TypeScript refactor (or later maybe), maybe non commonjs modules are being used somewhere. Importing yoga-layout-prebuilt crashes skpm-test, but that appears to be using commonjs though.

So for now, I'm creating a monolithic test example that is added to the test pipeline with npm run build --prefix __tests__/examples/, and can be manually run in Sketch GUI (or skpm-build --run), or will error out if there is a build breaking issue.

This should make it easy to test all examples with a single command.

This PR could be used later on to generate pngs from all the examples to do a pixel diff test, which would help a lot with the platform bridge refactor, and help out with alignment with React Native, and act as a benchmark for other React renderers to test implementation 🚀. This is something I noticed was mentioned in the React Primitives RFC (using Happo for cross-platform visual regression testing).

TODO

  • Fix skpm-test and use that instead for the examples test
  • Finish adding the rest of the examples

fixes #363

package.json Outdated
@@ -38,6 +38,7 @@
"test:unit": "jest --config jest.config.js --no-watchman",
"test:ci": "npm run test:unit -- --runInBand",
"test:e2e": "skpm-test",
"test:e2e:examples": "npm run build --prefix __tests__/examples/",
Copy link
Contributor Author

@macintoshhelper macintoshhelper May 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe npm run render:once --prefix ... is better for now? Later on this should probably run with test:e2e, together with the other e2e tests.

@macintoshhelper macintoshhelper changed the title Integration tests for examples e2e tests for examples May 10, 2020
@macintoshhelper
Copy link
Contributor Author

macintoshhelper commented May 25, 2020

@mathieudutour I've got e2e tests working again, and have the e2e example tests working now too 🙂.

Took a while to debug it: I found that importing yoga-layout off this branch was crashing Sketch, but the same version of yoga-layout wasn't crashing in a commit snapshot before the TypeScript refactor, so I downgraded @skpm/test-runner to ^0.3 and that fixed it, so should probably open an issue over at https://github.com/skpm/skpm – I'm not really sure how to debug to get enough info, reproduce steps are npm run test:e2e off master pretty much.

There's a bit of clean up to do before a merge, but would be ideal to have a review first. I'm using packages that are installed in __tests__/examples/package.json for now. If a new example is added, then it has to be added manually to the e2e examples test file, together with the dependencies in __tests__/examples/package.json, so maybe this is something that should be mentioned in CONTRIBUTING.md? There should probably be a script to install the skpm/example dependencies too, but not sure on the best way to do this.

I'm still running into #515 (in test runner, and in a normal react-sketchapp project also), which fails the examples test:

 PASS  basic
 PASS  render-context
 PASS  render-in-wrapped-object

 FAIL  examples

   examples  should render examples

     React SketchApp cannot render into this layer. You may be trying to render into a layer that does not take children. Try rendering into a LayerGroup, Artboard, or Page.
      at <anonymous> (../../lib/render.js:23:14)
      at <anonymous> (./examples.test.js:72:2)
      at <anonymous> (../../node_modules/promise-polyfill/lib/index.js:79:0)
      at <anonymous> (../../node_modules/@skpm/timers/timeout.js:18:0)

   should render examples

Patching

src/components/index.ts

import * as Svg from './Svg';

to

import Svg from './Svg';

fixes it and gets the example e2e tests to pass (and const { Path } = Svg; still works).

return sketch.getSelectedDocument() || document;
}

const examplePages = [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a lot fo this is duplicated in main.js. Could easily import it from there

@@ -0,0 +1,14 @@
{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need this file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled it from before the TypeScript refactor. I believe @babel/plugin-transform-modules-commonjs is needed for @skpm/test-runner? @skpm/test-runner@0.4.1 has this plugin, but without "noInterop": true.

Without the .babelrc file, the tests give this error:

undefined is not an object (evaluating 'module.exports.tests = __skpm_tests__')

@@ -0,0 +1,24 @@
const path = require('path');

const aliasedModules = [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we get this list from the package.json? Just to make sure nothing is missing

@@ -80,7 +81,7 @@
},
"devDependencies": {
"@skpm/babel-preset": "^0.2.1",
"@skpm/test-runner": "^0.4.1",
"@skpm/test-runner": "^0.3.2",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hum that's unfortunate. I'll try to see what's up

@@ -30,13 +30,15 @@
"docs:watch": "npm run docs:prepare && gitbook serve",
"docs:publish": "npm run docs:clean && npm run docs:build && cd _book && git init && git commit --allow-empty -m 'update book' && git fetch https://github.com/airbnb/react-sketchapp gh-pages && git checkout -b gh-pages && git add . && git commit -am 'update book' && git push https://github.com/airbnb/react-sketchapp gh-pages --force",
"lint-staged": "lint-staged",
"prepublishOnly": "npm run clean && npm run test:ci && npm run build",
"prepublishOnly": "npm run clean && npm run test:ci && npm run install:e2e && npm run build",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

npm run install:e2e triggers the __tests__/examples postinstall script. Not sure if this should be left in, it'll fail the prepublish script if the examples skpm-build fails, but not sure if it'll work in CI.

...config.resolve,
alias: {
...config.resolve.alias,
'react-sketchapp': path.resolve(__dirname, '../../lib'),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that @skpm/test-runner needs the commonjs build, and doesn't work with the ES build, strange.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automate e2e tests for all examples
2 participants