feat(Template): Introduce template for react extensions

[Pive01]

Motivation

To allow a better integration between extensions and other LocalStack related products we can allow the creations of an extensions with e pre-built react app inside it that reuses the theme of our web app. This will allow a matching theme and easy access for developers to use react with it.

Content

• Shifted the template into 2 folders, the first for the classic extension without UI and the second for the react one. To allow the cli to also work as expected I've raised another PR here
• The extension itself work like so:
  • Divided into fronted and backend folders: the first one contains the react code while the second contains the routes definitions
  • The react app will be built before packaging the extension and will be included in the pypi module such that the total weight will be fairly low (base is around 0.5MB) and will not include all the node modules. It uses esbuild with some other packages to handle assets.
  • The routing will assume that your extension is hosted following the standards with subdomains and submounts. The default route "/" will always fall back to the index.html while by default everything else will serve files from the build folder. You will still be able to add /hello handler and respond with what you want. Since for routing we use react-router-dom an url could be valid and invalid at the same time: once you land on the "/" page and the extension responds with the index.html then will be react itself to handle routing and not the extension, which means that navigating from the ui to pages such as /one and /dashboard will work because handled by react route, but going directly into them with a link will result in LocalStack try to resolve them and give you the file called "one" in your build folder (or if you have defined a handler for that route it will give you that).

Other

Before merging this:

• Merge this PR -> publish package on npm -> update package.json with new link to dependency

After merging this:

• Merge this PR to allow template to be generated with cli

On the next major release we could also restructure both templates into a single folder to keep the project more clean but doing it now would break the localstack extensions dev new command as it would be a breaking change for older cli versions

[thrau]

Congrats Luca, this is a huge step forward and I really look forward to using this!

There are a couple of minor fixes needed, and I'd like to discuss packaging and code organization:

packaging issue

the project is not configured to produce a correct package. when i run make dist and then extract the tar.gz file (source distribution), i see

 % tree dist/my_localstack_extension-0.1.0
dist/my_localstack_extension-0.1.0
├── my_localstack_extension
│   ├── extension.py
│   ├── frontend
│   │   ├── build
│   │   │   └── __init__.py
│   │   └── __init__.py
│   └── __init__.py
├── my_localstack_extension.egg-info
│   ├── dependency_links.txt
│   ├── entry_points.txt
│   ├── not-zip-safe
│   ├── PKG-INFO
│   ├── requires.txt
│   ├── SOURCES.txt
│   └── top_level.txt
├── PKG-INFO
├── README.md
├── setup.cfg
└── setup.py

Notice that it doesn't contain the built javascript sources.

code organization

Overall I wonder whether organizing the frontend code into the python module like this is the best option. currently we are "nesting" language projects into each other. there's the "outer" python project (indicated by the setup.cfg and my_localstack_extension module), and then the package.json which is nested inside the module.
i haven't thought it through, but maybe it makes sense to have a frontend or react folder next to the my_localstack_extension python module. then, during packaging, include the built files into the python distribution?
we would place the package.json into the root folder next to the setup.cfg, and during make dist we'd first build the minified json with dependencies and include it into the dist.

I don't have the perfect answer here, it needs some exploring. Packaging requirements:

• any dependencies need to be served by the backend (i assume?) as static files
• needs to work with EXTENSION_DEV_MODE=1
• should not include the source/project config files in the distribution after running make dist

[thrau]

here we should also parameterize from the cookiecutter parameters. perhaps {{ cookiecutter.project_name }} ?

[Pive01]

Yes, pardon 🙏

[Pive01]

Overall I wonder whether organizing the frontend code into the python module like this is the best option. currently we are "nesting" language projects into each other. there's the "outer" python project (indicated by the setup.cfg and my_localstack_extension module), and then the package.json which is nested inside the module.
i haven't thought it through, but maybe it makes sense to have a frontend or react folder next to the my_localstack_extension python module. then, during packaging, include the built files into the python distribution?
we would place the package.json into the root folder next to the setup.cfg, and during make dist we'd first build the minified json with dependencies and include it into the dist.

On one way I agree that having 2 languages intertwisted is a bit ugly; on the other way though your proposal doesn't convince me much either:
Having the package.json side to side with the setup.cfg seems wrong to me as setup.cfg defines the extension while package.json defined the react part of the extension...We could then move the /frontend folder as a whole side to side with the my_localstack_extension module....but I'm not too convinced about this either 😅 as the extension serves
the react app as part of it;
In the end is the same concept you applied with the jinja templates having them inside the module just that now is a whole react app to be built and served and not just the templates (even though in the end is just what is in the build folder to be served but details)

    my_extension
    ├── extension.py
    ├── __init__.py
    ├── static              <-- make sure static resources get packaged!
    │   ├── __init__.py
    │   ├── favicon.ico
    │   └── style.css
    └── templates            <-- jinja2 templates
        └── index.html

This aside I've addressed the other comment, thanks btw for the review @thrau 😄

[dominikschubert]

Great work so far! I have a few suggestions for changes for the next review iteration.

I'd also suggest adding a default .gitignore file that ignores (not exhaustive)

• build directories (dist, build, egg-info, ....)
• .venv
• frontend/.yarn
• frontend/node_modules
• ...

As discussed we're having some issues now due to the nature of how the extension dev mode works, which at the moment is incompatible with a src layout.
I was working on a workaround for this but we'll have to check if it's not better to potentially keep the flat layout instead

[dominikschubert]

I generally think we should start using pyproject.toml.

But maybe @thrau has more context on why we might still need the setup.cfg & setup.py?

[dominikschubert]

Talked with Thomas about this. The context here is that it's mostly legacy and he approves of moving toward unifying this via a pyproject.toml 👍

@Pive01 are you fine with me just pushing those changes directly?

[Pive01]

Sure!

[dominikschubert]

same comments apply here of course as with the react template below

[dominikschubert]

same arguments as below in the react template

[dominikschubert]

@Pive01 I've pushed the changes necessary for a single pyproject.toml that replaces both setup.cfg and setup.py.
I suggest we'll add some linting/formatting in a follow-up as well, just to make sure we stay fairly homogenous with the extensions there as well 👍

One thing I'm not sure yet about is if we should really use python -m plux entrypoints in the install-backend target. There's probably no need for this in an extension since we manually specify the entrypoint in the pyproject.toml file. We could remove either one of those and it should still work.

[thrau]

Changes look really good! This is a super clear and explicit packaging. much easier to understand and work with. kudos both for this iteration!

my only remaining gripe is that i cannot use it out of the box with my node setup. my node installation comes with yarn 1.22.22, and when i run make install, i get:

cd frontend && yarn install
error This project's package.json defines "packageManager": "yarn@3.2.3". However the current global version of Yarn is 1.22.21.

Presence of the "packageManager" field indicates that the project is meant to be used with Corepack, a tool included by default with all official Node.js distributions starting from 16.9 and 14.19.
Corepack must currently be enabled by running corepack enable in your terminal. For more information, check out https://yarnpkg.com/corepack.
make: *** [Makefile:30: install-frontend] Error 1

then, when i call yarn set version 3.2.3 in the repository, and run make install again, i get:

cd frontend && yarn install
Usage Error: The nearest package directory (/tmp/foo/my-ui-extens/frontend) doesn't seem to be part of the project declared in /tmp/foo/my-ui-extens.

- If /tmp/foo/my-ui-extens isn't intended to be a project, remove any yarn.lock and/or package.json file there.
- If /tmp/foo/my-ui-extens is intended to be a project, it might be that you forgot to list frontend in its workspace configuration.
- Finally, if /tmp/foo/my-ui-extens is fine and you intend frontend to be treated as a completely separate project (not even a workspace), create an empty yarn.lock file in it.

$ yarn install [--json] [--immutable] [--immutable-cache] [--check-cache] [--inline-builds] [--mode #0]
make: *** [Makefile:30: install-frontend] Error 1

It would be great if we could either make the config compatible with these environments, or add instructions for default node installations.