Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compilation to NodeJS module #111

Open
wants to merge 49 commits into
base: main
Choose a base branch
from

Conversation

smiasojed
Copy link
Collaborator

@smiasojed smiasojed commented Nov 6, 2024

Cross-compile the resolc binary to WebAssembly (WASM) using Emscripten.

@athei
Copy link
Member

athei commented Nov 8, 2024

I guess it was an accident that you checked in the build outputs?

Copy link
Member

@athei athei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think after this initial PR you should look into cross language LTO to bring the code size down (for web). As of right now the release profile isn't even using LTO for the rust code. Adding wasm-opt to the pipeline will also help a lot.

js/resolc.wasm Outdated Show resolved Hide resolved
.github/workflows/build-revive-wasm.yml Outdated Show resolved Hide resolved
.github/workflows/build-revive-wasm.yml Outdated Show resolved Hide resolved
@smiasojed
Copy link
Collaborator Author

I guess it was an accident that you checked in the build outputs?

I did it to share the output with @pgherveou

@smiasojed smiasojed marked this pull request as ready for review November 18, 2024 11:20
branches: ["main"]
pull_request:
branches: ["main"]
workflow_dispatch:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the manual dispatch even though we build for every PR and master commit?

Copy link
Collaborator Author

@smiasojed smiasojed Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it uses 1day retention period, I left it in case someone would like to recreate artefacts.

Comment on lines +41 to +45
# Install Emscripten
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install ${{ env.EMSCRIPTEN_VERSION }}
./emsdk activate ${{ env.EMSCRIPTEN_VERSION }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should check out a specific tag here. Otherwise we build against a different version every time.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are using the version defined in EMSCRIPTEN_VERSION env which is set now to the rev fd61bacaf40131f74987e649a135f1dd559aff60

Comment on lines +76 to +78
path: |
${{ env.REVIVE_WASM_INSTALL_DIR }}/resolc.js
${{ env.REVIVE_WASM_INSTALL_DIR }}/resolc.wasm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those all the files that are needed? What about the hand written javascript files in /js? Or are you planning to do the full npm package release work flow as follow up?

Copy link
Collaborator Author

@smiasojed smiasojed Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add npm package with its workflow in next PR. Now we need these 2 files only.

crates/solidity/Cargo.toml Outdated Show resolved Hide resolved
crates/solidity/Cargo.toml Outdated Show resolved Hide resolved
Copy link
Member

@xermicus xermicus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pulling this over the line! Here's a quick first review and some nitpicks

README.md Outdated

### Cross-compilation to WASM

Cross-compiles the Revive recompiler to WASM to enable it to run in a Node.js or browser environment.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Cross-compiles the Revive recompiler to WASM to enable it to run in a Node.js or browser environment.
Cross-compiles the revive compiler to Wasm for running it in a Node.js or browser environment.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


# Build LLVM, clang
LLVM_SRC_PREFIX=${PWD}/llvm-project
LLVM_SRC_DIR=${LLVM_SRC_PREFIX}/llvm
LLVM_BUILD_DIR=${PWD}/build/llvm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this changed? Before the target build directory was outside the llvm-project source directory

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have aligned it to this what i have in emscripten build:
LLVM_NATIVE=$LLVM_SRC/build-native
LLVM_WASM=$LLVM_SRC/build-wasm
We can reorganise it if you want

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please I think it is a standard practice to keep the build outside the source, i.e. leave it as it is now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -5,17 +5,12 @@ set -euo pipefail
INSTALL_DIR="${PWD}/llvm18.0"
mkdir -p ${INSTALL_DIR}


# Clone LLVM 18 (any revision after commit bd32aaa is supposed to work)
if [ ! -d "llvm-project" ]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should clone into $LLVM_SRC_PREFIX instead of a hardcoded path (if it goes into a dedicated script it should be an argument instead)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

crates/lld-sys/build.rs Outdated Show resolved Hide resolved
crates/lld-sys/build.rs Show resolved Hide resolved
crates/lld-sys/build.rs Show resolved Hide resolved
)
})?;
Ok(output)
pub trait Process {
Copy link
Member

@xermicus xermicus Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to understand why is this a trait? Because everywhere else it seems we decide on the configured target. But the trait makes it theoretically possible to use the worker process in native and vice-versa?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't need to be a trait now. I was considering introducing a Config trait with associated types for the compiler and process, but I haven't decided to add it yet. it should reduce the number of #cfgs.

Copy link
Member

@xermicus xermicus Nov 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay, I think both would work but I think the set of targets we want to support is finite so the feature gates should do fine. No hard opinion on which solution, but can you please make it consistent everywhere?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any issues with having a trait that provides a common interface for both native and worker processes. Would you like me to remove it? Similarly both SolcCompiler and SoljsonCompiler implement the Compiler trait, but in this case it is already used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just trying to understand the use case for the trait? I.e. it seems like we can just have the Wasm version gated behind a feature.

This makes it consistent with the rest of the codebase, where we gate the emscripten specific code behind features.

I guess the trait would be useful if we need to add many more but I can't think of anything else?

If it works and you want to leave it, fine for me. I just noticed this is a bit inconsistent.

emscripten-build-llvm.sh Outdated Show resolved Hide resolved
smiasojed and others added 2 commits November 21, 2024 12:49
Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Copy link
Member

@xermicus xermicus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

clone-llvm.sh Show resolved Hide resolved
Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants