feat: Adding self-extracting file as an optional feature

[prsabahrami]

Motivation

Closes #4

Changes

Similar to conda-constructor, I have added a header.sh file. After the environment has been packed and archived, that archive is appended onto header.sh which later extracts it when it runs.

Packing an environment

pixi-pack pack --create-executable

Unpacking an environment

either directly run the .sh file created

sh environment.sh

You can use below flags

Unpacks an environment packed with pixi-pack

-f           no error if environment already exists
-h           print this help message and exit
-p ENV       environment prefix, defaults to $PREFIX
-i INSTALLER create the environment using the specified installer defaulting to $INSTALLER
-a           create an activation script to activate the environment

or you can directly use

pixi-pack unpack environment.sh

I will try to add better tests and error handling in the upcoming days. Meanwhile, let me know if you have any recommendations.

[0xbe7a]

One of the biggest use-cases for us for pixi-pack is the ability to build up archives on Linux for foreign targets, like Windows and to extract it there. Having a self-extracting archive there would be probably the most useful scenario. Your prepended shell script will only work for Unix systems, right?

Also after taking a (brief) look it appears that the extraction script is expecting a existing conda or micromamba executable on PATH. I think in these scenarios it's already an attractive option to manually extract the archive and create the environment using the included lockfiles. I think it would be very useful, if we could prepend a standalone extraction tool, that does not require any system tools to be available on the system.

[prsabahrami]

@0xbe7a
Yes, it is correct.
I was thinking I could add a bat self-extracting script to be used on Windows in a separate PR. Then through the passed --platform we can decide to generate sh script or bat script (or both).
Please let me know if this works for the mentioned scenario.

Also, in regards to the extraction tool used in the script, fair enough.
I was thinking to implement the logic used in unpack.rs onto the script(s) so we could get rid of conda and micromambda. Let me know what you think. I'm not sure how efficient the script will turn out.

[0xbe7a]

The part that involves extracting and linking the packages is highly non-trivial and probably not really feasible with a bash script. Instead, we should try using rattler as we did in pixi-pack itself. rattler supports all the platforms we target. The approach I am considering involves including an executable object with the required unpack functionality and using this when unpacking the archive to make it truly standalone.

Additionally, upon reviewing your pull request again, please note that micromamba shell activate already exists and can create the entire activation script for micromamba. For unpacking the archive using micromamba, you can use micromamba to create the environment and then generate the activation script to get the final prefix. This way, a single shell script can be used to activate into this environment, similar to what pixi-pack unpack would have produced.

[prsabahrami]

Sounds good.
I will include an executable using rattler onto the script. Which the script will use to properly extract the archive.

Also, in regards to the micromamba shell activate, I will try to use rattler_shell inside the executable to create the activation script so we can remove the necessitiy of conda or micromambda.

[prsabahrami]

Apologies for the delay. Got busy with school work.
I have added an extractor subproject which then is built and its binary is added to script.
The original commands still work but in case the -i conda is passed to the script, one can not create an activation script since I'm using micromamba or rattler to create the script. If needed, I can modify header.sh to extract extractor binary at all times but I thought if someone has conda installed on their devices, there is no need to take extra time and extract the binary.
Let me know what you think and then I can improve the verbosity and testing.
Note: The logic used in extractor is just a simpler version of what is in unpack.rs.

[pavelzw]

there is no need to take extra time and extract the binary

I think it would be better for reproducibility if we always unpack the same way, i.e. always use extractor

[pavelzw]

Could you fix the merge conflicts as well?

[prsabahrami]

Could you fix the merge conflicts as well?

Done.

[pavelzw]

Really nice @prsabahrami! Thanks for persevering, looking forward to using this feature 🤩

note: tests are failing because it tries to download pixi-pack 0.3.0 in the create-executable tests which is not yet released