Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package distribution method for RISC-V Sail model #39

Open
13 tasks
jjscheel opened this issue Jan 30, 2024 · 43 comments
Open
13 tasks

Package distribution method for RISC-V Sail model #39

jjscheel opened this issue Jan 30, 2024 · 43 comments
Assignees

Comments

@jjscheel
Copy link
Contributor

Technical Group

Golden Model SIG

ratification-pkg

Technical Debt

Technical Liaison

Bill McSpadden

Task Category

SAIL model

Task Sub Category

  • gcc
  • binutils
  • gdb
  • intrinsics
  • Java
  • KVM
  • ld
  • llvm
  • Linux kernel
  • QEMU
  • Spike

Ratification Target

3Q2023

Statement of Work (SOW)

Component names:
Sail

Requirements:
Currently, the RISC-V Sail model must be built (compiled) by the user. It has been requested that the model be compiled and distributed using industry-standard package management tools.

Deliverables:

  1. A build a distribution method for the Sail model. Must include yum, apt and a package management for Windows and MacOS.
  2. Documentation for the method.

Acceptance Criteria:

  1. Accepted PRs into the Sail repo for all workflow activities
  2. Packages that build, install on the target OSs, and pass the ACT test.

Projected timeframe: (best guess date)
6 person months

SOW Signoffs: (delete those not needed)

  • Development partner sign-off
  • Golden Model SIG sign-off

Waiver

  • Freeze
  • Ratification

Pull Request Details

No response

@lazyparser
Copy link

I am interested in accepting this task, and I think PLCT Lab's technical background can help. Since this week and next week are the Chinese New Year, I estimate that I can return some research results and plans around February 28 at the earliest.

@jjscheel
Copy link
Contributor Author

@lazyparser and PLCT have expressed interest. I've also asked Greg Sterling to engage here too. I'll get him added to the issue as soon as I can.

@U2FsdGVkX1
Copy link

Fedora has already built all the dependencies and tested
please add this repo for testing
https://mirror.iscas.ac.cn/plct/distros/fedora/

@jjscheel
Copy link
Contributor Author

Thanks for the information. @gsterlin, please take a look at this.

@yuzibo
Copy link

yuzibo commented Mar 19, 2024

There are Debian packages about sail/sail-riscv and their dependent packages.
Source code is here.

Meanwhile, ocaml-linenoise, omd and lem has been pushed Debian salsa repo under Debian OCaml team and was waiting to sponsor them. Other packages are doing similar things.

@jrtc27
Copy link

jrtc27 commented Mar 19, 2024

Given how rapidly Sail is being modified at the moment to support new features for the model I’m not convinced having packages in Debian stable is at all useful. It seems highly likely that by the time they are released they won’t be usable.

@yuzibo
Copy link

yuzibo commented Mar 19, 2024

Make sense.

If we want to use a fresh deb/rpm package, we would better to keep one final package, in other words, to keep sail-riscv package with an upstream distribution. But before this, these dependent packages should be packaged into distribution to reduce the maintenance burden.

@jjscheel
Copy link
Contributor Author

@jrtc27, valid point.

@yuzibo, I agree that building packages now helps ensure that dependencies are available when Sail stabilizes.

@gsterlin
Copy link

My belief is that even though we may be iterating over packages faster than we'd like, it would also give us some ability to talk across differing distributions to discuss what versions of software, and specifically which packages were used, that gave specific results. This should help us consider branching strategies for the models, and being able to provide specifications about what went into the model. My hope is that this would give us a more common framework from which we could discuss changes, and test them out in more consistent ways.

It is also possible that I need to spend more time understanding how things work and come together for sail, and I am open to learning more about how things work today, especially where we see issues or possible room for improvement.

@yuzibo
Copy link

yuzibo commented Apr 2, 2024

linksem has been packaged and waited for lem uploaded first.

@yuzibo
Copy link

yuzibo commented Apr 16, 2024

bisect-ppx has been packaged to the Debian OCaml namespace and uploaded by a Debian Developer.

@U2FsdGVkX1
Copy link

U2FsdGVkX1 commented Apr 16, 2024

sail-riscv on Fedora docker image has been built.
by the way, we will upstream dependencies and sail as soon as possible, sorry for the delay

@jjscheel
Copy link
Contributor Author

@gsterlin, would you kindly collect all the progress and summarize this? I suspect that there's benefit in knowing what pkgs are in which ecosystem. Then, we can tie this into the whole broader plan. Thx.

@U2FsdGVkX1
Copy link

https://github.com/orgs/fedora-sail/repositories

The spec files for lem linksem ocaml-omd ocaml-linenoise ocaml-pprint have been pushed
They are dependencies of sail-riscv, but there is another part that is still in process
Now wait for the upstream

@yuzibo
Copy link

yuzibo commented Apr 30, 2024

On Debian side update:
ocaml-linenoise has been uploaded to Debian official archive.

So, until now. bisect-ppx has been uploaded to Debian NEW queue and ocaml-linenoise are already in Debian official. We still have omd, lem and linksem need to be uploaded to Debian for sail/riscv-sail.

@gsterlin
Copy link

Here is a breakdown of the current packages for the distributions discussed in this thread:

Fedora Repository: https://mirror.iscas.ac.cn/plct/distros/fedora/
coq-bbv-1.4-1.fc40.x86_64.rpm
coq-bbv-devel-1.4-1.fc40.x86_64.rpm
coq-sail-0.17.1-1.fc40.x86_64.rpm
coq-sail-devel-0.17.1-1.fc40.x86_64.rpm
linksem-0.8-1.fc40.x86_64.rpm
linksem-devel-0.8-1.fc40.x86_64.rpm
ocaml-lem-2022.12.10-1.fc40.x86_64.rpm
ocaml-lem-devel-2022.12.10-1.fc40.x86_64.rpm
ocaml-linenoise-1.5-1.fc40.x86_64.rpm
ocaml-linenoise-debuginfo-1.5-1.fc40.x86_64.rpm
ocaml-linenoise-debugsource-1.5-1.fc40.x86_64.rpm
ocaml-linenoise-devel-1.5-1.fc40.x86_64.rpm
ocaml-omd-1.3.2-1.fc40.x86_64.rpm
ocaml-omd-debuginfo-1.3.2-1.fc40.x86_64.rpm
ocaml-omd-debugsource-1.3.2-1.fc40.x86_64.rpm
ocaml-omd-devel-1.3.2-1.fc40.x86_64.rpm
ocaml-pprint-20230830-1.fc40.x86_64.rpm
ocaml-pprint-debuginfo-20230830-1.fc40.x86_64.rpm
ocaml-pprint-debugsource-20230830-1.fc40.x86_64.rpm
ocaml-pprint-devel-20230830-1.fc40.x86_64.rpm
ott-0.33-1.fc40.x86_64.rpm
ott-devel-0.33-1.fc40.x86_64.rpm
sail-0.17.1-1.fc40.x86_64.rpm
sail-debuginfo-0.17.1-1.fc40.x86_64.rpm
sail-debugsource-0.17.1-1.fc40.x86_64.rpm
sail-devel-0.17.1-1.fc40.x86_64.rpm
sail-riscv-debuginfo-master-1.fc40.x86_64.rpm
sail-riscv-debugsource-master-1.fc40.x86_64.rpm
sail-riscv-devel-master-1.fc40.x86_64.rpm
sail-riscv-master-1.fc40.x86_64.rpm

Debian Repository: https://mirror.iscas.ac.cn/plct/distros/debian/
bisect-ppx-dbgsym_2.8.3-1_amd64.deb
bisect-ppx_2.8.3-1_amd64.deb
libbisect-ppx-dev-dbgsym_2.8.3-1_amd64.deb
libbisect-ppx-dev_2.8.3-1_amd64.deb
liblem-dbgsym_2022.12.10-1_amd64.deb
liblem-dev_2022.12.10-1_amd64.deb
liblem_2022.12.10-1_amd64.deb
liblinksem-ocaml-dev_0.8-1_amd64.deb
liblinenoise-ocaml-dbgsym_1.5-1_amd64.deb
liblinenoise-ocaml-dev-doc_1.5-1_amd64.deb
liblinenoise-ocaml-dev_1.5-1_amd64.deb
liblinenoise-ocaml_1.5-1_amd64.deb
libomd-ocaml-dev-dbgsym_1.3.2-1_amd64.deb
libomd-ocaml-dev_1.3.2-1_amd64.deb
omd-dbgsym_1.3.2-1_amd64.deb
omd-doc_1.3.2-1_amd64.deb
omd_1.3.2-1_amd64.deb
ott-dbgsym_0.33-1_amd64.deb
ott_0.33-1_amd64.deb
libsail-c-backend-ocaml-dev_0.17.1-1_amd64.deb
libsail-coq-backend-ocaml-dev_0.17.1-1_amd64.deb
libsail-doc-backend-ocaml-dev_0.17.1-1_amd64.deb
libsail-latex-backend-ocaml-dev_0.17.1-1_amd64.deb
libsail-lem-backend-ocaml-dev_0.17.1-1_amd64.deb
libsail-manifest-ocaml-dev-dbgsym_0.17.1-1_amd64.deb
libsail-manifest-ocaml-dev_0.17.1-1_amd64.deb
libsail-ocaml-backend-ocaml-dev_0.17.1-1_amd64.deb
libsail-ocaml-dev-dbgsym_0.17.1-1_amd64.deb
libsail-ocaml-dev_0.17.1-1_amd64.deb
libsail-output-ocaml-dev_0.17.1-1_amd64.deb
libsail-smt-backend-ocaml-dev_0.17.1-1_amd64.deb
libsail-sv-backend-ocaml-dev_0.17.1-1_amd64.deb
sail-ocaml-dbgsym_0.17.1-1_amd64.deb
sail-ocaml_0.17.1-1_amd64.deb
sail-riscv-dbgsym_0.5+git20240226.9602e3a-1_amd64.deb
sail-riscv_0.5+git20240226.9602e3a-1_amd64.deb

@yuzibo
Copy link

yuzibo commented May 14, 2024

Debian: Still waiting for omd to be reviewed/uploaded. No progress on the other packages now.

@U2FsdGVkX1
Copy link

U2FsdGVkX1 commented May 14, 2024

Fedora: all dependencies have been pushed, and is waiting for review.

@gsterlin
Copy link

I have been working on a tool with Bill M. to help compare environments. The intent is to report on/verify if an environment has what is believed to be the necessary settings and software in place to function properly for a given repository. I have catalogued roughly 10 environment variables that indicate if an environment has been setup properly, and roughly 10 versions of software or modules that appear to be necessary to build sail-riscv.

I am working to wrap up the scripting over the next week or so, and removing hard coded values and transforming them into a json file so that we can make adjustments for different repositories and branches as necessary.

Over the next week or two I believe I will have the code in a state worth reviewing, and will then begin working with Bill M. to determine where best to begin working with teams for this kind of environment validation.

@yuzibo
Copy link

yuzibo commented May 28, 2024

@gsterlin thanks for your work, please tell me if I can help here.

Update on Debian side: omd was sponsored by my sponsor.:)

@U2FsdGVkX1
Copy link

U2FsdGVkX1 commented May 28, 2024

Sorry for the late reply
Fedora: No updates yet, still under review

@yuzibo
Copy link

yuzibo commented Jun 11, 2024

Update on Debian side: omd was uploaded to Debian archives and lem was waiting to be reviewed.

@U2FsdGVkX1
Copy link

@jjscheel
Copy link
Contributor Author

@gsterlin, perhaps when you run the June 25 meeting for me you could talk more about your plans here. Thoughts?

@yuzibo
Copy link

yuzibo commented Jun 25, 2024

Debian: lem was reviewing for the past two weeks.

@U2FsdGVkX1
Copy link

Fedora package still waiting for review……

@yuzibo
Copy link

yuzibo commented Jul 9, 2024

Debian: linksem was uploaded to salsa under the Debian OCaml team to wait to be reviewed. lem was still reviewing.

@U2FsdGVkX1
Copy link

ocaml-omd ocaml-linenoise ocaml-pprint
It has been reviewed. I have pushed the new version and waiting for another review.

@yuzibo
Copy link

yuzibo commented Jul 23, 2024

Debian: lem has been uploaded to FTP NEW queue.

@U2FsdGVkX1
Copy link

ocaml-pprint has been reviewed and approved
ocaml-linenoise has some issues and cannot meet Fedora packaging rules. I have already pull request to the upstream
ocaml-omd is awaiting further review

@jjscheel
Copy link
Contributor Author

@gsterlin, would you mind sharing your progress for the next meeting on August 6?

@gsterlin
Copy link

Absolutely! A quick note is that I have been working with Carl to leverage some tools (asdf/mise, and direnv) to do the environment variable/configuration and tool version management. This replaces the code and configuration files I had written, greatly simplifying the problem. We have made a lot of progress over the last two weeks and will have some content to share shortly that will help new comers work with our environments, and provide the building blocks for enabling discussions about which versions of packages or environment variables are necessary for proper environments per repository.

@yuzibo
Copy link

yuzibo commented Aug 6, 2024

Debian packaging: FTP master has commented on lem and waiting for review again.

@U2FsdGVkX1
Copy link

ocaml-omd was also approved!
Other packages waiting for upstream response

@yuzibo
Copy link

yuzibo commented Aug 20, 2024

Debian: lem was rejected due to d/copyright issue and uploaded to NEW waiting for review again.

@yuzibo
Copy link

yuzibo commented Sep 3, 2024

Debian packaging: no update in the past two weeks, still waiting for lem to be approved to upload.

@U2FsdGVkX1
Copy link

fedora: No updates in the past two weeks
No response to pull requests for ocaml-linenoise, maybe we should consider other methods…

@jjscheel
Copy link
Contributor Author

jjscheel commented Sep 3, 2024

@gsterlin, I wonder if Wei Fu can provide some guidance on the ocaml-linenoise PR.

@U2FsdGVkX1
Copy link

Wei Fu is my mentor, but I think we should contact the maintainer of ocaml-linenoise (or Fedora packager)

@U2FsdGVkX1
Copy link

Sorry, my English may cause some misunderstand, let me clarify
"consider other methods" means that ocaml-linenoise has made some changes, and these changes have not been upstreamed to linenoise, so they behave inconsistently, and the fedora's reviewer hopes to be able to upstream these changes, so "other methods" should be
Contact the maintainer of linenoise to accept this PR
or
Explain to the fedora's reviewer why there are differences between ocaml-linenoise and linenoise, and then accept this package

@yuzibo
Copy link

yuzibo commented Sep 17, 2024

Debian packaging: lem was uploaded to Debian official archives already. linksem and sail is packaging again.

@U2FsdGVkX1
Copy link

Since there has been no response from the upstream of linenoise, I sent the patch to the downstream of Fedora, but I think Fedora may not accept this patch (because Fedora requires the upstream to accept the patch first).
However, without this patch, linenoise and ocaml-linenoise will behave differently, which will cause ocaml-linenoise to not meet the packaging requirements :(

@U2FsdGVkX1
Copy link

Good news! Fedora, as the downstream, has already merged this change!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: As-planned
Development

No branches or pull requests

7 participants