Added document detailing org roadmap to Core 1.0.0 #159

nathan-weinberg · 2024-11-19T20:20:44Z

Resolves #157

nathan-weinberg · 2024-11-19T20:22:49Z

@RobotSail I didn't include the UI piece in this doc since it's only for the CLI, but let me know if you wanna talk that out or have me mention something in the doc somewhere about a UI anyway, maybe under the Q&A

RobotSail · 2024-11-19T20:26:54Z

@nathan-weinberg Ah gotcha, we can find a better place for it.

cdoern

just two small comments, I like the very general approach here.

docs/instructlab-cli-1.0.0.md

cdoern · 2024-11-19T20:32:34Z

docs/instructlab-cli-1.0.0.md

+
+**Q. What about the libraries? Will they 1.0.0 as well?**
+
+A. It depends - we historically have not aligned the CLI and Library releases on a particular version numbering scheme, apart from matching Y-Streams to Y-Streams (e.g., InstructLab CLI 0.20 used SDG 0.4, Training 0.5, and Eval 0.3). At this stage, this document scopes only the prereqs we want for the CLI.


I think the libraries will inherently need a large version bump as we standardize around a REST API

good thought! I will let others weigh-in before changing (particular folks from the Library Maintainer teams) but I'm definitely amicable to this

We'll need to plan this but I agree, the libraries will have to catch up to the backgroundable/RESTy version of ilab.

I don't think the CLI would be able to meet any of its backwards compatibility goals stated here unless the libraries also committed to those goals? This is perhaps a more general issue outside of the scope of just a CLI 1.0.0 as to what level of maturity and backwards compatibility of our Python APIs we want to commit to across repos in the org, as that ultimately determines what level of backwards compatibility consumers of those APIs (like CLI) can commit to.

also a good point @bbrowning - what do you think might be a reasonable expectation, both for ourselves and the community?

Barring any input, this will go in as-is on Friday - can update it in a followup

JamesKunstle · 2024-11-20T00:01:31Z

docs/instructlab-cli-1.0.0.md

+- An official v1 REST API schema
+- Integration of the InstructLab CLI with RAG
+- An upgrade path to subsequent Y-Streams and an eventual 2.0
+- Backwards compatibility across the 1.y stream


IMO this is a really big commitment that we don't necessarily have to make. It's common for consumers of python projects to expect minimal backward compat and to take the most recent version as that one with fewest feature errors.

Would a compromise be to commit to backward compatibility across all 1.0.y versions? I.e., that API breaking changes will not happen on third digit point releases.

Good point, the vagueness of this makes it quite the lift - maybe we can commit to backwards compatibility for N-1 Y-Streams?

E.g., 1.4 is backwards-compatible with 1.3 but not necessarily 1.2

Or do you think even that might be more than worth committing to?

N-1-Y sounds complicated to manage to me because breaking changes need to be staged out across multiple releases.

Def good with backwards compatibility across Z-Streams only - @JamesKunstle wdyt?

Barring any input, this will go in as-is on Friday - can update it in a followup

docs/instructlab-cli-1.0.0.md

jwm4

There seem to be a few open issues here especially around naming and backward compatibility.

docs/instructlab-cli-1.0.0.md

anastasds · 2024-11-21T19:37:40Z

Thanks for putting this together!

I suggest considering architectural goals even if they are implied by things already listed, e.g. the REST API work will probably drive modularization. Do we want to stabilize our systems architecture before declaring a 1.0?

What about API guarantees? State of CI? Documentation? Testing? I think it can definitely be reasonable for these things to be out of scope for this doc - but if so, I think it would be prudent to say so and at least hint at how and where those will end up being addressed.

nathan-weinberg · 2024-11-21T19:51:28Z

Thanks for putting this together!

I suggest considering architectural goals even if they are implied by things already listed, e.g. the REST API work will probably drive modularization. Do we want to stabilize our systems architecture before declaring a 1.0?

What about API guarantees? State of CI? Documentation? Testing? I think it can definitely be reasonable for these things to be out of scope for this doc - but if so, I think it would be prudent to say so and at least hint at how and where those will end up being addressed.

Of course! I definitely agree with the sentiment - I have some concrete items under the MVP for an InstructLab CLI 1.0.0 - I will make some additions there and we can further refine them

anastasds · 2024-11-22T04:02:02Z

I'm happy to help discuss architectural goals if we want to scope those into this document.

I would also point out that what you've put together here is in the direction of defining a technical vision to guide us for somewhere between 6 and 12 months probably. More would need to be done to make it that, but this a first step on the way there. I think it would be appropriate to spend some time on this the next time the org gets together in a planning meeting or some such - it might deserve its own one-off meeting.

Next steps - which would not be in the scope of this document - would be to work backwards from whatever goals are defined here to identify milestones etc with buy-in from the whole org.

docs/instructlab-cli-1.0.0.md

nathan-weinberg · 2024-12-03T15:29:12Z

Possible new names for "CLI" (note "CLI" will continued to be used for specific CLI discussions, just not the instructlab/instructlab repo at large)

Core (react ❤️)
Engine (react 👍)
Orchestrator (react 🎉)

Feel free to respond with a different suggestion as well!

Signed-off-by: Nathan Weinberg <nweinber@redhat.com>

nathan-weinberg · 2024-12-06T18:38:37Z

Possible new names for "CLI" (note "CLI" will continued to be used for specific CLI discussions, just not the instructlab/instructlab repo at large)
* Core (react ❤️)

* Engine (react 👍)

* Orchestrator (react 🎉)
Feel free to respond with a different suggestion as well!

At this time, voting has closed with Core being the winner

nathan-weinberg · 2024-12-06T21:27:42Z

As previously communicated, I am going to merge this

Any desired changes may be proposed as follow-ups

cdoern self-requested a review November 19, 2024 20:27

cdoern reviewed Nov 19, 2024

View reviewed changes

JamesKunstle reviewed Nov 20, 2024

View reviewed changes

docs/instructlab-cli-1.0.0.md Outdated Show resolved Hide resolved

JamesKunstle reviewed Nov 20, 2024

View reviewed changes

docs/instructlab-cli-1.0.0.md Show resolved Hide resolved

JamesKunstle reviewed Nov 20, 2024

View reviewed changes

docs/instructlab-cli-1.0.0.md Show resolved Hide resolved

jwm4 reviewed Nov 20, 2024

View reviewed changes

docs/instructlab-cli-1.0.0.md Outdated Show resolved Hide resolved

jwm4 suggested changes Nov 20, 2024

View reviewed changes

junaruga reviewed Nov 20, 2024

View reviewed changes

docs/instructlab-cli-1.0.0.md Outdated Show resolved Hide resolved

jwm4 mentioned this pull request Nov 21, 2024

Refactor preprocessing and postprocessing in SDG #155

Open

nathan-weinberg force-pushed the cli-1.0 branch 2 times, most recently from 86624bb to aead764 Compare November 21, 2024 21:10

mairin reviewed Nov 22, 2024

View reviewed changes

nathan-weinberg force-pushed the cli-1.0 branch 2 times, most recently from 8d5ad47 to 468b1f4 Compare November 22, 2024 20:53

nathan-weinberg changed the title ~~Added document detailing org roadmap to CLI 1.0.0~~ Added document detailing org roadmap to Core 1.0.0 Dec 6, 2024

Added document detailing org roadmap to CLI 1.0.0

e7f3506

Signed-off-by: Nathan Weinberg <nweinber@redhat.com>

nathan-weinberg force-pushed the cli-1.0 branch from 468b1f4 to e7f3506 Compare December 6, 2024 17:55

nathan-weinberg merged commit 1102817 into instructlab:main Dec 6, 2024
4 checks passed

nathan-weinberg deleted the cli-1.0 branch December 6, 2024 21:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added document detailing org roadmap to Core 1.0.0 #159

Added document detailing org roadmap to Core 1.0.0 #159

nathan-weinberg commented Nov 19, 2024

nathan-weinberg commented Nov 19, 2024

RobotSail commented Nov 19, 2024

cdoern left a comment

cdoern Nov 19, 2024

nathan-weinberg Nov 19, 2024

JamesKunstle Nov 20, 2024

bbrowning Nov 20, 2024

nathan-weinberg Nov 20, 2024

nathan-weinberg Dec 4, 2024

JamesKunstle Nov 20, 2024

jwm4 Nov 20, 2024

nathan-weinberg Nov 20, 2024

jwm4 Nov 20, 2024

nathan-weinberg Nov 21, 2024

nathan-weinberg Dec 4, 2024

jwm4 left a comment

anastasds commented Nov 21, 2024 •

edited

Loading

nathan-weinberg commented Nov 21, 2024

anastasds commented Nov 22, 2024

nathan-weinberg commented Dec 3, 2024

nathan-weinberg commented Dec 6, 2024

nathan-weinberg commented Dec 6, 2024


		Q. What about the libraries? Will they 1.0.0 as well?

		A. It depends - we historically have not aligned the CLI and Library releases on a particular version numbering scheme, apart from matching Y-Streams to Y-Streams (e.g., InstructLab CLI 0.20 used SDG 0.4, Training 0.5, and Eval 0.3). At this stage, this document scopes only the prereqs we want for the CLI.

Added document detailing org roadmap to Core 1.0.0 #159

Added document detailing org roadmap to Core 1.0.0 #159

Conversation

nathan-weinberg commented Nov 19, 2024

nathan-weinberg commented Nov 19, 2024

RobotSail commented Nov 19, 2024

cdoern left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jwm4 left a comment

Choose a reason for hiding this comment

anastasds commented Nov 21, 2024 • edited Loading

nathan-weinberg commented Nov 21, 2024

anastasds commented Nov 22, 2024

nathan-weinberg commented Dec 3, 2024

nathan-weinberg commented Dec 6, 2024

nathan-weinberg commented Dec 6, 2024

anastasds commented Nov 21, 2024 •

edited

Loading