Code

RAG ingestion and chat pipelines#161

Code

Draft

dmartinol wants to merge 23 commits intoinstructlab:maininstructlab/dev-docs:mainfrom dmartinol:rag-embed-and-chatdmartinol/dev-docs:rag-embed-and-chatCopy head branch name to clipboard

+12,971

Commits

Commits on Dec 18, 2024

• initial draft

  Show description for 7ca3ab6
  dmartinol

  committed
  7ca3ab6

• typo

  Show description for f18442a
  dmartinol

  committed
  f18442a

• shortcuts for v1.4 and contextual description

  Show description for 643578d
  dmartinol

  committed
  643578d

• semantic similarity search

  Show description for 997e743
  dmartinol

  committed
  997e743

• added query refinement step

  Show description for 8b457c3
  dmartinol

  committed
  8b457c3

• config examples, defining refinement query

  Show description for 4562d3a
  dmartinol

  committed
  4562d3a

• replaced quotes in json

  Show description for b7d7d6d
  dmartinol

  committed
  b7d7d6d

• updated ingestion diagram

  Show description for 4b827bf
  dmartinol

  committed
  4b827bf

• query refinement only when no good context

  Show description for 9566e27
  dmartinol

  committed
  9566e27

• removed one future option

  Show description for 828ede6
  dmartinol

  committed
  828ede6

• an option to run docs transform and ingestion with an existing command

  Show description for 9bad6d1
  dmartinol

  committed
  9bad6d1

• Reviewed proposal after meeting ilab-runtime team

  Show description for 438dacc
  dmartinol

  committed
  438dacc

• Integrated some comments, others remain in TBD or TODO

  Show description for 6ffdb69
  dmartinol

  committed
  6ffdb69

• option to specify models dir and clarified the use of local models

  Show description for 20d4c7b
  dmartinol

  committed
  20d4c7b

• split process and ingest sub-commands and other comments from latest review meeting

  Show description for c355d09
  dmartinol

  committed
  c355d09

• added User Experiences section

  Show description for 007a742
  dmartinol

  committed
  007a742

• design considerations

  Show description for 21ec857
  dmartinol

  committed
  21ec857

• added taxonomy path

  Show description for 1b18ebb
  dmartinol

  committed
  1b18ebb

• removed unneeded image

  Show description for a975407
  dmartinol

  committed
  a975407

• reviewed images

  Show description for 30d4c98
  dmartinol

  committed
  30d4c98

• specified the details of model training path. fixed command option names

  Show description for f292b33
  dmartinol

  committed
  f292b33

• moved md to rag folder (with images). added export from excalidraw

  Show description for 440ad39
  dmartinol

  committed
  440ad39

Commits on Dec 20, 2024

• Replacing command args with command flags

  Show description for 3d7757f
  dmartinol

  committed
  3d7757f