Skip to content

Latest commit

 

History

History
261 lines (168 loc) · 7.17 KB

README.md

File metadata and controls

261 lines (168 loc) · 7.17 KB
Raw

Morrigan

OpenAI/ChatGPT Backend with conversation and API

Features

  • Import documents of knowledge base from a table (CSV), save them into PostgreSQL
  • Based on the title and content of the document, generate and fill in QA items in the document using some APIs
  • Summarize the questions and generate corresponding vectors
  • Implement high-quality Q&A using vector search
  • Welcome message and preset messages
  • Chat History for Conversation (based on redis)
  • RESTful API
  • Support text/event-stream
  • Login with OAuth2 client for general Security Provider

Supported Frontend

chatgpt-svelte based on Svelte ⤸

chatgpt-svelte

forked: https://github.com/liut/chatgpt-svelte

chatgpt-web based on Vue.js ⤸

chatgpt-web

forked: https://github.com/liut/chatgpt-web

APIs

Get welcome message and new conversation ID

GET /api/welcome
Parameters

None

Responses
http code content-type response
200 application/json {"message": "welcome message", "id": "new-cid"}

Get user information of that has been verified or signed in

GET /api/me
Parameters

None

Responses
http code content-type response
200 application/json {"data": {"avatar": "", "name": "name", "uid": "uid"}}
401 application/json {"error": "", "message": ""}

Post chat prompt and return Streaming messages

POST /api/chat-sse or /api/chat with {stream: true}
Parameters
name type data type description
csid optional string conversation ID
prompt required string message for ask
stream optional bool enable event-stream, force on /api/chat-sse
Responses
http code content-type response
200 text/event-stream {"delta": "message fragments", "id": "conversation ID"}
401 application/json {"status": "Unauthorized", "message": ""}
POST /api/chat-process for chatgpt-web only
Parameters
name type data type description
prompt required string message for ask
options optional object { conversationId: "" }
Responses
http code content-type response
200 application/octet-stream {"delta": "message fragments", "text": "message", "conversationId": ""}
401 application/json {"status": "Unauthorized", "message": ""}

Getting started

test -e .env || cp .env.example .env
# Edit .env and change api key of OpenAI
# Embedding and replacing frontend resources

make deps

forego start

# or

make dist

Prepare preset data file in YAML

welcome:
  content: Hello, I am your virtual assistant. How can I help you?
  role: assistant

messages:
  - content: You are a helpful assistant.
    role: system
  - content: When is my birthday?
    role: user
  - content: How would I know?
    role: assistant

 # more messages

Prepare database

CREATE USER morrigan WITH LOGIN PASSWORD 'mydbusersecret';
CREATE DATABASE morrigan WITH OWNER = morrigan ENCODING = 'UTF8';
GRANT ALL PRIVILEGES ON DATABASE morrigan to morrigan;


\c morrigan

-- optional: install extension from https://github.com/pgvector/pgvector
CREATE EXTENSION vector;

Command line usage


USAGE:
   morrigan [global options] command [command options] [arguments...]

COMMANDS:
   usage, env                   show usage
   initdb                       init database schema
   import                       import documents from a csv
   fill-qa, fillQAs             fill QA in documents
   export-qa, exportQAs         export QA from documents
   embedding, embedding-pormpt  read prompt documents and embedding
   web, run                     run a web server
   help, h                      Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h  show help

Change settings with environment

Show all local settings

go run . usage

Example:

MORRIGAN_OPENAI_API_KEY=oo-xx
MORRIGAN_HTTP_LISTEN=:3002

# optional preset data
MORRIGAN_PRESET_FILE=./data/preset.yaml

# optional OAuth2 login
MORRIGAN_AUTH_REQUIRED=true
OAUTH_PREFIX=https://portal.my-company.xyz

# optional proxy
HTTPS_PROXY=socks5://proxy.my-company.xyz:1081

The operation steps for generating data.

  1. Prepare a CSV file for the corpus document.
  2. Import documents.
  3. Generate Questions and Anwsers from documents with Completion.
  4. Generate Prompts and vector from QAs with Embedding
  5. Done and go to chat

CSV template of documents

title heading content
my company introducion A great company stems from a genius idea. 
go run . initdb
go run . import mycompany.csv
go run . fill-qa
go run . embedding

Attach frontend resources

  1. Go to frontend project directory
  2. Build frontend pages and accompanying static resources.
  3. Copy them into ./htdocs

Example:

cd ../chatgpt-svelte
npm run build
rsync -a --delete dist/* ../morrigan/htdocs/
cd -

During the development and debugging phase, you can still use with proxy to collaborate with the front-end project.