If you want to try PokemonBot without too many hassles, just follow these simple steps:
git
toolsetconda
docker
anddocker compose
- Discord and a Discord account
- Cohere API key (get yours here): you should keep this under
cohere_api_key
in your.env
file.
How do you create a Discord bot? It is really simple!
- Go to Discord and create an account (or log into yours, if you already have one)
- Create a new server by clicking on "Add a server" (a big green "+" button) and name it as you want
- Go to Discord developers portal and click on "New application"
- Name your application, than save the changes and click on "Bot", on the left
- There you will get the chance to name your bot and copy its token: paste it under the
discord_bot
field in your.env
file. - After that, go on OAuth2 > URL generator and generate a URL that you will paste in your navigation bar, to add the bot to your newly created server.
- In the
channel_id
field in your.env
file, insert the last number that is displayed in your server's #general channel url when you open it on the web.
- Clone this GitHub repository
git clone https://github.com/AstraBert/Pokemon-Bot.git
- Access the local folder
cd Pokemon-Bot
- Create a
conda
environment with all the needed dependencies
conda env create -f conda-environment.yaml
- Move
.env.example
to.env
and modify the values of the keys according to the instructions in Pre-requisites and Create a Discord Bot. You can leavepgql_user
,pgql_psw
andpgql_db
as they are.
mv .env.example .env
We first need to launch third-party services through docker compose
:
# The -d option is not mandatory
docker compose up [-d]
Then we activate the conda
environment:
conda activate pokemon_bot
And finally we launch the bot script:
python3 PokemonBot.py
It will take some time before the bot actually activates, because it has to load the necessary data and to configure its backend databases and caches.
The activation time may vary according to hardware capacity, GPU availability and internet connection
Once the bot is set up, it will send an introductory message on your server and will explain how to use it. You should chat with the bot on direct messages, and not on the server channels:
- Direct message (ask for info about Pokemon): if you send the bot a direct message, it will produce a response in 15 to 60s
- Command
!whatpokemon
(identify a Pokemon image): If you put the command!whatpokemon
, you should attach an image of a Pokemon and you will get out the name of it - Command
!cardpackage
(extract 5 Pokemon cards): If you input the!cardpackage
command, you will get out 5 Pokemon cards with their description
If a direct message with no command is provided to the bot, it will follow these step to produce a reply:
- Look in the semantic cache (managed with Qdrant) if the content of the message corresponds to something that was already asked. If so, the same response is given, if not, the bot adds the question to the cache and proceeds with the creation of a new answer. The search is performed with dense retrieval with the text encoder
sentence-transformers/LaBSE
available on HuggingFace - Perform hybrid search with dense (
sentence-transformers/LaBSE
) and sparse (prithivida/Splade_PP_en_v1
, available on HuggingFace) retrieval combined. The database on which the search is carried on is a Qdrant collection that contains dense and sparse vectorial representations of the English texts fromwanghaofan/pokemon-wiki-captions
(available on HuggingFace) - The results from hybrid search are reranked thanks to Cohere
rerank v3.5
and the top three hits are returned as context. - The original user message and the retrieved context form the prompt, which is added to the chat history managed by a Postgres database instance.
- The chat history is retrieved and used to query Cohere
command-r-plus-08-2024
via API - The response is produced, returned to the user and added to chat history
- The image attached to the
!whatpokemon
command message is downloaded and saved under thepokemon_images/
directory aspokemon_${time}.png
- The image is then vectorized by the feature extractor
facebook/dinov2-large
(available on HuggingFace) - The vector is used to perform dense retrieval from an image database. The image database is a Qdrant collection that contains vectorial representations of the images in
Karbo31881/Pokemon_images
(available on HuggingFace) - The label associated to the retrieved image is returned and used as response
- Five cards are randomly drawn from
TheFusion21/PokemonCards
(available on HuggingFace) - The image URLs are embedded as images in Markdown format
- The descriptions of the cards are used to produce an overall, long description of the package through Cohere
command-r-plus-08-2024
via API - The overall, long description of the card package is summarized thanks to a Cohere
command-r-plus-08-2024
instance instracted to extract a brief but enthusiastic description of the package from the longer one. - The description and the embedded images are returned as a response
Contributions are always welcome!
Find contribution guidelines at CONTRIBUTING.md.
This project is open-source and is provided under an MIT License.
If you found it useful, please consider funding it.