Add perspectiveapi for moderation #406
Conversation
… add new languages with perspective. Move language check before moderation. This change is NON BREAKING, the FORCE_ENGLISH environement variable still works.
…vice, get_force_language, get_perspective_api_key, get_max_perspective_requests_per_second functions This is NON BREAKING, the FORCE_ENGLISH env variable still works.
|
Nice! What does aiolimiter do? |
|
Soooo
The added environement variables are:
Openai language detection works only for english, and if another language is set while openai is the language detection service, lanuage detection will be disabled and a message will be printed in the terminal. Language detection automatically sends a message in the user's language if it is applicable when telling the user his message is not in {language}. I moved language detection before moderation to avoid unnecessary api calls. |
It allows me to ratelimit the calls of the api to one per second to avoid being rate limited. See the perspective_model.py file in models |
|
I will let this draft for some more time so that I can test it more and find any possible bugs. |
|
I think that It could be nice to add some knd of multi-language message replying to the enforce language, when using perspective. So it will warn the user in its language. Edit: Just pushed it to the branch |
…ignature to include an optional override_model parameter fix(moderations_service.py): update send_moderations_request method to check for override_model parameter and use it to determine which model to use fix(moderations_service.py): update set_thumbnail method in Moderation class to conditionally set the thumbnail URL based on the model type fix(moderations_service.py): update set_thumbnail method in Moderation class to conditionally set the thumbnail URL based on the language_detect_type
…PTDiscord into google-perspective
…ntent in the notification embed to hide sensitive information
…ne breaks for better readability feat(moderations_service.py): add support for language detection using OpenAI and Perspective models feat(moderations_service.py): add moderation messages for different languages to be sent to users when their messages are moderated fix(moderations_service.py): fix method signature and add missing line breaks for better readability
Yeah this would be nice, I guess we can have a layer one level above moderations that just checks what language the message was in and then outputs a locale string, and then that locale string can be passed into some sort of config file that we have locally that contains the translations of the error message in the different locales, keyed by the locale string, no need to have an LLM call for this |
|
How is this going generally? When would you like me to give it a full review? @Paillat-dev |
I am still testing a bit, and I would like that add some more error handling before, I'll let you know. |
…d ANALYZE_REQUEST_NOT_EN to allow language detection fix(perspective_model.py): add exception handling for languageNotSupportedByAttribute when using ANALYZE_REQUEST_NOT_EN fix(moderations_service.py): add docstrings to send_language_detect_request and send_moderations_request methods fix(moderations_service.py): add exception handling for languageNotSupportedByAttribute in send_moderations_request method fix(moderations_service.py): add exception handling for languageNotSupportedByAttribute in moderation loop to ignore messages with unsupported languages
…ass to improve code readability and documentation
|
I'll also write some docs in the AI-MODERATION md |
… Moderation guide The AI Moderation guide has been reorganized and improved for better readability and understanding. The changes include: - Renaming the section titles to be more descriptive. - Adding a section for choosing between the OpenAI and PerspectiveAPI moderation services. - Providing separate commands and thresholds for each moderation service. - Adding a section for language detection and the force language feature. - Including additional configuration options and instructions. - Removing the note about the server not being for support or discussions about GPT3Discord. These changes aim to make it easier for users to understand and configure the AI Moderation feature.
|
@Kav-K I'm good. |
|
Sorry for the delays on this, been a bit busy, will take a look ASAP @Paillat-dev |
Yup, take the time needed, there is no hurry :) |
|
Heyy, do I keep this open or do I close it? |
|
@Paillat-dev I've just been busy and sick recently, will review when I can |
|
@Paillat-dev Before I go ahead and test this thoroughly on my end, did you test this thoroughly locally on your end? |
I did test it, and tried to recreate as many situations as I could. |
|
Sounds good thanks, will test asap
…On Sun, Dec 31, 2023 at 5:50 AM Paillat ***@***.***> wrote:
@Paillat-dev <https://github.com/Paillat-dev> Before I go ahead and test
this thoroughly on my end, did you test this thoroughly locally on your end?
I did test it, and tried to recreate as many situations as I could.
—
Reply to this email directly, view it on GitHub
<#406 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AFBOJDOFCSO32D32VMSSRATYME7PPAVCNFSM6AAAAAA7KSI3HGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZSHEYTQNZTGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Just rebased with your main but no real change |
Signed-off-by: Paillat <me@paillat.dev>
|
@Kav-K Do I keep this open? |
No description provided.