Skip to content

Commit

Permalink
Add answers and outcome types to ai description context
Browse files Browse the repository at this point in the history
  • Loading branch information
IanPhilips committed Nov 25, 2024
1 parent 22c720e commit 2672512
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 19 deletions.
54 changes: 39 additions & 15 deletions backend/api/src/generate-ai-description.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,31 @@ import { track } from 'shared/analytics'
import { anythingToRichText } from 'shared/tiptap'
import { largePerplexityModel, perplexity } from 'shared/helpers/perplexity'
import { models, promptClaude } from 'shared/helpers/claude'
import { outcomeTypeDescriptions } from 'common/ai-creation-prompts'

export const generateAIDescription: APIHandler<
'generate-ai-description'
> = async (props, auth) => {
const { question, description } = props
const { question, description, answers, outcomeType, shouldAnswersSumToOne } =
props
const includeAnswers =
answers &&
answers.length > 0 &&
outcomeType &&
['MULTIPLE_CHOICE', 'POLL'].includes(outcomeType)
const outcomeKey =
outcomeType == 'MULTIPLE_CHOICE'
? shouldAnswersSumToOne
? 'DEPENDENT_MULTIPLE_CHOICE'
: 'INDEPENDENT_MULTIPLE_CHOICE'
: outcomeType

const userQuestionAndDescription = `Question: ${question} ${
description ? `\n\nCurrent description: ${description}` : ''
}`
description && description !== '<p></p>'
? `\nDescription: ${description}`
: ''
} ${includeAnswers ? `\nMaybe: ${answers.join(', ')}` : ''}
`

log('Generating AI description for:', userQuestionAndDescription)
try {
Expand All @@ -28,18 +44,26 @@ export const generateAIDescription: APIHandler<
const perplexityResponse =
[messages].join('\n') + '\n\nSources:\n' + citations.join('\n\n')

const systemPrompt = `You are a helpful AI assistant that generates detailed descriptions for prediction markets. Your goal is to provide relevant context, background information, and clear resolution criteria that will help traders make informed predictions.
Guidelines:
- Keep descriptions concise but informative
- Include relevant sources and data when available
- Clearly state how the market will be resolved
- Try to think of any edge cases or special scenarios that traders should be aware of, mention them in the description and how the market will be resolved in those cases
- Don't repeat the question in the description
- Focus on objective facts rather than opinions
- Format the response as markdown with sections such as "Background", "Resolution criteria", "Things to consider", etc.
- Here is current information from the internet that is related to the user's prompt. Include information from it in the description that traders or other readers may want to know if it's relevant to the user's question:
${perplexityResponse}
`
const systemPrompt = `
You are a helpful AI assistant that generates detailed descriptions for prediction markets. Your goal is to provide relevant context, background information, and clear resolution criteria that will help traders make informed predictions.
${
outcomeKey
? `Their market is of type ${outcomeKey}\n${outcomeTypeDescriptions}`
: ''
}
Guidelines:
- Keep descriptions concise but informative
- If the market is personal, (i.e. I will attend the most parties, or I will get a girlfriend) word resolution criteria in the first person
- Include relevant sources and data when available
- Clearly state how the market will be resolved
- Try to think of any edge cases or special scenarios that traders should be aware of, mention them in the description and how the market will be resolved in those cases
- Don't repeat the question in the description
- Focus on objective facts rather than opinions
- If the market has a precondition, such as 'If I attend, will I enjoy the party?', or 'If Biden runs, will he win?', markets should resolve N/A if the precondition is not met
- Format the response as markdown with sections such as "Background", "Resolution criteria", "Things to consider", etc.
- Here is current information from the internet that is related to the user's prompt. Include information from it in the description that traders or other readers may want to know if it's relevant to the user's question:
${perplexityResponse}
`

const prompt = `${userQuestionAndDescription}\n\n Only return the markdown description, nothing else`
const claudeResponse = await promptClaude(prompt, {
Expand Down
11 changes: 7 additions & 4 deletions common/src/ai-creation-prompts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,12 @@ Following each market suggestion, add a "Reasoning:" section that addresses the
1. A clear explanation of why this market follows from the user's prompt and related source material
2. Why it's a good prediction market (e.g., has clear resolution criteria, neither a yes nor no outcome is overwhelmingly likely, etc. from above)
`
export const outcomeTypeDescriptions = `
- "BINARY" means there are only two answers, true (yes) or false (no)
- "INDEPENDENT_MULTIPLE_CHOICE" means there are multiple answers, and ANY of them can resolve yes, no, or N/A e.g. What will happen during the next presidential debate? Which companies will express interest in buying twitter?
- "DEPENDENT_MULTIPLE_CHOICE" means there are multiple answers, but ONLY one can resolve yes, (while the rest resolve no, or alternatively the entire market resolves N/A if a precondition is not met) e.g. Who will win the presidential election?, Who will be the first to express interest in buying twitter?
- "POLL" means the question is about a personal matter, i.e. "Should I move to a new city?", "Should I get a new job?", etc.
`

export const formattingPrompt = `
Convert these prediction market ideas into valid JSON objects that abide by the following Manifold Market schema. Each object should include:
Expand All @@ -71,10 +77,7 @@ export const formattingPrompt = `
- closeDate (string, date in YYYY-MM-DD format, required)
- The close date is when trading stops for the market, and resolution can be made. E.g. if the title includes 'by january 1st 2025', the close date should be 2025-12-31
- outcomeType ("BINARY", "INDEPENDENT_MULTIPLE_CHOICE", "DEPENDENT_MULTIPLE_CHOICE", "POLL", required)
- "BINARY" means there are only two answers, true (yes) or false (no)
- "INDEPENDENT_MULTIPLE_CHOICE" means there are multiple answers, and ANY of them can resolve yes, e.g. What will happen during the next presidential debate? Which companies will express interest in buying twitter?
- "DEPENDENT_MULTIPLE_CHOICE" means there are multiple answers, but ONLY one can resolve yes, (while the rest resolve no) e.g. Who will win the presidential election?, Who will be the first to express interest in buying twitter?
- "POLL" means the question is about a personal matter, i.e. "Should I move to a new city?", "Should I get a new job?", etc.
${outcomeTypeDescriptions}
- answers (array of strings, recommended only if outcomeType is one of the "DEPENDENT_MULTIPLE_CHOICE" or "INDEPENDENT_MULTIPLE_CHOICE" types)
- addAnswersMode ("DISABLED", "ONLY_CREATOR", or "ANYONE", required if one of the "DEPENDENT_MULTIPLE_CHOICE" or "INDEPENDENT_MULTIPLE_CHOICE" types is provided)
- "DISABLED" means that the answers list covers all possible outcomes and no more answers can be added after the market is created
Expand Down
3 changes: 3 additions & 0 deletions common/src/api/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1838,6 +1838,9 @@ export const API = (_apiTypeCheck = {
.object({
question: z.string(),
description: z.string().optional(),
answers: z.array(z.string()).optional(),
outcomeType: z.string().optional(),
shouldAnswersSumToOne: coerceBoolean.optional(),
})
.strict(),
},
Expand Down
3 changes: 3 additions & 0 deletions web/components/new-contract/contract-params-form.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -499,6 +499,9 @@ export function ContractParamsForm(props: {
const result = await api('generate-ai-description', {
question,
description: editor?.getHTML(),
answers,
outcomeType,
shouldAnswersSumToOne,
})
if (result.description && editor) {
const endPos = editor.state.doc.content.size
Expand Down

0 comments on commit 2672512

Please sign in to comment.