Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open Leo from search #26082

Open
wants to merge 22 commits into
base: master
Choose a base branch
from
Open

Open Leo from search #26082

wants to merge 22 commits into from

Conversation

yrliou
Copy link
Member

@yrliou yrliou commented Oct 17, 2024

Resolves brave/brave-browser#41711

open_leo_from_search_480p.mov

UI was updated on search side after the above demo video was recorded, it now looks like this, with a "Save chat in Brave Leo" at upper-right corner:
Screenshot 2024-10-30 at 4 13 46 PM

  • Add a new permission for opening Leo from Brave Search
  • Add a navigation throttle to intercept the request for opening Leo from Brave Search (specifically, https://search.brave.com/leo#noncevalue). If the request is from Brave Search (last committed URL) and nonce is valid (identical between the one in URL and the one in nonce property), a permission prompt will be shown by default and Leo side panel would be opened with entries from Brave Search conversations staged in Leo side panel.
  • A side change which removes the limitation of only have staged entries at the beginning of the conversation to accommodate that user could choose to click continue with Leo button in search UI multiple times and not only when there's empty Leo chat history. When need to remove staged entries, for example, when page context is unlinked, all staged entries will be cleared instead of whole chat history.

See Requirements #2 and #3 in https://docs.google.com/document/d/1idelFPpUEcKDNcyKYf3M5yw91tuIddjrlYfpaWEJjNk/edit?tab=t.0 for reference.

S&P review: https://github.com/brave/reviews/issues/1776

Submitter Checklist:

  • I confirm that no security/privacy review is needed and no other type of reviews are needed, or that I have requested them
  • There is a ticket for my issue
  • Used Github auto-closing keywords in the PR description above
  • Wrote a good PR/commit description
  • Squashed any review feedback or "fixup" commits before merge, so that history is a record of what happened in the repo, not your PR
  • Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
  • Checked the PR locally:
    • npm run test -- brave_browser_tests, npm run test -- brave_unit_tests wiki
    • npm run presubmit wiki, npm run gn_check, npm run tslint
  • Ran git rebase master (if needed)

Reviewer Checklist:

  • A security review is not needed, or a link to one is included in the PR description
  • New files have MPL-2.0 license header
  • Adequate test coverage exists to prevent regressions
  • Major classes, functions and non-trivial code blocks are well-commented
  • Changes in component dependencies are properly reflected in gn
  • Code follows the style guide
  • Test plan is specified in PR before merging

After-merge Checklist:

Test Plan:

  1. Go to search.brave.com with experimental flags enabled.
  2. Type a query and click Answer with AI
  3. Type a follow-up question in search conversation mode UI
  4. Once answer is generated, click "Save chat in Brave Leo" at the upper-right corner.
  5. Check permission prompt is shown, and Leo side panel should be opened when permission is allowed, entries from search.brave.com should be staged in Leo.
  6. Visit brave://settings/content -> Additional permissions -> Leo AI chat, search.brave.com should be in the allow list.
  7. Remove the allow entry in settings UI.
  8. Open a new tab and repeat step 1 to 4.
  9. Permission prompt should be shown again, reject the prompt, Leo panel should not be opened.

Test opt-out case:
Try the feature in a profile that hasn't opted-in to Leo, when the continue with Leo button is clicked, accept the permission prompt, Leo should be opened with opt-in screen with disclaimer, search entries should appear after opt-in.

Test private window:
Try the feature in private window, shouldn't have any actions such as opening Leo or permission prompt.

@yrliou yrliou self-assigned this Oct 17, 2024
@github-actions github-actions bot added CI/storybook-url Deploy storybook and provide a unique URL for each build CI/run-upstream-tests Run upstream unit and browser tests on Linux and Windows (otherwise only on Linux) labels Oct 17, 2024
@yrliou yrliou force-pushed the open_leo_from_search branch 3 times, most recently from dce4a3c to 1f2c4c1 Compare October 19, 2024 00:13
@brave-builds
Copy link
Collaborator

A Storybook has been deployed to preview UI for the latest push

@yrliou yrliou force-pushed the open_leo_from_search branch 6 times, most recently from 0c99be2 to 9ef822a Compare October 19, 2024 06:15
@yrliou yrliou marked this pull request as ready for review October 21, 2024 17:17
@yrliou yrliou requested review from a team as code owners October 21, 2024 17:17
Copy link
Collaborator

@cdesouza-chromium cdesouza-chromium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving some comments for now. Be back later.

app/brave_settings_strings.grdp Outdated Show resolved Hide resolved
browser/ai_chat/page_content_fetcher_browsertest.cc Outdated Show resolved Hide resolved
browser/ai_chat/page_content_fetcher_browsertest.cc Outdated Show resolved Hide resolved
browser/ai_chat/page_content_fetcher_browsertest.cc Outdated Show resolved Hide resolved
browser/ui/sidebar/sidebar_utils.cc Outdated Show resolved Hide resolved
Copy link
Member

@petemill petemill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work - my review mainly focuses on naming and keeping consistent with "AI Chat" and not "Leo" - sorry!


namespace ai_chat {

void OpenLeo(content::WebContents* web_contents);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's avoid the brand name in the code, since this is subject to change. Let's keep it AIChat. In this case we want to open a conversation associated with a Tab, so perhaps the function name should be OpenAIChatForTab?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to OpenAIChatForTab in ab4ea07.

@@ -96,6 +102,14 @@ void BravePrivacyHandler::AddLoadTimeData(content::WebUIDataSource* data_source,
"isLocalhostAccessFeatureEnabled",
base::FeatureList::IsEnabled(
brave_shields::features::kBraveLocalhostAccessPermission));
data_source->AddBoolean(
"isOpenLeoFromBraveSearchEnabled",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isOpenAIChatFromBraveSearchEnabled

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -24,6 +24,7 @@ enum ContentSettingsType {
BRAVE_HTTPS_UPGRADE,
BRAVE_REMEMBER_1P_STORAGE,
BRAVE_LOCALHOST_ACCESS,
BRAVE_AI_CHAT,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do think it would be clearer to understand if it was something like BRAVE_CONTINUE_IN_AI_CHAT as it's not "enabling AI Chat" for that site.

@@ -24,6 +24,9 @@ enum ContentSettingsType {
BRAVE_HTTPS_UPGRADE,
BRAVE_REMEMBER_1P_STORAGE,
BRAVE_LOCALHOST_ACCESS,
// Allow a site to open Leo (in side panel on Desktop).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Chat

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

void OnPermissionPromptResult(blink::mojom::PermissionStatus status);
void OnOpenLeo();

void OpenLeoWithStagedConversations();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it's clearer as OpenConversationWithStagedEntries?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renames as OpenAIChatWithStagedEntries in ab4ea07.

@@ -60,4 +60,16 @@ bool IsPageContentRefineEnabled() {
return base::FeatureList::IsEnabled(features::kPageContentRefine);
}

BASE_FEATURE(kOpenLeoFromBraveSearch,
"OpenLeoFromBraveSearch",
#if !BUILDFLAG(IS_ANDROID)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this condition also include iOS? Also, why is it disabled on android? Nearly all the logic should be the same, aside from the UI-opening delegate.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could just enable the whole feature when it's done.
Currently this feature flag is mainly gating the navigation throttle, and without UI opening, there's really no much point have it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add iOS, tho currently this feature flag is used only in webui and contents folder, so not really affecting anything on iOS atm.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added ios in ab4ea07


namespace permissions {

class BraveAIChatPermissionContext : public PermissionContextBase {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this could never encompass all possible AI Chat content settings permissions (should we gain any more in the future), should this be named specifically for the permission it's describing? e.g. BraveAIChatContinueConversationPermissionContext?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a similar comment about the content setting BRAVE_AI_CHAT

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@petemill How about BraveOpenAIChatPermissionContext?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to BraveOpenAIChatPermissionContext and BRAVE_OPEN_AI_CHAT in ab1e047

@@ -21,4 +21,10 @@
<message name="IDS_PERMISSIONS_BUBBLE_SITE_PERMISSION_LINK" desc="Site permission link in the footnote description.">
site permission
</message>
<message name="IDS_OPEN_AI_CHAT_PERMISSION_FRAGMENT" desc="Label for Leo AI chat permission prompt.">
Allow opening Leo in Brave?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps Allow opening Leo AI conversations in Brave?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allow opening Leo in Brave?
</message>
<message name="IDS_AI_CHAT_INFOBAR_TEXT" desc="Label for Leo AI chat permission prompt (Android).">
Allow <ph name="URL">$1<ex>search.brave.com</ex></ph> to open Leo in Brave?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Allow search.brave.com to open related Leo AI conversations in Brave?"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Allow search.brave.com to open Leo AI conversations in Brave?"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Allow search.brave.com to open Leo AI conversations in Brave?" in ab4ea07

@@ -0,0 +1,22 @@
<!DOCTYPE html>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: continue_conversation_button.html or anything without the leo string.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the directory should be aichat as well? Internally everything should be aichat for consistency?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or are we differentiating between aichat in general and the sidebar specifically? If so I would go with aichat_sidebar or something. It's very confusing when some of the code uses leo and other parts use aichat as we discussed before @petemill

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would go for open_ai_chat_button.html for now.
Historically we have two folders under test/data, I'll do a quick follow-up after this PR to only have test/data/ai_chat.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit also
- updates permission strings
- adds IOS buildflag in feature flag
Copy link
Contributor

[puLL-Merge] - brave/brave-core@26082

Description

This pull request adds functionality to open Leo AI chat from Brave Search. The changes include new permission handling, UI modifications, and backend logic to support this feature. The implementation is primarily focused on desktop platforms, with some groundwork laid for potential future Android support.

Changes

Changes

  1. app/brave_settings_strings.grdp:

    • Added new strings for Leo AI chat settings in the site settings menu.
  2. browser/about_flags.cc:

    • Refactored AI chat feature entries for better organization.
    • Added a new feature flag for opening Leo AI chat from Brave Search.
  3. browser/ai_chat/BUILD.gn:

    • Added new source files for AI chat functionality.
  4. browser/brave_content_browser_client.cc:

    • Implemented logic to create AIChatBraveSearchThrottle for navigations.
  5. browser/resources/settings/:

    • Modified various TypeScript files to add UI elements for Leo AI chat settings.
  6. browser/ui/ai_chat/:

    • Added new utility functions for opening AI chat.
  7. components/ai_chat/:

    • Implemented AIChatBraveSearchThrottle to handle opening Leo from Brave Search.
    • Modified AIChatTabHelper, PageContentFetcher, and other related files to support the new functionality.
  8. components/permissions/:

    • Added a new permission context for Leo AI chat.
    • Updated various files to handle the new permission type.
  9. chromium_src/:

    • Modified various Chromium files to integrate the new Leo AI chat permission.
  10. test/data/leo/:

    • Added test HTML files for Leo AI chat functionality.

Possible Issues

  1. The implementation is primarily focused on desktop platforms. Android support may require additional work.
  2. The interaction between the new Leo AI chat feature and existing AI chat functionality should be thoroughly tested to ensure there are no conflicts.

Security Hotspots

  1. components/ai_chat/content/browser/ai_chat_brave_search_throttle.cc:

    • The AIChatBraveSearchThrottle class handles opening Leo AI chat. Ensure that proper checks are in place to prevent unauthorized access or potential abuse of this feature.
  2. components/permissions/contexts/brave_open_ai_chat_permission_context.cc:

    • The permission context for Leo AI chat should be carefully reviewed to ensure it properly restricts access to only authorized domains (i.e., Brave Search).
  3. browser/ui/ai_chat/utils.cc:

    • The OpenAIChatForTab function should be reviewed to ensure it cannot be exploited to open AI chat in unintended contexts.

Overall, this PR introduces a significant new feature with careful implementation of permissions and security checks. The main security considerations revolve around ensuring that the Leo AI chat can only be opened from legitimate Brave Search pages and that the permission system is robust.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI/run-upstream-tests Run upstream unit and browser tests on Linux and Windows (otherwise only on Linux) CI/storybook-url Deploy storybook and provide a unique URL for each build needs-security-review puLL-Merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[SERP->Leo] Support Continue with Leo button from Brave Search