Open Leo from search #26082

yrliou · 2024-10-17T22:22:03Z

open_leo_from_search_480p.mov

UI was updated on search side after the above demo video was recorded, it now looks like this, with a "Save chat in Brave Leo" at upper-right corner:

Add a new permission for opening Leo from Brave Search
Add a navigation throttle to intercept the request for opening Leo from Brave Search (specifically, https://search.brave.com/leo#noncevalue). If the request is from Brave Search (last committed URL) and nonce is valid (identical between the one in URL and the one in nonce property), a permission prompt will be shown by default and Leo side panel would be opened with entries from Brave Search conversations staged in Leo side panel.
A side change which removes the limitation of only have staged entries at the beginning of the conversation to accommodate that user could choose to click continue with Leo button in search UI multiple times and not only when there's empty Leo chat history. When need to remove staged entries, for example, when page context is unlinked, all staged entries will be cleared instead of whole chat history.

See Requirements #2 and #3 in https://docs.google.com/document/d/1idelFPpUEcKDNcyKYf3M5yw91tuIddjrlYfpaWEJjNk/edit?tab=t.0 for reference.

S&P review: https://github.com/brave/reviews/issues/1776

Submitter Checklist:

I confirm that no security/privacy review is needed and no other type of reviews are needed, or that I have requested them
There is a ticket for my issue
Used Github auto-closing keywords in the PR description above
Wrote a good PR/commit description
Squashed any review feedback or "fixup" commits before merge, so that history is a record of what happened in the repo, not your PR
Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
Checked the PR locally:
- npm run test -- brave_browser_tests, npm run test -- brave_unit_tests wiki
- npm run presubmit wiki, npm run gn_check, npm run tslint
Ran git rebase master (if needed)

Reviewer Checklist:

A security review is not needed, or a link to one is included in the PR description
New files have MPL-2.0 license header
Adequate test coverage exists to prevent regressions
Major classes, functions and non-trivial code blocks are well-commented
Changes in component dependencies are properly reflected in gn
Code follows the style guide
Test plan is specified in PR before merging

After-merge Checklist:

The associated issue milestone is set to the smallest version that the
changes has landed on
All relevant documentation has been updated, for instance:

Test Plan:

Go to search.brave.com with experimental flags enabled.
Type a query and click Answer with AI
Type a follow-up question in search conversation mode UI
Once answer is generated, click "Save chat in Brave Leo" at the upper-right corner.
Check permission prompt is shown, and Leo side panel should be opened when permission is allowed, entries from search.brave.com should be staged in Leo.
Visit brave://settings/content -> Additional permissions -> Leo AI chat, search.brave.com should be in the allow list.
Remove the allow entry in settings UI.
Open a new tab and repeat step 1 to 4.
Permission prompt should be shown again, reject the prompt, Leo panel should not be opened.

Test opt-out case:
Try the feature in a profile that hasn't opted-in to Leo, when the continue with Leo button is clicked, accept the permission prompt, Leo should be opened with opt-in screen with disclaimer, search entries should appear after opt-in.

Test private window:
Try the feature in private window, shouldn't have any actions such as opening Leo or permission prompt.

browser/about_flags.cc

brave-builds · 2024-10-19T00:46:59Z

A Storybook has been deployed to preview UI for the latest push

cdesouza-chromium

Leaving some comments for now. Be back later.

app/brave_settings_strings.grdp

browser/ai_chat/ai_chat_brave_search_throttle_browsertest.cc

browser/ai_chat/page_content_fetcher_browsertest.cc

browser/ui/ai_chat/ai_chat_brave_search_throttle_delegate_impl.h

browser/ui/sidebar/sidebar_utils.cc

components/ai_chat/renderer/page_content_extractor.cc

components/ai_chat/content/browser/page_content_fetcher.cc

- Only get nonce from HTML in PageContentExtractor

components/ai_chat/content/browser/page_content_fetcher.cc

petemill

Nice work - my review mainly focuses on naming and keeping consistent with "AI Chat" and not "Leo" - sorry!

petemill · 2024-10-30T00:02:40Z

browser/ui/ai_chat/utils.h

+
+namespace ai_chat {
+
+void OpenLeo(content::WebContents* web_contents);


Let's avoid the brand name in the code, since this is subject to change. Let's keep it AIChat. In this case we want to open a conversation associated with a Tab, so perhaps the function name should be OpenAIChatForTab?

Renamed to OpenAIChatForTab in ab4ea07.

petemill · 2024-10-30T00:03:16Z

browser/ui/webui/settings/brave_privacy_handler.cc

@@ -96,6 +102,14 @@ void BravePrivacyHandler::AddLoadTimeData(content::WebUIDataSource* data_source,
      "isLocalhostAccessFeatureEnabled",
      base::FeatureList::IsEnabled(
          brave_shields::features::kBraveLocalhostAccessPermission));
+  data_source->AddBoolean(
+      "isOpenLeoFromBraveSearchEnabled",


isOpenAIChatFromBraveSearchEnabled

petemill · 2024-10-30T00:07:37Z

chromium_src/components/content_settings/core/common/content_settings_types.mojom

@@ -24,6 +24,7 @@ enum ContentSettingsType {
  BRAVE_HTTPS_UPGRADE,
  BRAVE_REMEMBER_1P_STORAGE,
  BRAVE_LOCALHOST_ACCESS,
+  BRAVE_AI_CHAT,


I do think it would be clearer to understand if it was something like BRAVE_CONTINUE_IN_AI_CHAT as it's not "enabling AI Chat" for that site.

petemill · 2024-10-30T00:07:50Z

chromium_src/components/content_settings/core/common/content_settings_types.mojom

@@ -24,6 +24,9 @@ enum ContentSettingsType {
  BRAVE_HTTPS_UPGRADE,
  BRAVE_REMEMBER_1P_STORAGE,
  BRAVE_LOCALHOST_ACCESS,
+  // Allow a site to open Leo (in side panel on Desktop).


petemill · 2024-10-30T00:08:49Z

components/ai_chat/content/browser/ai_chat_brave_search_throttle.h

+  void OnPermissionPromptResult(blink::mojom::PermissionStatus status);
+  void OnOpenLeo();
+
+  void OpenLeoWithStagedConversations();


Perhaps it's clearer as OpenConversationWithStagedEntries?

Renames as OpenAIChatWithStagedEntries in ab4ea07.

petemill · 2024-10-30T00:28:33Z

components/ai_chat/core/common/features.cc

@@ -60,4 +60,16 @@ bool IsPageContentRefineEnabled() {
  return base::FeatureList::IsEnabled(features::kPageContentRefine);
 }

+BASE_FEATURE(kOpenLeoFromBraveSearch,
+             "OpenLeoFromBraveSearch",
+#if !BUILDFLAG(IS_ANDROID)


should this condition also include iOS? Also, why is it disabled on android? Nearly all the logic should be the same, aside from the UI-opening delegate.

I think we could just enable the whole feature when it's done.
Currently this feature flag is mainly gating the navigation throttle, and without UI opening, there's really no much point have it.

I'll add iOS, tho currently this feature flag is used only in webui and contents folder, so not really affecting anything on iOS atm.

added ios in ab4ea07

petemill · 2024-10-30T00:31:13Z

components/permissions/contexts/brave_ai_chat_permission_context.h

+
+namespace permissions {
+
+class BraveAIChatPermissionContext : public PermissionContextBase {


If this could never encompass all possible AI Chat content settings permissions (should we gain any more in the future), should this be named specifically for the permission it's describing? e.g. BraveAIChatContinueConversationPermissionContext?

I had a similar comment about the content setting BRAVE_AI_CHAT

@petemill How about BraveOpenAIChatPermissionContext?

Changed to BraveOpenAIChatPermissionContext and BRAVE_OPEN_AI_CHAT in ab1e047

petemill · 2024-10-30T00:32:50Z

components/resources/permissions_strings.grdp

@@ -21,4 +21,10 @@
  <message name="IDS_PERMISSIONS_BUBBLE_SITE_PERMISSION_LINK" desc="Site permission link in the footnote description.">
    site permission
  </message>
+  <message name="IDS_OPEN_AI_CHAT_PERMISSION_FRAGMENT" desc="Label for Leo AI chat permission prompt.">
+    Allow opening Leo in Brave?


Perhaps Allow opening Leo AI conversations in Brave?

petemill · 2024-10-30T00:33:24Z

components/resources/permissions_strings.grdp

+    Allow opening Leo in Brave?
+  </message>
+  <message name="IDS_AI_CHAT_INFOBAR_TEXT" desc="Label for Leo AI chat permission prompt (Android).">
+    Allow <ph name="URL">$1<ex>search.brave.com</ex></ph> to open Leo in Brave?


"Allow search.brave.com to open related Leo AI conversations in Brave?"?

"Allow search.brave.com to open Leo AI conversations in Brave?"?

"Allow search.brave.com to open Leo AI conversations in Brave?" in ab4ea07

petemill · 2024-10-30T00:35:22Z

test/data/leo/open_leo_button.html

@@ -0,0 +1,22 @@
+<!DOCTYPE html>


nit: continue_conversation_button.html or anything without the leo string.

I think the directory should be aichat as well? Internally everything should be aichat for consistency?

or are we differentiating between aichat in general and the sidebar specifically? If so I would go with aichat_sidebar or something. It's very confusing when some of the code uses leo and other parts use aichat as we discussed before @petemill

I would go for open_ai_chat_button.html for now.
Historically we have two folders under test/data, I'll do a quick follow-up after this PR to only have test/data/ai_chat.

This commit also - updates permission strings - adds IOS buildflag in feature flag

github-actions · 2024-10-30T20:33:36Z

[puLL-Merge] - brave/brave-core@26082

Description

This pull request adds functionality to open Leo AI chat from Brave Search. The changes include new permission handling, UI modifications, and backend logic to support this feature. The implementation is primarily focused on desktop platforms, with some groundwork laid for potential future Android support.

Changes

Changes

app/brave_settings_strings.grdp:
- Added new strings for Leo AI chat settings in the site settings menu.
browser/about_flags.cc:
- Refactored AI chat feature entries for better organization.
- Added a new feature flag for opening Leo AI chat from Brave Search.
browser/ai_chat/BUILD.gn:
- Added new source files for AI chat functionality.
browser/brave_content_browser_client.cc:
- Implemented logic to create AIChatBraveSearchThrottle for navigations.
browser/resources/settings/:
- Modified various TypeScript files to add UI elements for Leo AI chat settings.
browser/ui/ai_chat/:
- Added new utility functions for opening AI chat.
components/ai_chat/:
- Implemented AIChatBraveSearchThrottle to handle opening Leo from Brave Search.
- Modified AIChatTabHelper, PageContentFetcher, and other related files to support the new functionality.
components/permissions/:
- Added a new permission context for Leo AI chat.
- Updated various files to handle the new permission type.
chromium_src/:
- Modified various Chromium files to integrate the new Leo AI chat permission.
test/data/leo/:
- Added test HTML files for Leo AI chat functionality.

Possible Issues

The implementation is primarily focused on desktop platforms. Android support may require additional work.
The interaction between the new Leo AI chat feature and existing AI chat functionality should be thoroughly tested to ensure there are no conflicts.

Security Hotspots

components/ai_chat/content/browser/ai_chat_brave_search_throttle.cc:
- The AIChatBraveSearchThrottle class handles opening Leo AI chat. Ensure that proper checks are in place to prevent unauthorized access or potential abuse of this feature.
components/permissions/contexts/brave_open_ai_chat_permission_context.cc:
- The permission context for Leo AI chat should be carefully reviewed to ensure it properly restricts access to only authorized domains (i.e., Brave Search).
browser/ui/ai_chat/utils.cc:
- The OpenAIChatForTab function should be reviewed to ensure it cannot be exploited to open AI chat in unintended contexts.

Overall, this PR introduces a significant new feature with careful implementation of permissions and security checks. The main security considerations revolve around ensuring that the Leo AI chat can only be opened from legitimate Brave Search pages and that the permission system is robust.

yrliou self-assigned this Oct 17, 2024

github-actions bot added CI/storybook-url Deploy storybook and provide a unique URL for each build CI/run-upstream-tests Run upstream unit and browser tests on Linux and Windows (otherwise only on Linux) labels Oct 17, 2024

yrliou commented Oct 17, 2024

View reviewed changes

browser/about_flags.cc Show resolved Hide resolved

yrliou force-pushed the open_leo_from_search branch 3 times, most recently from dce4a3c to 1f2c4c1 Compare October 19, 2024 00:13

yrliou force-pushed the open_leo_from_search branch 6 times, most recently from 0c99be2 to 9ef822a Compare October 19, 2024 06:15

yrliou marked this pull request as ready for review October 21, 2024 17:17

yrliou requested review from a team as code owners October 21, 2024 17:17

yrliou requested review from petemill, fallaciousreasoning and darkdh October 21, 2024 17:18

cdesouza-chromium reviewed Oct 21, 2024

View reviewed changes

github-actions bot added the puLL-Merge label Oct 21, 2024

github-actions bot reviewed Oct 21, 2024

View reviewed changes

components/ai_chat/renderer/page_content_extractor.cc Outdated Show resolved Hide resolved

components/ai_chat/content/browser/page_content_fetcher.cc Show resolved Hide resolved

github-actions bot added the needs-security-review label Oct 21, 2024

github-actions bot assigned bridiver and diracdeltas Oct 21, 2024

yrliou unassigned iefremov and goodov Oct 28, 2024

yrliou added 4 commits October 28, 2024 22:07

Add back content setting registration for Android

c366be1

Revise permission string

0a9c5a2

Add RequestType::kBraveMinValue and RequestType::kBraveMaxValue

35c4751

Move the nonce validation to AIChatSearchThrottle

762e961

- Only get nonce from HTML in PageContentExtractor

github-actions bot reviewed Oct 29, 2024

View reviewed changes

components/ai_chat/content/browser/page_content_fetcher.cc Show resolved Hide resolved

github-actions bot assigned bridiver, diracdeltas, goodov, iefremov and thypon Oct 29, 2024

yrliou added 2 commits October 29, 2024 11:33

Fix method typo

0e5edde

Add comment for BRAVE_AI_CHAT content setting

0db3a85

yrliou force-pushed the open_leo_from_search branch from c973c0b to 0db3a85 Compare October 29, 2024 18:52

yrliou unassigned bridiver, diracdeltas, thypon, iefremov and goodov Oct 29, 2024

petemill reviewed Oct 30, 2024

View reviewed changes

More unit tests

be1b396

yrliou force-pushed the open_leo_from_search branch from 763155e to be1b396 Compare October 30, 2024 03:22

bridiver approved these changes Oct 30, 2024

View reviewed changes

yrliou added 2 commits October 30, 2024 11:04

Rename OpenLeo -> OpenAIChat

ab4ea07

This commit also - updates permission strings - adds IOS buildflag in feature flag

Rename permission/content setting to OpenAIChat

ab1e047

yrliou added 3 commits October 30, 2024 15:26

fixup! Rename OpenLeo -> OpenAIChat

c07dcff

update patch after content setting type name change

4b7d3d4

fixup! Rename permission/content setting to OpenAIChat

e882251

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Open Leo from search #26082

Open Leo from search #26082

yrliou commented Oct 17, 2024 •

edited

Loading

brave-builds commented Oct 19, 2024

cdesouza-chromium left a comment

petemill left a comment

petemill Oct 30, 2024

yrliou Oct 30, 2024

petemill Oct 30, 2024

yrliou Oct 30, 2024

petemill Oct 30, 2024

petemill Oct 30, 2024

yrliou Oct 30, 2024

petemill Oct 30, 2024

yrliou Oct 30, 2024

petemill Oct 30, 2024

yrliou Oct 30, 2024

yrliou Oct 30, 2024

yrliou Oct 30, 2024

petemill Oct 30, 2024

bridiver Oct 30, 2024

yrliou Oct 30, 2024

yrliou Oct 30, 2024

petemill Oct 30, 2024

yrliou Oct 30, 2024

petemill Oct 30, 2024

yrliou Oct 30, 2024

yrliou Oct 30, 2024

petemill Oct 30, 2024

bridiver Oct 30, 2024

bridiver Oct 30, 2024

yrliou Oct 30, 2024

yrliou Oct 30, 2024

github-actions bot commented Oct 30, 2024

Changes


		namespace ai_chat {

		void OpenLeo(content::WebContents* web_contents);


		namespace permissions {

		class BraveAIChatPermissionContext : public PermissionContextBase {

Open Leo from search #26082

Are you sure you want to change the base?

Open Leo from search #26082

Conversation

yrliou commented Oct 17, 2024 • edited Loading

Submitter Checklist:

Reviewer Checklist:

After-merge Checklist:

Test Plan:

brave-builds commented Oct 19, 2024

cdesouza-chromium left a comment

Choose a reason for hiding this comment

petemill left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 30, 2024

Description

Changes

Possible Issues

Security Hotspots

yrliou commented Oct 17, 2024 •

edited

Loading