Bootstrap Web Discovery Native, add credential manager and server config loading #24969

DJAndries · 2024-08-01T23:08:12Z

Resolves brave/brave-browser#39439

Submitter Checklist:

I confirm that no security/privacy review is needed and no other type of reviews are needed, or that I have requested them
There is a ticket for my issue
Used Github auto-closing keywords in the PR description above
Wrote a good PR/commit description
Squashed any review feedback or "fixup" commits before merge, so that history is a record of what happened in the repo, not your PR
Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
Checked the PR locally:
- npm run test -- brave_browser_tests, npm run test -- brave_unit_tests wiki
- npm run presubmit wiki, npm run gn_check, npm run tslint
Ran git rebase master (if needed)

Reviewer Checklist:

A security review is not needed, or a link to one is included in the PR description
New files have MPL-2.0 license header
Adequate test coverage exists to prevent regressions
Major classes, functions and non-trivial code blocks are well-commented
Changes in component dependencies are properly reflected in gn
Code follows the style guide
Test plan is specified in PR before merging

After-merge Checklist:

The associated issue milestone is set to the smallest version that the
changes has landed on
All relevant documentation has been updated, for instance:

Test Plan:

brave-builds · 2024-08-02T00:02:14Z

A Storybook has been deployed to preview UI for the latest push

fmarier

script/brave_license_helper.py change looks good

deeppandya · 2024-08-05T08:04:31Z

android/java/org/chromium/chrome/browser/privacy/settings/BravePrivacySettings.java

@@ -682,6 +696,16 @@ private void updateBravePreferences() {
            getPreferenceScreen().removePreference(mSendP3A);
        }

+        if (mSendWebDiscovery != null) {
+            mSendWebDiscovery.setTitle(


we can remove title/summary here as we already have those mentioned in .xml

deeppandya

Android changes LGTM with minor comment.

goodov · 2024-08-05T10:08:20Z

android/BUILD.gn

@@ -5,6 +5,7 @@

 import("//brave/components/ai_chat/core/common/buildflags/buildflags.gni")
 import("//brave/components/p3a/buildflags.gni")
+import("//brave/components/web_discovery/common/buildflags/buildflags.gni")


please store it in //brave/components/web_discovery/buildflags/buildflags.gni

to be clear that means that everything in browser should also move into the root of components/web_discovery. https://github.com/brave/brave-core/pull/24970/files adds some renderer code, but I'm not sure we actually need that as there are existing mechanisms that could possibly be used https://github.com/brave/brave-core/pull/24970/files

to be clear that means that everything in browser should also move into the root of components/web_discovery.

I don't get why browser should be moved to the root of a component. buildflags is unconditional target, other targets should be conditional, including common, browser, renderer, etc., and they should stay in their own directories.

There's other change we should do (after discussing with @atuchin-m): browser should actually be renamed into core if we're aiming to use most of the parts on iOS.

actually looks like upstream code is putting buildflags in the root even when there are process/common/core directories. Not sure if that changed at some point or if we've just always been doing it incorrectly when there are process specific directories (which there will be in the next PR). There are only a few examples of buildflags in components in upstream and it doesn't specify in the component docs

browser/ui/BUILD.gn

browser/ui/webui/brave_settings_ui.cc

browser/sources.gni

browser/web_discovery/BUILD.gn

goodov · 2024-08-05T12:41:44Z

components/web_discovery/browser/server_config_loader.cc

+  return map;
+}
+
+std::optional<std::string> GunzipContents(std::string gzipped_contents) {


why this machinery is required? Is server missing Content-Encoding: gzip header?

URLLoader should handle this automatically. There's a test: SimpleURLLoaderTest, GzipBody.

Is server missing Content-Encoding: gzip header?

Yes, the server does not provide the header because it's serving a pre-gzipped file.

Is server missing Content-Encoding: gzip header?

Yes, the server does not provide the header because it's serving a pre-gzipped file.

Sorry, but that's not an excuse. Is it served from AWS/Cloudflare? Just set "Content-Encoding: gzip" on server, do not perform manual unzipping.

example how to set during aws s3 cp: https://github.com/brave/brave-variations/blob/55500374116ea45c0c1cd9e3cb2416326c80a285/.github/workflows/generate-test-seed.yml#L82

that modification may have ramifications for the original extension, since it also performs manual gunzipping. will investigate

components/web_discovery/browser/util.cc

components/web_discovery/browser/web_discovery_service.cc

goodov · 2024-08-05T12:52:46Z

components/web_discovery/browser/web_discovery_service.cc

+  server_config_loader_ = nullptr;
+  credential_manager_ = nullptr;
+
+  profile_prefs_->ClearPref(kWebDiscoveryNativeEnabled);


I don't think Stop should clear kWebDiscoveryNativeEnabled. Other prefs cleanup is understandable.

using Start and Stop seems confusing to me. Shouldn't this be Enable and Disable? As mentioned above having separate prefs for native and extension seems unnecessarily complex and confusing.

i think the current names are appropriate. Start also gets called on profile startup, so the name Enable doesn't seem correct.

components/web_discovery/common/features.cc

components/web_discovery/browser/BUILD.gn

bridiver · 2024-08-06T20:35:57Z

components/constants/pref_names.h

@@ -85,7 +85,8 @@ inline constexpr char kBraveShieldsSettingsVersion[] =
 inline constexpr char kDefaultBrowserPromptEnabled[] =
    "brave.default_browser_prompt_enabled";

-inline constexpr char kWebDiscoveryEnabled[] = "brave.web_discovery_enabled";
+inline constexpr char kWebDiscoveryExtensionEnabled[] =


I'm a bit confused here, usages of kWebDiscoveryExtensionEnabled are guarded by extensions enabled (in some places and missing in others), but this is not so why are there guards for the pref itself?

It seems there's no reason to have a guard around the kWebDiscoveryExtensionEnabled pref and the guards are also inconsistent and sometimes include the native pref so I think we should just drop them. Guards should generally be consistent between gn and cpp so if neither this file nor the pref declaration itself is guarded by enable extensions then access to it shouldn't be guarded by extensions enabled either

i added the missing guards in two places. in general, if there is a guard around both the extension pref and the native pref, it's for pref migration

but there's no reason for this added complexity and there are still missing guards. Also this pref should move out of components/constants and into web_discovery.

added more guards. also decided to keep the pref here since it's shared between the extension and the native component. since the native component is behind a separate flag, it didn't seem like the best place for the shared pref.

components/web_discovery/browser/web_discovery_service.cc

bridiver · 2024-08-06T20:41:44Z

components/web_discovery/common/buildflags/buildflags.gni

+# You can obtain one at https://mozilla.org/MPL/2.0/.
+
+declare_args() {
+  enable_web_discovery_native = is_win || is_mac || is_linux || is_android


why not ios?

ios will not be supported initially

components/web_discovery/browser/anonymous_credentials/rs/cxx/BUILD.gn

components/web_discovery/browser/credential_manager.h

android/java/org/chromium/chrome/browser/privacy/settings/BravePrivacySettings.java

bridiver · 2024-08-08T00:52:59Z

components/web_discovery/browser/web_discovery_service.cc

+
+void WebDiscoveryService::RegisterLocalStatePrefs(
+    PrefRegistrySimple* registry) {
+  registry->RegisterTimePref(kPatternsRetrievalTime, {});


why is this a pref? Is it configurable?

the value of the pref is managed by the ServerConfigLoader. it refers to the last retrieval time for the patterns file, and is used for scheduling the next patterns retrieval request.

components/web_discovery/browser/web_discovery_service.cc

brave-builds · 2024-08-08T07:56:03Z

A Storybook has been deployed to preview UI for the latest push

components/web_discovery/browser/credential_manager.h

components/web_discovery/browser/patterns.cc

goodov · 2024-08-08T07:29:44Z

components/web_discovery/browser/server_config_loader.cc

+  return map;
+}
+
+std::optional<std::string> GunzipContents(std::string gzipped_contents) {


Is server missing Content-Encoding: gzip header?

Yes, the server does not provide the header because it's serving a pre-gzipped file.

Sorry, but that's not an excuse. Is it served from AWS/Cloudflare? Just set "Content-Encoding: gzip" on server, do not perform manual unzipping.

example how to set during aws s3 cp: https://github.com/brave/brave-variations/blob/55500374116ea45c0c1cd9e3cb2416326c80a285/.github/workflows/generate-test-seed.yml#L82

components/web_discovery/browser/server_config_loader.cc

components/web_discovery/browser/credential_manager.cc

brave-builds · 2024-08-08T23:55:53Z

A Storybook has been deployed to preview UI for the latest push

…fig loading

…r Web Discovery native + extension

brave-builds · 2024-10-25T03:11:34Z

A Storybook has been deployed to preview UI for the latest push

DJAndries requested review from deeppandya, fmarier and a team as code owners August 1, 2024 23:08

github-actions bot assigned DJAndries Aug 1, 2024

github-actions bot added the CI/storybook-url Deploy storybook and provide a unique URL for each build label Aug 1, 2024

DJAndries mentioned this pull request Aug 1, 2024

Native re-implementation of Web Discovery (part 1) #24421

Closed

24 tasks

fmarier approved these changes Aug 2, 2024

View reviewed changes

deeppandya reviewed Aug 5, 2024

View reviewed changes

deeppandya approved these changes Aug 5, 2024

View reviewed changes

goodov reviewed Aug 5, 2024

View reviewed changes

bridiver reviewed Aug 6, 2024

View reviewed changes

components/web_discovery/browser/BUILD.gn Outdated Show resolved Hide resolved

bridiver reviewed Aug 6, 2024

View reviewed changes

components/web_discovery/browser/web_discovery_service.cc Outdated Show resolved Hide resolved

bridiver reviewed Aug 6, 2024

View reviewed changes

DJAndries force-pushed the wdp-native-anon-creds branch from 0eb2038 to 87e92c0 Compare August 7, 2024 20:33

DJAndries requested review from a team as code owners August 7, 2024 20:33

bridiver reviewed Aug 8, 2024

View reviewed changes

components/web_discovery/browser/anonymous_credentials/rs/cxx/BUILD.gn Outdated Show resolved Hide resolved

bridiver reviewed Aug 8, 2024

View reviewed changes

components/web_discovery/browser/credential_manager.h Outdated Show resolved Hide resolved

bridiver reviewed Aug 8, 2024

View reviewed changes

android/java/org/chromium/chrome/browser/privacy/settings/BravePrivacySettings.java Show resolved Hide resolved

bridiver reviewed Aug 8, 2024

View reviewed changes

components/web_discovery/browser/web_discovery_service.cc Outdated Show resolved Hide resolved

DJAndries force-pushed the wdp-native-cred-mgr-srv-config branch from 4f11211 to 40d9461 Compare August 8, 2024 06:48

goodov reviewed Aug 8, 2024

View reviewed changes

DJAndries added 8 commits October 24, 2024 17:37

Bootstrap Web Discovery Native, add credential manager and server con…

5715814

…fig loading

Address Web Discovery feedback

ba4c64a

Address Web Discovery feedback

a7d9315

Address Web Discovery feedback

4aaff0b

Move Web Discovery Android prefs to separate target

fecbbfd

Add extension API for checking WDP native feature, use single pref fo…

da9920d

…r Web Discovery native + extension

Fix new RSA key check

2f7eded

Address Web Discovery feedback

5b90bd3

DJAndries force-pushed the wdp-native-cred-mgr-srv-config branch from 97f3219 to 5b90bd3 Compare October 25, 2024 01:51

DJAndries requested review from bsclifton, a team, iefremov and simonhong as code owners October 25, 2024 01:51

DJAndries force-pushed the wdp-native-anon-creds branch from 323061b to f98e64b Compare October 25, 2024 01:51

Address Web Discovery feedback

8b00999

DJAndries force-pushed the wdp-native-cred-mgr-srv-config branch from b0de36a to 8b00999 Compare October 30, 2024 20:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bootstrap Web Discovery Native, add credential manager and server config loading #24969

Bootstrap Web Discovery Native, add credential manager and server config loading #24969

DJAndries commented Aug 1, 2024

brave-builds commented Aug 2, 2024

fmarier left a comment

deeppandya Aug 5, 2024

deeppandya left a comment

goodov Aug 5, 2024

bridiver Aug 6, 2024

goodov Aug 7, 2024

bridiver Aug 8, 2024 •

edited

Loading

goodov Aug 5, 2024

DJAndries Aug 8, 2024 •

edited

Loading

goodov Aug 8, 2024

DJAndries Aug 8, 2024

goodov Aug 5, 2024

bridiver Aug 8, 2024

DJAndries Aug 8, 2024 •

edited

Loading

bridiver Aug 6, 2024

bridiver Aug 6, 2024

DJAndries Aug 8, 2024

bridiver Aug 19, 2024

DJAndries Aug 23, 2024

bridiver Aug 6, 2024

DJAndries Aug 8, 2024

bridiver Aug 8, 2024

DJAndries Aug 8, 2024

brave-builds commented Aug 8, 2024

goodov Aug 8, 2024

brave-builds commented Aug 8, 2024

brave-builds commented Oct 25, 2024

Bootstrap Web Discovery Native, add credential manager and server config loading #24969

Are you sure you want to change the base?

Bootstrap Web Discovery Native, add credential manager and server config loading #24969

Conversation

DJAndries commented Aug 1, 2024

Submitter Checklist:

Reviewer Checklist:

After-merge Checklist:

Test Plan:

brave-builds commented Aug 2, 2024

fmarier left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deeppandya left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bridiver Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DJAndries Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DJAndries Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brave-builds commented Aug 8, 2024

Choose a reason for hiding this comment

brave-builds commented Aug 8, 2024

brave-builds commented Oct 25, 2024

bridiver Aug 8, 2024 •

edited

Loading

DJAndries Aug 8, 2024 •

edited

Loading

DJAndries Aug 8, 2024 •

edited

Loading