Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add information about proposed note previews #156

Merged
merged 5 commits into from
Sep 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added documentation/images/proposed-note-previews.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
35 changes: 19 additions & 16 deletions documentation/under-the-hood/timeline-tabs.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,38 +13,41 @@ Community Notes participants are able to see a tab of notes that could use their

This tab is only visible to contributors. It is designed to increase the likelihood that people from diverse perspectives rate each note, so that Community Notes can elevate notes that people from a wide range of perspectives will find helpful. It gives contributors an easy way to have immediate impact.

It contains a set of 10 posts that have notes that need more ratings (although there may be fewer than 10 posts if one of the posts was recently deleted by the author, or if not enough posts have new notes that meet the criteria to appear in the NYH tab). posts in this tab are filtered to those that the contributor hasn’t rated any of the notes on, and posts with notes from the past day, unless no posts pass those filters for you (that will only happen if you’re a very active rater!). The tab is updated roughly every hour or two, so when the contributor has rated notes in the tab, they can come back later to see fresh posts.
It contains a set of 10 posts that have notes that need more ratings (although there may be fewer than 10 posts if there are fewer that meet the [criteria](#criteria-for-needs-your-help-tab-alerts-and-note-proposal-previews) to appear). The tab is updated as new notes that meet the criteria are written.

In order to appear in any of the tabs in the Community Notes site, a post must have received at least 100 total likes plus reposts.
### Proposed note previews

Additionally, Community Notes offers a way for post authors to [request additional review](../contributing/additional-review.md) on notes on their posts. If an author requests additional review, the relevant post will appear in all contributors’ Needs Your Help tabs. If there are more than 10 active requests for additional review, posts will be sorted by by the ranking score described below, which incorporates the viewpoints of raters.
If a post has proposed notes, contributors will see a prompt to help rate them. Sometimes the prompt will show a preview of a proposed note. Previews are shown to help gather ratings from a wide range of people quickly, and are only shown some of the time, and only if a note meets certain [criteria](#criteria-for-needs-your-help-tab-alerts-and-note-proposal-previews).

![Graphic showing two screenshots of the X app. The one on the left shows the note proposal prompt without a note preview, and the one on the right shows the note proposal prompt with a preview of one proposed note](../images/proposed-note-previews.png)

Previews can appear anywhere you see proposed notes, for example in the Needs Your Help tab, in the Home timeline, etc.

### Needs your help alerts

To ensure contributors don't miss the opportunity to rate notes on posts getting a lot of attention, Community Notes sends alerts requesting help from time to time.

![One screenshot showing a Community Notes Contributor profile page, highlighting the settings button on the top right. Another screenshot showing the settings screen where contributors can edit their alert frequency](../images/alerts-settings.png)

### How does Community Notes decide which notes trigger alerts for help?
### Criteria for Needs Your Help tab, alerts, and proposed note previews

The Needs Your Help tab, alerts, and proposed note previews are designed to increase the chance that potentially helpful notes on posts with high predicted visibility get rated by enough people that they have the chance to earn a status of Helpful quickly. At the same time, they aim to gather enough ratings from a people of different points of view to identify when a proposed note is not broadly helpful enough to show.

Alerts are optimized to increase the chance that potentially helpful notes on posts with high predicted visibility get rated by enough people that they have the chance to earn a status of Helpful quickly. Here's our current approach:
A variety of criteria are considered when determining whether to include a note, notably:

**Posts are chosen based on:**
**Criteria related to the post on which the note is written:**

- Recency of the post (e.g. is it from within the last 6 hours, 24 hours, etc)
- Projected future Likes and Reposts the post will receive.
- The rater can see the post (for example, excludes posts from authors you've blocked)

**Notes are chosen based on meeting some of the following:**
**Criteria related to the note:**

- Written by an author with positive Writing Impact and a high ratio of Helpful notes (Writing Impact / Total Notes Written) or high average helpfulness score of notes they authored.
- Currently have a status of "Needs More Ratings"
- Currently have a high helpfulness score, nearing the threshold to earn status of "Helpful"
- Do not have a large number of ratings (such that more ratings could change the note's status)

Notes are sent to a random selection of contributors, excluding the note author and those who have already rated the note. Notifications are also limited by the recipient's notification frequency setting.

We will continue to experiment with this logic to help ensure Needs Your Help notifications feel high impact and satisfying for contributors.
- Current status (e.g. "Needs More Ratings")
- Current helpfulness score (e.g. not a low helpfulness score, highly rated by initial raters, possibly nearing the threshold to earn status of "Helpful")
- Does not have a large number of ratings (such that more ratings could change the note's status)

### Rater similarity score for Needs Your Help
In this case of alerts, notifications are sent to a random selection of contributors, excluding the note author and those who have already rated the note. Notifications are also limited by the recipient's notification frequency setting.

posts on the Needs Your Help tab are sorted by a ranking score, where posts are ranked higher based on the proportion of notes on the post that are labeled Needs More Rating. Posts are also ranked higher if they have notes with moderately high intercept scores (>=0.25) but which don’t have at least 3 raters with a similar viewpoint as yours (measured by whether your viewpoint factor is positive or negative). This is a mechanism to increase the likelihood that people from diverse perspectives rate each note.
We will continue to experiment with this logic to help ensure the Needs Your Help tab, alerts and proposed note previews feel high impact and satisfying for contributors.
Loading