Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Flag to fully disable pubsub and all cross-node syncing #3286

Merged
merged 2 commits into from
Oct 15, 2024

Conversation

stbrody
Copy link
Contributor

@stbrody stbrody commented Oct 11, 2024

Flag can be enabled by setting ipfs.disablePeerDataSync in the config file, or with the new CERAMIC_DISABLE_PEER_DATA_SYNC env var.

turning on this flag does 3 things:

  1. Fully disable pubsub. The node will not subscribe to the ceramic pubsub topic, and will not publish or receive any pubsub messages
  2. Turns down the timeout when getting blocks from ipfs from 30 seconds to 1 second
  3. Uses the "offline" flag whenever loading blocks form ipfs so that the request will fail immediately if the block is not found in the ipfs node's local blockstore. IPFS will never try to load the block via bitswap.

Blocks published to ipfs with this flag enabled should still be fetchable by other ipfs nodes over bitswap.

@stbrody stbrody self-assigned this Oct 11, 2024
@stbrody stbrody marked this pull request as ready for review October 11, 2024 21:28
@stbrody stbrody requested review from gvelez17 and smrz2001 October 11, 2024 21:28
Copy link
Contributor

@christianlavoie christianlavoie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simpler than I would have expected.

@@ -498,7 +497,7 @@ export class Dispatcher {
* @param tip - Commit CID
*/
publishTip(streamId: StreamID, tip: CID, model?: StreamID): Subscription {
if (process.env.CERAMIC_DISABLE_PUBSUB_UPDATES == 'true' || EnvironmentUtils.useRustCeramic()) {
if (process.env.CERAMIC_DISABLE_PUBSUB_UPDATES == 'true' || !this.enableSync) {
return empty().subscribe()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the difference between CERAMIC_DISABLE_PEER_DATA_SYNC and CERAMIC_DISABLE_PUBSUB_UPDATES ?

I take it this means we could just not publish, or we could not publish and not read or sync with other nodes? but neither applies under rust ceramic?

If we were going to keep this i think it woudl be confusing but if we are going to leave js-ceramic entirely soon i guess it is fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CERAMIC_DISABLE_PUBSUB_UPDATES is much more limited in scope. It just disables publishing of update messages. We'll still listen to pubsub and still publish query and response messages.

Honestly it's not really very useful, we put it in once a while ago to try to troubleshoot an issue gitcoin was having, I don't think anyone uses it anymore and we could frankly probably get rid of it.

Copy link
Contributor

@gvelez17 gvelez17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one comment about possibly confusing option combinations, but given that we're trying to transition away anyway i think its fine

Copy link
Contributor

@smrz2001 smrz2001 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@stbrody stbrody enabled auto-merge (squash) October 15, 2024 19:27
@stbrody stbrody merged commit b8b44e1 into develop Oct 15, 2024
5 checks passed
@stbrody stbrody deleted the disable-pubsub branch October 15, 2024 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants