Document node's store structure #1603
Conversation
|
Caution Review failedThe pull request is closed. WalkthroughThe recent updates enhance the documentation for the Celestia node by adding a sidebar link titled "Datastore structure" for improved navigation. This entry links to the path "how-to-guides/celestia-node-store-structure," providing users with a reference to the datastore structure related to the Celestia node. Changes
Assessment against linked issues
Possibly related PRs
Suggested labels
Poem
Warning Rate limit exceeded@jcstein has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 11 minutes and 4 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
@coderabbitai review |
Actions performedReview triggered.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Outside diff range and nitpick comments (1)
nodes/celestia-node-store-structure.md (1)
34-34: Ensure the file ends with a single newline character.Please add a newline at the end of the file to comply with Markdown best practices.
Tools
Markdownlint
34-34: null (MD047, single-trailing-newline)
Files should end with a single newline character
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- .vitepress/config.ts (1 hunks)
- nodes/celestia-node-store-structure.md (1 hunks)
Files skipped from review due to trivial changes (1)
- .vitepress/config.ts
Additional context used
LanguageTool
nodes/celestia-node-store-structure.md
[uncategorized] ~14-~14: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...tained in the node's store: -/blocks: This directory stores blocks. Each file...
[uncategorized] ~15-~15: “It’s” is the contracted form of “it is”. Did you mean to use the possessive determiner “its”? (AI_HYDRA_LEO_SCY_IT_S)
Context: ...esents a block on Celestia and contains it's associated data. -/data: This direc...
[uncategorized] ~17-~17: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...ontains it's associated data. -/data: This directory contains the block heade...
[uncategorized] ~19-~19: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...ins the block header files. -/config: This directory stores the files used to...
[uncategorized] ~22-~22: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...ing configuration files: -app.toml: The configuration file to define applic...
[uncategorized] ~24-~24: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...d performance tuning. -client.toml: used to configure parameters for client...
[uncategorized] ~26-~26: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ... endpoints or timeouts. -config.toml: This is the node's primary configuratio...
[uncategorized] ~29-~29: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...the node's identity. -inverted_index: This file stores indexed data that can ...
[uncategorized] ~29-~29: Possible missing preposition found. (AI_HYDRA_LEO_MISSING_TO)
Context: ...le stores indexed data that can be used make blockchain queries. -/keys: This di...
[uncategorized] ~31-~31: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...used make blockchain queries. -/keys: This directory stores the cryptographic...
[uncategorized] ~33-~33: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...ed to operate the node. -/transients: This directory stores temporary data su...
Markdownlint
nodes/celestia-node-store-structure.md
34-34: null (MD047, single-trailing-newline)
Files should end with a single newline character
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Hi, I followed the draft in the linked issue as a guide. Is there somewhere else I could reference? |
yeah, I remember writing this but don't know where the extra directories and files are from. Are there more with full and bridge nodes? @Mackenzie-OO7 you can figure out the node store structure by initializing each node type $ tree .celestia-light
.celestia-light
├── config.toml
├── data
│ ├── 000002.vlog
│ ├── 00022.mem
│ ├── 001174.sst
│ ├── DISCARD
│ ├── KEYREGISTRY
│ ├── LOCK
│ └── MANIFEST
└── keys
├── NJ3XILLTMVRXEZLUFZVHO5A
├── OAZHALLLMV4Q
└── keyring-test
├── 968218...32d7c.address
└── my_celes_key.info
4 directories, 129 files |
Ohh. Thanks!😅 |
|
Hi @jcstein could you explain what |
|
hey, @distractedm1nd could you please help summarize what these are? thanks! |
|
Belongs to badger, I have a general idea of what LOCK does but not the others - but i don't recommend describing these any further than "Various files belonging to Node LSM storage system" |
|
Awesome, thanks @distractedm1nd. let's use "Various files belonging to Node LSM storage system" for all of those @Mackenzie-OO7 |
|
Kind bump here @Mackenzie-OO7 !
|
There was a problem hiding this comment.
Actionable comments posted: 5
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- .vitepress/config.ts (1 hunks)
- nodes/celestia-node-store-structure.md (1 hunks)
Additional context used
LanguageTool
nodes/celestia-node-store-structure.md
[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: -/blocks: This directory stores blocks. Each file...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~15-~15: Possible missing comma found.
Context: ... node's datastore does not contain this directory as light nodes do not store blocks. - ...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. -/data: This directory contains block headers a...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...KEYREGISTRY, andMANIFEST. -/index`: This directory stores the index files t...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~19-~19: Possible missing comma found.
Context: ...ndex files that handle mapping specific keys such as block heights, to the correspon...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. -/inverted_index: This directory stores the inverted inde...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. -/keys: This directory stores the cryptographic...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. -/transients: This directory contains temporary data ...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. -config.toml: This is the node's primary configuratio...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~28-~28: Possible missing comma found.
Context: ...ration file. It defines the node's core settings such as the network parameters.(AI_HYDRA_LEO_MISSING_COMMA)
Additional comments not posted (2)
nodes/celestia-node-store-structure.md (1)
2-2: Correct grammar in the description.Ensure grammatical accuracy by using "its" for possessive form.
.vitepress/config.ts (1)
459-462: Sidebar entry addition looks good.The new sidebar entry for "Datastore structure" is correctly added and consistent with the existing structure.
There was a problem hiding this comment.
Actionable comments posted: 0
Outside diff range, codebase verification and nitpick comments (8)
nodes/celestia-node-store-structure.md (8)
14-14: Add a colon for clarity.Add a colon after "datastore" for better readability.
- The following are the directories and files found in the datastore + The following are the directories and files found in the datastore:Tools
LanguageTool
[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: -/blocks: This directory stores blocks. Each file...(UNLIKELY_OPENING_PUNCTUATION)
14-15: Ensure consistency in punctuation.Ensure consistent use of punctuation at the end of each bullet point. Consider adding a period at the end of the sentence for consistency.
- represents a block on Celestia and contains its associated data. However, the light node's datastore does not contain this directory, as light nodes do not store blocks + represents a block on Celestia and contains its associated data. However, the light node's datastore does not contain this directory, as light nodes do not store blocks.Tools
LanguageTool
[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: -/blocks: This directory stores blocks. Each file...(UNLIKELY_OPENING_PUNCTUATION)
17-17: Add a comma for clarity.Add a comma before "such as" for better readability.
- and various files belonging to node LSM storage system such as `DISCARD`, `KEYREGISTRY`, and `MANIFEST`. + and various files belonging to node LSM storage system, such as `DISCARD`, `KEYREGISTRY`, and `MANIFEST`.Tools
LanguageTool
[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. -/data: This directory contains block headers a...(UNLIKELY_OPENING_PUNCTUATION)
19-19: Add a comma for clarity.Add a comma before "as they do not perform indexing" for better readability.
- does not include this directory, as they do not perform indexing. + does not include this directory, as they do not perform indexing.Tools
LanguageTool
[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...EYREGISTRY, andMANIFEST. -/index`: This directory stores the index files t...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~19-~19: Possible missing comma found.
Context: ...ndex files that handle mapping specific keys such as block heights, to the correspon...(AI_HYDRA_LEO_MISSING_COMMA)
21-21: Add a comma for clarity.Add a comma before "such as" for better readability.
- and various files belonging to node LSM storage system, such as `DISCARD`, `KEYREGISTRY`, `LOCK`, and `MANIFEST`. + and various files belonging to node LSM storage system, such as `DISCARD`, `KEYREGISTRY`, `LOCK`, and `MANIFEST`.Tools
LanguageTool
[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. -/inverted_index: This directory stores the inverted inde...(UNLIKELY_OPENING_PUNCTUATION)
23-23: Ensure consistent punctuation.Ensure consistent use of punctuation at the end of each bullet point. Consider adding a period at the end of the sentence for consistency.
- This directory stores the cryptographic key pairs that are used to operate the node + This directory stores the cryptographic key pairs that are used to operate the node.Tools
LanguageTool
[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. -/keys: This directory stores the cryptographic...(UNLIKELY_OPENING_PUNCTUATION)
25-25: Ensure consistent punctuation.Ensure consistent use of punctuation at the end of each bullet point. Consider adding a period at the end of the sentence for consistency.
- that are used while the node is operating, but are not a part of the permanent blockchain state + that are used while the node is operating, but are not a part of the permanent blockchain state.Tools
LanguageTool
[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. -/transients: This directory contains temporary data ...(UNLIKELY_OPENING_PUNCTUATION)
28-28: Ensure consistent punctuation.Ensure consistent use of punctuation at the end of each bullet point. Consider adding a period at the end of the sentence for consistency.
- It defines the node's core settings, such as the network parameters + It defines the node's core settings, such as the network parameters.Tools
LanguageTool
[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. -config.toml: This is the node's primary configuratio...(UNLIKELY_OPENING_PUNCTUATION)
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- nodes/celestia-node-store-structure.md (1 hunks)
Additional context used
LanguageTool
nodes/celestia-node-store-structure.md
[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: -/blocks: This directory stores blocks. Each file...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. -/data: This directory contains block headers a...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...EYREGISTRY, andMANIFEST. -/index`: This directory stores the index files t...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~19-~19: Possible missing comma found.
Context: ...ndex files that handle mapping specific keys such as block heights, to the correspon...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. -/inverted_index: This directory stores the inverted inde...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. -/keys: This directory stores the cryptographic...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. -/transients: This directory contains temporary data ...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. -config.toml: This is the node's primary configuratio...(UNLIKELY_OPENING_PUNCTUATION)
Additional comments not posted (2)
nodes/celestia-node-store-structure.md (2)
1-3: LGTM!The description is clear and grammatically correct.
5-10: LGTM!The introduction provides a clear and concise overview of the node's datastore.
|
let's close this and rework it into the new structure. wdyt? also after reviewing, I think a lot of what is maybe AI generated here needs to go. many of these aren't in my node store... |
.vitepress/config.ts
Outdated
| { | ||
| text: "Datastore structure", | ||
| link: "nodes/celestia-node-store-structure", | ||
| }, |
There was a problem hiding this comment.
Update the link path to follow documentation conventions.
The link path nodes/celestia-node-store-structure doesn't follow the established pattern used throughout the documentation (which typically starts with /how-to-guides/). This could lead to broken links.
Apply this diff to align with the documentation structure:
{
text: "Datastore structure",
- link: "nodes/celestia-node-store-structure",
+ link: "/how-to-guides/celestia-node-store-structure",
},📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| { | |
| text: "Datastore structure", | |
| link: "nodes/celestia-node-store-structure", | |
| }, | |
| { | |
| text: "Datastore structure", | |
| link: "/how-to-guides/celestia-node-store-structure", | |
| }, |
so initially I kinda mixed up the files & directories that I listed with the ones in celestia-app. but I changed that already, and now the directories listed here are all in the celestia-node store.
The directories you don't have here are in the bridge and full nodes |
|
makes sense! thank you for clarifying |
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Outside diff range and nitpick comments (2)
how-to-guides/celestia-node-store-structure.md (2)
7-10: Consider mentioning node types in the introductionSince the datastore structure varies between light, bridge, and full nodes, it would be helpful to mention this upfront in the introduction.
The node's datastore refers to the storage structure used to manage the data that supports the node's operation. It consists of directories and files that contain the node's state, -configuration, and other information relevant to the node. +configuration, and other information relevant to the node. The structure +varies depending on the node type (light, bridge, or full node).
14-15: Improve clarity of blocks directory descriptionConsider adding more context about why light nodes don't store blocks.
-represents a block on Celestia and contains its associated data. This directory is present in the datastore for bridge and full nodes but not light nodes, as light nodes do not store blocks. +represents a block on Celestia and contains its associated data. This directory is present in the datastore for bridge and full nodes but not light nodes, as light nodes only verify headers and don't need to store full blocks.🧰 Tools
🪛 LanguageTool
[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: -/blocks: This directory stores blocks. Each file...(UNLIKELY_OPENING_PUNCTUATION)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (2)
- .vitepress/config.ts (1 hunks)
- how-to-guides/celestia-node-store-structure.md (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- .vitepress/config.ts
🧰 Additional context used
🪛 LanguageTool
how-to-guides/celestia-node-store-structure.md
[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: -/blocks: This directory stores blocks. Each file...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. -/data: This directory contains block headers a...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...EYREGISTRY, andMANIFEST. -/index`: This directory stores the index files t...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. -/inverted_index: This directory stores the inverted inde...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. -/keys: This directory stores the cryptographic...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. -/transients: This directory contains temporary data ...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. -config.toml: This is the node's primary configuratio...(UNLIKELY_OPENING_PUNCTUATION)
🔇 Additional comments (1)
how-to-guides/celestia-node-store-structure.md (1)
12-28: Verify directory structure completenessAs suggested by @jcstein in the PR discussion, it would be valuable to verify this documentation against the actual directory structure created by initializing different node types (e.g.,
celestia bridge init,celestia light init).🧰 Tools
🪛 LanguageTool
[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: -/blocks: This directory stores blocks. Each file...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. -/data: This directory contains block headers a...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...EYREGISTRY, andMANIFEST. -/index`: This directory stores the index files t...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. -/inverted_index: This directory stores the inverted inde...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. -/keys: This directory stores the cryptographic...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. -/transients: This directory contains temporary data ...(UNLIKELY_OPENING_PUNCTUATION)
[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. -config.toml: This is the node's primary configuratio...(UNLIKELY_OPENING_PUNCTUATION)
|
didn't fully resolve in that PR! 😅 |
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Overview
closes #1341
Summary by CodeRabbit