HowTo topics for Kafka tiered storage #2162
Conversation
Deploying with
|
| Latest commit: |
69f2489
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://de991e48.devportal.pages.dev |
| Branch Preview URL: | https://harshini-kafka-tiered-storag-3es7.devportal.pages.dev |
| 4. Select **True** to activate tiered storage for the topic. | ||
|
|
||
| .. note:: | ||
| If you leave the value as **Default**, it implies that tiered storage is enabled for this topic. |
There was a problem hiding this comment.
I thought the default was not to use it, interesting.
There was a problem hiding this comment.
I just tried setting Remote storage enable to Default, and it does nothing. But this is going to be an obvious question from the user. Why have Default in the drop-down when it does nothing? Is there a way to remove it?
For now, I will remove this note.
There was a problem hiding this comment.
Yeah i think so. Default in the topic configs means "not set", meaning it's as if it was never touched by user. (Maybe it should be "Not set" instead of "Default" 🤔) In this case, as you need to explicitly enable remote storage per topic, the default is that it's not enabled. Does that make sense?
| - If you are creating a new service: | ||
|
|
||
| a. On the **Create Apache Kafka® service** page, scroll down to the **Tiered storage** section. | ||
| b. To enable tiered storage, select the **Enable tiered storage** toggle. |
There was a problem hiding this comment.
Should we mention that the price can be seen in the overview section?
|
|
||
| Aiven for Apache Kafka®'s tiered storage expands storage beyond local disks. It stores frequently accessed data on faster tiers and less active data on cost-effective, slower tiers, ensuring both performance and cost efficiency are optimized. | ||
|
|
||
| For an in-depth understanding of tiered storage, how it works, and its benefits, see `Tiered Storage in Aiven for Apache Kafka®`. |
| View of the current configurations for tiered storage: | ||
|
|
||
| - **Local Cache**: Shows the current cache configuration. | ||
| - **Default Local Retention Time (ms)**: Shows the current local data retention set in milliseconds. |
There was a problem hiding this comment.
Wondering if these should mention the actual setting name? 🤔
There was a problem hiding this comment.
I used the name the user would see on the "Tiered storage overview" page. Do you mean add this - kafka.log_local_retention_bytes and kafka.log_local_retention_ms?
There was a problem hiding this comment.
Yes, those are the actual settings names. I'm just thinking that if for example you use the CLI, you would use the real setting names so it could be reassuring that we are in fact talking about the exact same service level settings.
I think we should just use the real setting name in console as well to avoid confusion.
There was a problem hiding this comment.
Very nice work, looks good! :) I did not tag all uses of local cache as i assume these will be removed in one batch.
I did not try the instructions yet in production but might be a good idea to do some click testing with people having console + dev docs open and see if they manage to do everything.
General question, do we need screenshots to help with steps?
As a general practice, we avoid screenshots due to maintenance issues. If the screen changes, we need to update them in the docs. Hence, we try to be detailed in the steps. |
|
|
||
| Aiven for Apache Kafka®'s tiered storage expands storage beyond local disks. It stores frequently accessed data on faster tiers and less active data on cost-effective, slower tiers, ensuring both performance and cost efficiency are optimized. | ||
|
|
||
| For an in-depth understanding of tiered storage, how it works, and its benefits, see `Tiered Storage in Aiven for Apache Kafka®`. |
There was a problem hiding this comment.
I guess it's the name of a section? I can't find it.
There was a problem hiding this comment.
This content is related to the concept topics in the PR - #2142. I'm unable to create direct links at the moment, as they would lead to broken links and prevent me from merging the content into the main branch.
Co-authored-by: Ivan Yurchenko <ivanyu@aiven.io>
There was a problem hiding this comment.
Adding suggestions to align with https://github.com/aiven/aiven-core/pull/40511 changes
Co-authored-by: Jorge Esteban Quilcate Otoya <jorge.quilcate@aiven.io>
Co-authored-by: Jorge Esteban Quilcate Otoya <jorge.quilcate@aiven.io>
Co-authored-by: Jorge Esteban Quilcate Otoya <jorge.quilcate@aiven.io>
Co-authored-by: Jorge Esteban Quilcate Otoya <jorge.quilcate@aiven.io>
What changed, and why it matters
Added HowTo topics for Kafka tiered storage (instructions for Aiven Console):