-
Notifications
You must be signed in to change notification settings - Fork 342
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set jaeger-spark-dependency date in operator #1068
Comments
It sounds reasonable, would you like to submit a PR?
It seems controversial as at 2-3 AM there should be less load. |
I'm not sure what you mean by this? Happy to submit a PR, I don't have a ton of Go experience but this change seems fairly simple so I'll git it a shot! |
I could not find a clean way to achieve this in the operator. Instead opened a PR to add this as feature in the Spark Dependencies repo here. Feedback is welcome! |
We should really use |
By default the jaeger-spark-dependency CronJob runs just before midnight. I have several large clusters sharing a single Jaeger instance so the CronJob uses a lot of resources. The problem I am facing is that ~12 AM UTC (cluster time) is not an ideal time to run a memory intensive job as the clusters are still heavily used at this time. I would like to move it to more like 2-3 AM the day after, but the job defaults to the current day and never finishes when I try this setup. I assume because it is reading the index of the current day which is growing faster than it can read the traces/spans.
The Jaeger Spark Dependency can be set to use a particular date using the
DATE
environment variable (docs) and they include some bash examples to dynamically get the previous day.It would be great if the Jaeger Operator exposed this setting or added an
env
field for arbitrary environment variables to be set.Thanks!
The text was updated successfully, but these errors were encountered: