upgrade bazel buildfarm to v2.7.0 #12
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Avoid dependency on subscriber to update state changes when removing workers. This prevents an NPE which will occur invariably when workers are allowably configured with subscribeToBackplane: false.
The individual metric controls for ttl are not necessary either for performance or feature support. Use Files' attributes acquisition mechanism for modified time.
Java logging definitions must now match java.util.logging.config.file, update these specifications in our README.md
Use interrupt to halt the OperationQueuer. Join on both operationQueuer and dispatchMonitor before instance stop return.
Include Match and Report Result stages in output Record the active operationName occupying slots in each of the stages and present them with WorkerProfile Avoid several unnecessary casts with interfaces for operation slot stages.
A shard server is impractical without operation subscription, partition subscription confirmation between servers and workers. The failsafe execution is configuration that is likely not desired on workers. This change removes the failsafe behavior from workers via backplane config, and relegates the setting of failsafe boolean to server config. If the option is restored for workers, it can be added to worker configs so that configs may continue to be shared between workers and servers and retain independent addressability.
Internally driven metrics and scaling controls have low, if any, usage rates. Prometheus has largely succeeded independent publication of metrics, and externally driven scaling is the norm. These modules have been incomplete between cloud providers, and for the functional side of AWS, bind us to springboot. Removing them for the sake of reduced dependencies and complexity.
Remove this unused OperationQueue feature which provides no invocations on any use.
This reverts commit f651cdb.
Continue the loop while we have *not* matched successfully and avoid a confusing inversion in getMatched()
Distinguish the valid/unique/propagating methods of entry listening.
The only signal to a waiting match that will halt its current listen loop for a valid unique operation is an interrupt.
Distinguish target param with GRPC type storage from FILESYSTEM definition
Reinstate prior usage of LoggingMain for safe shutdown, with added release mechanism for interrupted processes. All invoked shutdowns are graceful, with vastly improved shutdown speed for empty workers waiting for pipeline stages.
Tiny code cleanup
Will include operation root and inform directory cache effectiveness.
Selecting realInputDirectories by regex permits flexible patterns that can yield drastic improvements in directory reuse for specialized deployments. runfiles in particular are hazardous expansions of nearly-execroot in the case of bazel. Care must be taken to match directories exclusively. The entire input tree is traversed for matches against expanded paths under the root, to allow for nested selection. Each match thus costs the number of input directories. Counterintuitively, OutputFiles are augmented to avoid the recursive check for OutputDirectories which only applies to actual reported results, resulting in a path match when creating the exec root. Regex style is java.util.Pattern, and must match the full input directory.
This will include the path to the missed directory and the operation which required it.
Prevent adding duplicate realInputDirectories matches
Ensure that the last leg of the execution presents a directory, rather than the parent, per OutputDirectory's stamping.
Support a `--redis_uri` command line option for start-up.
also fixed some spelling typos.
Bumping 4.2 -> 5.3
Bump fro 0.0.6 -> 0.0.9
Don't get transitive grpc dependencies, use the ones from our `maven_install(...)`
To aid connection troubleshooting
buildfarm#1486) * Add unit test * Signal Write on complete
Closes buildfarm#1508 Cleanups: - remove the unused `ubuntu-bionic` base image - replace `ubuntu-jammy:jammy-java11-gcc` with `ubuntu-mantic:mantic-java17-gcc` - replace `amazoncorretto:19` with `ubuntu-mantic:mantic-java17-gcc` - swap inverted log file names in a file
Specifying any number of OutputPaths will ignore OutputFiles (consistent with uploads). Where an OutputPath specifies an output directory, the action must be able to create the directory itself.
Partial specification of the absolute symlink response per REAPI. Remaining work will be in output identification.
Trying to get more info on the Lombok stamping issue on Windows CI. See also bazelbuild/bazel#10363 and bazelbuild/bazel#18185
Default disabled, available with createSymlinkOutputs option in Worker config.
Problem
Enabling the findMissingBlobsViaBackplane flag in BuildfarmServer eliminates the need for the BuildfarmWorker's fmb API call. This BuildfarmWorker:fmb call was also responsible for tracking CAS entry access. As result, our CAS cache eviction strategy shifted from LRU to FIFO.
When the findMissingBlobsViaBackplane flag is enabled, the buildfarm relies on the backplane as the definitive source for CAS availability. Since we don't update CAS expiry on each access, the backplane will independently expire CAS entries based on the specified cas_expire duration, even if they are actively being read.
Solution
Updated bfServer:fmb call to perform non-blocking fmb calls to workers, allowing these workers to record access for the relevant CAS entries.
Extended expiry duration for available CAS entries in the backplane on each fmb call.
With these changes, we can utilize Bazel's experimental_remote_cache_lease_extension and experimental_remote_cache_ttl flags for incremental builds.
Closes buildfarm#1428
…m#1516) Bumps [org.json:json](https://github.com/douglascrockford/JSON-java) from 20230227 to 20231013. - [Release notes](https://github.com/douglascrockford/JSON-java/releases) - [Changelog](https://github.com/stleary/JSON-java/blob/master/docs/RELEASES.md) - [Commits](https://github.com/douglascrockford/JSON-java/commits) --- updated-dependencies: - dependency-name: org.json:json dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…remove-custom-spring-exit
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.