Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restart execution enhancement #63

Closed
anthonysena opened this issue Apr 6, 2023 · 2 comments
Closed

Restart execution enhancement #63

anthonysena opened this issue Apr 6, 2023 · 2 comments

Comments

@anthonysena
Copy link
Collaborator

Scenario: you execute a Strategus analysis specification (with several modules) and find there is a problem with a module that causes it to fail. Ideally, we'd like to update the module reference and pass the same analysisSpecifications to the execute function with restart = TRUE and have the targets script only run the updated targets.

Currently, when Strategus' execute function is invoked, we write out the analysisSpecifications parameter to the file system as shown here:

Strategus/R/Execution.R

Lines 92 to 102 in a12dee1

fileName <- file.path(executionScriptFolder, "script.R")
if (restart) {
return(fileName)
}
# Store settings objects in the temp folder so they are available in targets
analysisSpecificationsFileName <- gsub("\\\\", "/", file.path(executionScriptFolder, "analysisSpecifications.rds"))
saveRDS(analysisSpecifications, analysisSpecificationsFileName)
executionSettingsFileName <- gsub("\\\\", "/", file.path(executionScriptFolder, "executionSettings.rds"))
saveRDS(executionSettings, executionSettingsFileName)
keyringSettingsFileName <- gsub("\\\\", "/", file.path(executionScriptFolder, "keyringSettings.rds"))
saveRDS(list(keyringName = keyringName), keyringSettingsFileName)

The challenge here (I believe) is that in the internal targets folder where targets tracks dependencies, the analysisSpecifications object it manages internally remains unchanged and so targets continues to use the older specification in place of the newer one. It may be possible to restructure the way in which we create the targets so that we have an analysisSpecifications per module such that if a module's specification is updated, targets can restart and only run updated targets.

@anthonysena
Copy link
Collaborator Author

This feature of the targets package may be useful here when restarting: https://docs.ropensci.org/targets/reference/tar_cue.html

@anthonysena anthonysena added sos-challenge Issues raised during the Save Our Sisyphus challenge and removed sos-challenge Issues raised during the Save Our Sisyphus challenge labels Oct 2, 2023
@anthonysena anthonysena added this to the v0.1.1 milestone Dec 4, 2023
@anthonysena anthonysena modified the milestones: v0.2.x, v0.3.x Jan 26, 2024
@anthonysena
Copy link
Collaborator Author

Closing this as this scenario is better handled by an approach like the one described in #128 and would add more complexity to the existing v0.x line.

@anthonysena anthonysena removed this from the v0.3.x milestone May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant