-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial implementation of likelihood profiling #37
Merged
Merged
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
4ca65f4
Initial implementation of likelihood profiling (fix #1)
billdenney 6c540b8
Additional feature updates
billdenney f4503f4
Intermediate work
billdenney 612ca73
Merge branch 'main' into fix-1
billdenney fc6e1b5
Update documentation
billdenney b516473
Merge branch 'main' into fix-1
billdenney c7a3a63
Update documentation
billdenney 725830e
Fix check issues
billdenney 56ac86e
A much-improved work-in-progress. `profile.nlmixr2FitCore()` now wor…
billdenney f8d87f5
Stop estimation based on number of significant digits in estimated pa…
billdenney 589a622
Use the default RSE when it is not available
billdenney d3c5272
Rename rse_theta to rseTheta
billdenney 114ccf2
Update CI
billdenney 1a7b123
Remove `quiet` argument
billdenney 26b965a
Add more development version packages for CI
billdenney c482d93
correct dparser repo location
billdenney 4c1891d
Remove unused `ignoreBounds` argument
billdenney 5159186
Fix check issues
billdenney 98805e1
Remove unused `maxpts` argument
billdenney 3bfb925
Switch to a generalized profile method with `method` and `control` ar…
billdenney 8e5f79f
Ensure that initial estimates are within the limits for the parameter…
billdenney cd720f6
Test residual error profiling
billdenney 8e42770
Update documentation
billdenney 101d16a
Documentation fix
billdenney 1b705ca
Add omega values and covariances to profiling output
billdenney 509db7b
Add rxUiDeparse method for llpControl
mattfidler 64a4f43
Version bump
mattfidler b17bbe1
Update check config
mattfidler File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,149 @@ | ||
profile.nlmixr2FitData <- function(fitted, ..., which = NULL, maxpts = 10, ofvIncrease = 3.84, normQuantile = 1.96, rse_theta, tol = 0.001, ignoreBounds = FALSE, quiet = TRUE) { | ||
browser() | ||
if (is.null(which)) { | ||
which <- names(fixef(fitted)) | ||
} else { | ||
checkmate::assert_subset(x = which, choices = names(fixef(fitted)), empty.ok = FALSE) | ||
} | ||
if (missing(rse_theta)) { | ||
fittedVcov <- vcov(fitted) | ||
if (is.null(fittedVcov)) { | ||
# Handle when vcov() is an error | ||
rse_theta <- 30 | ||
cli::cli_alert_info(paste("covariance is unavailable, using default rse_theta of", rse_theta)) | ||
} else { | ||
sd_theta <- sqrt(diag(fittedVcov)) | ||
rse_theta <- 100*abs(sd_theta/fixef(fitted)[names(sd_theta)]) | ||
} | ||
} | ||
if (length(rse_theta) == 1 & is.null(names(rse_theta))) { | ||
rse_theta <- setNames(rep(rse_theta, length(which)), which) | ||
} else { | ||
checkmate::assert_names(names(rse_theta), subset.of = names(fixef(fitted))) | ||
checkmate::assert_numeric(rse_theta, lower = 0, any.missing = FALSE, min.len = 1) | ||
} | ||
if (quiet) { | ||
# TODO: set print = 0 for all the control arguments | ||
} | ||
if (length(which) > 1) { | ||
ret <- data.frame() | ||
# Make the general case be a concatenation of single cases | ||
for (currentWhich in which) { | ||
ret <- | ||
rbind( | ||
ret, | ||
profile.nlmixr2FitData(fitted = fitted, ..., which = currentWhich, maxpts = maxpts, ofvIncrease = ofvIncrease, normQuantile = normQuantile, rse_theta = rse_theta, tol = tol, quiet = quiet) | ||
) | ||
} | ||
} else { | ||
ret <- profileNlmixr2FitDataRet(fitted = fitted, which = which) | ||
for (direction in c(-1, 1)) { | ||
found <- FALSE | ||
boundCol <- ifelse(direction < 0, "lower", "upper") | ||
if (ignoreBounds) { | ||
bound <- Inf*direction | ||
} else { | ||
bound <- setNames(fitted$ui$iniDf[[boundCol]], fitted$ui$iniDf$name)[which] | ||
} | ||
if (abs(bound - ret[[which]][1]/ret[[which]][1]) < tol) { | ||
found <- TRUE | ||
cli::cli_warn( | ||
"parameter %s is close to the %s boundary and likelihood will not be profiled in that direction", | ||
which, boundCol | ||
) | ||
} | ||
currentEst <- ret[[which]][1] + direction*normQuantile*rse_theta[which]/100*abs(ret[[which]][1]) | ||
# Confirm that the initial estimate is within the bounds | ||
if (direction < 0) { | ||
currentEst <- max(bound, currentEst) | ||
} else { | ||
currentEst <- min(bound, currentEst) | ||
} | ||
currentMaxpts <- maxpts | ||
# Prepare the current parameter to be fixed for each estimation | ||
fittedFix <- do.call(ini, append(list(x=fitted), setNames(list(as.name("fix")), which))) | ||
while (currentMaxpts > 0 & !found) { | ||
currentMaxpts <- currentMaxpts - 1 | ||
newEst <- setNames(list(currentEst), which) | ||
# TODO: How can I detect the current estimation method from the fitted | ||
# object? Can you even do likelihood profiling with something other | ||
# than focei? | ||
newFit <- try(nlmixr(ini(fittedFix, newEst), est = "focei", control = list(print = 0))) | ||
if (inherits(newFit, "try-error")) { | ||
# TODO: Work around failed fits | ||
stop("Fit failed") | ||
} else { | ||
ret <- rbind(ret, profileNlmixr2FitDataRet(fitted = newFit, which = which, rowname = nrow(ret) + 1)) | ||
ret <- ret[order(ret[[which]]), ] | ||
currentEstList <- profileNlmixr2FitDataNewEst(estimates = ret, which = which, direction = direction, bound = bound, ofvIncrease = ofvIncrease, tol = tol) | ||
if (is.character(currentEstList$found)) { | ||
found <- TRUE | ||
cli::cli_alert_danger(sprintf("Stopping search for %s in %s direction because %s", which, boundCol, currentEstList$found)) | ||
} else { | ||
currentEst <- currentEstList$newEst | ||
found <- currentEstList$found | ||
} | ||
} | ||
} | ||
} | ||
} | ||
ret | ||
} | ||
|
||
#' Give the output data.frame for a single model for profile.nlmixr2FitData | ||
#' @inheritParams profile.nlmixr2FitData | ||
#' @return A data.frame with columns for Parameter (the parameter name), OFV | ||
#' (the objective function value), and the current estimate for each of the | ||
#' parameters | ||
#' @noRd | ||
profileNlmixr2FitDataRet <- function(fitted, which, rowname = 0) { | ||
if (any(names(fixef(fitted)) %in% c("Parameter", "OFV"))) { | ||
cli::cli_abort("Cannot profile a model with a parameter named either 'Parameter' or 'OFV'") | ||
} | ||
ret <- | ||
cbind( | ||
data.frame(Parameter = which, OFV = fitted$objDf$OBJF), | ||
data.frame(t(fixef(fitted))) | ||
) | ||
rownames(ret) <- as.character(rowname) | ||
ret | ||
} | ||
|
||
#' Generate a new estimate, and return a list indicating if the new estimate | ||
#' should not be run (because the current data find the value within the | ||
#' tolerance) | ||
#' | ||
#' @param estimates A data.frame with the parameter estimates | ||
profileNlmixr2FitDataNewEst <- function(estimates, which, direction, bound, ofvIncrease, tol, method = "linapprox") { | ||
minOFV <- min(estimates$OFV, na.rm = TRUE) | ||
minRow <- which(estimates$OFV %in% minOFV) | ||
maskDirection <- !is.na(estimates$OFV) & estimates[[which]] <= estimates[[which]][minRow] | ||
dOFV <- estimates$OFV[maskDirection] - minOFV | ||
# Estimates in the correct direction | ||
estDir <- estimates[[which]][maskDirection] | ||
if (all(dOFV < ofvIncrease)) { | ||
# Expand the search | ||
newEst <- estimates[[which]][minRow] + direction*2*abs(diff(range(estDir))) | ||
found <- FALSE | ||
} else { | ||
# Check that the OFV is monotonic | ||
monotonic <- length(unique(sign(diff(dOFV)/diff(estDir)))) == 1 | ||
if (!monotonic) { | ||
newEst <- NA | ||
found <- "OFV is not monotonic" | ||
} else if (method == "linapprox") { | ||
newEst <- approx(x = dOFV, y = estDir, xout = ofvIncrease)$y | ||
} else { | ||
cli::cli_abort(sprintf("method %s is not implemented", method)) | ||
} | ||
if (!is.na(newEst)) { | ||
estAbove <- min(estimates[[which]][estimates[[which]] > newEst]) | ||
estBelow <- min(estimates[[which]][estimates[[which]] < newEst]) | ||
found <- abs((estAbove - estBelow)/estBelow) < tol | ||
} | ||
} | ||
list( | ||
newEst = newEst, | ||
found = found | ||
) | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Bill (@billdenney),
Thanks for the initial implementation. Very interesting! I have a question to you: Your code only allows the profiling of fixed effect parameters. Why not profiling e.g. the variance-covariances?
Thanks in advance!
Simon
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no specific reason for that right now. Generally, profiling fixed effects is easier than random effects because random effects can be correlated. So, I think that you would need to detect correlation and only allow profiling of the random effects without correlations (or remove the correlation). @mattfidler, can you fix a random effect or its correlation when it is correlated without fixing the other parts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You have to fix the whole block
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was referring the omegas not the individual etas, since they are part of the marginal likelihood.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@simonbeyer1, yes, I understood that you were referring to omegas. The idea I was getting to is that if you specify an eta like
etacl + etavc ~ c(0.2, 0.1, 0.2)
, we would have to fix or not fix the entirety of that, all or none of thec(0.2, 0.1, 0.2)
part. I don't immediately have a good idea of how to profile correlated BSV, so that will remain out of scope for this initial version of profiling.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ahh okay! Now I got it! Thanks!