Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: recover from configuration failures #151

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Nelsonochoam
Copy link
Contributor

Description of changes:
Enables CDA to recover from certificateAuthority configuration failures with requiring a manual restart.

Why is this change necessary:
Without this when a customer provides a bad config the component will error and can't recover without un installing and re installing

How was this change tested:
Integration test

@github-actions
Copy link

github-actions bot commented Oct 5, 2022

Unit Tests Coverage Report

File Coverage Lines Branches
All files 69% 76% 61%
com.aws.greengrass.clientdevices.auth.infra.NetworkState 36% 36% 0%
com.aws.greengrass.clientdevices.auth.infra.NetworkState$ConnectionState 100% 100% 0%
com.aws.greengrass.clientdevices.auth.infra.NetworkState$1 20% 20% 0%
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils 78% 82% 74%
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils$Operation 100% 100% 0%
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils$Resource 100% 100% 0%
com.aws.greengrass.clientdevices.auth.CertificateManager 77% 91% 63%
com.aws.greengrass.clientdevices.auth.ClientDevicesAuthService 85% 93% 76%
com.aws.greengrass.clientdevices.auth.DeviceAuthClient 70% 76% 64%
com.aws.greengrass.clientdevices.auth.iot.registry.ThingRegistry 79% 86% 71%
com.aws.greengrass.clientdevices.auth.iot.registry.RegistryConfig 0% 0% 0%
com.aws.greengrass.clientdevices.auth.iot.registry.CertificateRegistry$1 75% 100% 50%
com.aws.greengrass.clientdevices.auth.iot.registry.CertificateRegistry 100% 100% 0%
com.aws.greengrass.clientdevices.auth.iot.registry.ThingRegistry$1 75% 100% 50%
com.aws.greengrass.clientdevices.auth.certificate.ClientCertificateGenerator 96% 93% 100%
com.aws.greengrass.clientdevices.auth.certificate.CertificateHelper 73% 92% 54%
com.aws.greengrass.clientdevices.auth.certificate.CertificateHelper$ProviderType 100% 100% 0%
com.aws.greengrass.clientdevices.auth.certificate.CertificateStore 64% 77% 52%
com.aws.greengrass.clientdevices.auth.certificate.CertificateExpiryMonitor 77% 88% 67%
com.aws.greengrass.clientdevices.auth.certificate.ServerCertificateGenerator 95% 90% 100%
com.aws.greengrass.clientdevices.auth.certificate.CertificateGenerator 70% 90% 50%
com.aws.greengrass.clientdevices.auth.certificate.CertificateStore$CAType 100% 100% 0%
com.aws.greengrass.clientdevices.auth.certificate.CertificateExpiryMonitor$CertRotationDecider 90% 100% 80%
com.aws.greengrass.clientdevices.auth.certificate.CertificatesConfig 100% 100% 100%
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyIotCertificate 57% 63% 50%
com.aws.greengrass.ipc.IPCUtils 58% 67% 50%
com.aws.greengrass.ipc.VerifyClientDeviceIdentityOperationHandler 64% 79% 50%
com.aws.greengrass.ipc.GetClientDeviceAuthTokenOperationHandler 87% 98% 75%
com.aws.greengrass.ipc.AuthorizeClientDeviceActionOperationHandler 80% 93% 67%
com.aws.greengrass.ipc.SubscribeToCertificateUpdatesOperationHandler 77% 87% 67%
com.aws.greengrass.clientdevices.auth.session.SessionConfig 92% 100% 83%
com.aws.greengrass.clientdevices.auth.session.SessionManager$1 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.MqttSessionFactory 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.SessionCreator 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.SessionManager 88% 100% 75%
com.aws.greengrass.clientdevices.auth.session.SessionImpl 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.SessionCreator$SessionFactorySingleton 100% 100% 0%
com.aws.greengrass.clientdevices.auth.session.MqttSessionFactory$MqttCredential 100% 100% 0%
com.aws.greengrass.clientdevices.auth.certificate.handlers.CACertificateChainChangedHandler 100% 100% 0%
com.aws.greengrass.clientdevices.auth.certificate.handlers.CAConfigurationChangedHandler 100% 100% 100%
com.aws.greengrass.clientdevices.auth.certificate.handlers.CertificateRotationHandler 96% 91% 100%
com.aws.greengrass.clientdevices.auth.certificate.usecases.ConfigureManagedCertificateAuthority 77% 77% 0%
com.aws.greengrass.clientdevices.auth.certificate.usecases.ConfigureCustomCertificateAuthority 75% 75% 0%
com.aws.greengrass.clientdevices.auth.certificate.usecases.RegisterCertificateAuthorityUseCase 59% 68% 50%
com.aws.greengrass.clientdevices.auth.configuration.AuthorizationPolicyStatement$Effect 100% 100% 0%
com.aws.greengrass.clientdevices.auth.configuration.GroupManager 89% 94% 83%
com.aws.greengrass.clientdevices.auth.configuration.ConfigurationFormatVersion 100% 100% 0%
com.aws.greengrass.clientdevices.auth.configuration.CAConfiguration 94% 98% 89%
com.aws.greengrass.clientdevices.auth.configuration.RuntimeConfiguration 100% 100% 0%
com.aws.greengrass.clientdevices.auth.configuration.CDAConfiguration 100% 100% 100%
com.aws.greengrass.clientdevices.auth.configuration.GroupDefinition 75% 100% 50%
com.aws.greengrass.clientdevices.auth.configuration.ExpressionVisitor 84% 94% 75%
com.aws.greengrass.clientdevices.auth.configuration.GroupConfiguration 90% 95% 86%
com.aws.greengrass.clientdevices.auth.api.ClientDevicesAuthServiceApi 81% 78% 83%
com.aws.greengrass.clientdevices.auth.api.Result 54% 83% 25%
com.aws.greengrass.clientdevices.auth.api.DomainEvents 100% 100% 100%
com.aws.greengrass.clientdevices.auth.api.UseCases 69% 88% 50%
com.aws.greengrass.clientdevices.auth.api.DomainEvent 100% 100% 0%
com.aws.greengrass.clientdevices.auth.api.GetCertificateRequestOptions$CertificateType 100% 100% 0%
com.aws.greengrass.clientdevices.auth.api.Result$Status 100% 100% 0%
com.aws.greengrass.clientdevices.auth.connectivity.usecases.GetConnectivityInformationUseCase 100% 100% 0%
com.aws.greengrass.clientdevices.auth.connectivity.usecases.RecordConnectivityChangesUseCase 100% 100% 100%
com.aws.greengrass.clientdevices.auth.session.attribute.StringLiteralAttribute 100% 100% 0%
com.aws.greengrass.clientdevices.auth.session.attribute.WildcardSuffixAttribute 88% 100% 75%
com.aws.greengrass.clientdevices.auth.util.ResizableLinkedBlockingQueue 90% 80% 100%
com.aws.greengrass.clientdevices.auth.util.ParseIPAddress 90% 95% 84%
com.aws.greengrass.clientdevices.auth.certificate.events.CACertificateChainChanged 100% 100% 0%
com.aws.greengrass.clientdevices.auth.iot.Certificate$Status 100% 100% 0%
com.aws.greengrass.clientdevices.auth.iot.IotAuthClient$Default 69% 72% 67%
com.aws.greengrass.clientdevices.auth.iot.Thing 89% 92% 86%
com.aws.greengrass.clientdevices.auth.iot.Certificate 92% 92% 0%
com.aws.greengrass.clientdevices.auth.iot.Component 100% 100% 0%
com.aws.greengrass.clientdevices.auth.connectivity.CISShadowMonitor 61% 77% 46%
com.aws.greengrass.clientdevices.auth.connectivity.RecordConnectivityChangesResponse 100% 100% 100%
com.aws.greengrass.clientdevices.auth.connectivity.HostAddress 67% 67% 0%
com.aws.greengrass.clientdevices.auth.connectivity.RecordConnectivityChangesRequest 100% 100% 0%
com.aws.greengrass.clientdevices.auth.connectivity.ConnectivityInformation 100% 100% 100%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionConstants 100% 100% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.TokenMgrError 22% 32% 12%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionTokenManager 61% 65% 58%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTStart 33% 33% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTAnd 67% 67% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.Token 58% 58% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionDefaultVisitor 0% 0% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTOr 67% 67% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.SimpleCharStream 28% 31% 25%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionTreeConstants 0% 0% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.JJTRuleExpressionState 67% 65% 70%
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTThing 67% 67% 0%
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpression 63% 63% 62%
com.aws.greengrass.clientdevices.auth.configuration.parser.SimpleNode 27% 35% 19%

Minimum allowed coverage is 50%

Generated by 🐒 cobertura-action against d88c097

@Nelsonochoam Nelsonochoam marked this pull request as ready for review October 5, 2022 17:13
private CDAConfiguration cdaConfiguration;
private volatile boolean configurationErrored = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is necessary at all, but future reference, you could just synchronize on configurationErrored instead and then you also don't need volatile

Comment on lines +146 to +147
// service to error, but we don't want to check again until the nucleus has run the remediation steps (when the
// service errors, the nucleus will try to call shutdown -> install -> startup).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you help me understand this? My mental model is that we shouldn't need to care about what Nucleus is doing, and that we can rely entirely on configuration updates. If we are broken, and a configuration change fixes us, then request reinstall? Are there scenarios where that will not work?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it should be possible to make this as simple as, on config change, if state is broken, then request reinstall.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbutler and I chatted about this offline - we are going to dig deeper and see if it is required

Comment on lines +219 to +224
synchronized (configLock) {
if (configurationErrored) {
configurationErrored = false;
onConfigurationChanged();
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the issue was that if we are broken, then Nucleus doesn't call startup?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants