Could not find SunPKCS11-NSS-FIPS provider for FIPS mode

[WilburZjh]

Related to the issue 15656.

[WilburZjh]

The issue occurs because it cant find the SunPKCS11-NSS-FIPS provider for FIPS mode after successfully enabling FIPS in the SecureRandom.java file.

if (FIPSConfigurator.enableFIPS()) {
            Provider p = Security.getProvider("SunPKCS11-NSS-FIPS");
            prng = "PKCS11";
            if (p == null) {
                throw new RuntimeException("could not find SunPKCS11-NSS-FIPS provider for FIPS mode");
            }

From the grinder, this test passed the first time with run testng ReflectionFactoryTest
But it failed on the second time with run testng/othervm/policy=security.policy ReflectionFactoryTest

First, I can create the SecureRandom with SunPKCS11-NSS-FIPS outside of this test in FIPS mode with the following simple code.

import java.security.SecureRandom;

public class SecureRandomTest {
	public static void main(String[] args) {
		SecureRandom random = new SecureRandom();
		System.out.println(random.getProvider());
	}
}

Therefore, it should be a policy configuration problem.

[WilburZjh]

This Could not find SunPKCS11-NSS-FIPS provider for FIPS mode issue can be reproduced by replacing ${JAVA_HOME}/jre/lib/security/java.policy with the content in jdk/test/sun/reflect/ReflectionFactory/security.policy by running the following command.
${JAVA_HOME}/bin/java -Dsemeru.fips=true -Djava.security.manager -Djava.security.debug=access:failure -Djava.security.manager -Djava.security.policy=openj9-openjdk-jdk8/jdk/test/sun/reflect/ReflectionFactory/security.policy SecureRandomTest

[WilburZjh]

Now considering the difference between the content in original ${JAVA_HOME}/jre/lib/security/java.policy file and jdk/test/sun/reflect/ReflectionFactory/security.policy

[pshipton]

Doesn't the -Djava.security.debug=access:failure tell you which permission failed?

[pshipton]

It seems there are only two possibilities for causing the failure.

grant codeBase "file:${{java.ext.dirs}}/*" {
        permission java.security.AllPermission;
};

       // allows anyone to listen on dynamic ports
        permission java.net.SocketPermission "localhost:0", "listen";

[pshipton]

Keeping AllPermission for the ext.dirs fixes it, or the following more specific permissions.

grant codeBase "file:${{java.ext.dirs}}/*" {
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.util";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.ec";
        permission java.lang.RuntimePermission "loadLibrary.sunec";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.pkcs11";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.action";
        permission java.security.SecurityPermission "putProviderProperty.SunEC";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.pkcs11.wrapper";
        permission java.lang.RuntimePermission "loadLibrary.j2pkcs11";
        permission java.security.SecurityPermission "putProviderProperty.SunPKCS11-NSS-FIPS";
        permission java.lang.RuntimePermission "accessClassInPackage.openj9.internal.security";
};

[pshipton]

Although I saw them denied along the way to getting the full set, these ones aren't actually required.

        permission java.lang.RuntimePermission "loadLibrary.sunec";
        permission java.security.SecurityPermission "putProviderProperty.SunEC";

i.e. a more compact set of perms that works

grant codeBase "file:${{java.ext.dirs}}/*" {
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.*";
        permission java.lang.RuntimePermission "accessClassInPackage.openj9.internal.security";
        permission java.lang.RuntimePermission "loadLibrary.j2pkcs11";
        permission java.security.SecurityPermission "putProviderProperty.SunPKCS11-NSS-FIPS";
};

[WilburZjh]

Keeping AllPermission for the ext.dirs fixes it, or the following more specific permissions.

grant codeBase "file:${{java.ext.dirs}}/*" {
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.util";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.ec";
        permission java.lang.RuntimePermission "loadLibrary.sunec";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.pkcs11";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.action";
        permission java.security.SecurityPermission "putProviderProperty.SunEC";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.pkcs11.wrapper";
        permission java.lang.RuntimePermission "loadLibrary.j2pkcs11";
        permission java.security.SecurityPermission "putProviderProperty.SunPKCS11-NSS-FIPS";
        permission java.lang.RuntimePermission "accessClassInPackage.openj9.internal.security";
};

Keeping AllPermission for the ext.dirs fixes it, or the following more specific permissions.

grant codeBase "file:${{java.ext.dirs}}/*" {
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.util";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.ec";
        permission java.lang.RuntimePermission "loadLibrary.sunec";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.pkcs11";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.action";
        permission java.security.SecurityPermission "putProviderProperty.SunEC";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.pkcs11.wrapper";
        permission java.lang.RuntimePermission "loadLibrary.j2pkcs11";
        permission java.security.SecurityPermission "putProviderProperty.SunPKCS11-NSS-FIPS";
        permission java.lang.RuntimePermission "accessClassInPackage.openj9.internal.security";
};

Yes, I can see the denied message now.

[WilburZjh]

After updating the security.policy file with these specific additional permissions.

grant codeBase "file:${{java.ext.dirs}}/*" {
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.util";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.ec";
        permission java.lang.RuntimePermission "loadLibrary.sunec";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.pkcs11";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.action";
        permission java.security.SecurityPermission "putProviderProperty.SunEC";
        permission java.lang.RuntimePermission "accessClassInPackage.sun.security.pkcs11.wrapper";
        permission java.lang.RuntimePermission "loadLibrary.j2pkcs11";
        permission java.security.SecurityPermission "putProviderProperty.SunPKCS11-NSS-FIPS";
        permission java.lang.RuntimePermission "accessClassInPackage.openj9.internal.security";
};

The Caused by: java.lang.RuntimeException: could not find SunPKCS11-NSS-FIPS provider for FIPS mode problem is solved.

Here is the result from <<grinder>>.

The new problem is
java.lang.RuntimeException: javax.net.ssl.SSLException: Unsupported signature algorithm: rsa_pss_rsae_sha256
which is not supported in FIPS, so this issue should be fine for now.