Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RANGER-4038: Upgrading to spring 6 #419

Draft
wants to merge 38 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
0c1333f
add support for java17
Oct 8, 2024
5f0dbcb
replace direct javax dependencies to jakarta
Oct 9, 2024
76844af
adapt script with fer changes
Oct 22, 2024
9aa23f2
wip: blocked by hadoop transitive dependency
Oct 22, 2024
16ab0ba
remove wrapper of hadoop classes for jakarta migration
Oct 31, 2024
6ffcd32
added shaded for hadoop-common and hadoop-auth. compile ranger-kms an…
Oct 31, 2024
6855da2
migrating jersey
Nov 4, 2024
1ba04a3
change ClientResponse for jakarta.Response
Nov 4, 2024
789f1c9
migrating jersey. profile security-admin COMPILE :rocket: :tada:
Nov 5, 2024
5232d4e
install hadoop shaded libs in m2 local repository
Nov 11, 2024
8b77af8
update all xx-applicationContext to spring 6
Nov 11, 2024
5dc854c
migrating jersey. web.xml changes
Nov 11, 2024
28827a7
fix: both log4j-over-slf4j and bound slf4j-reload4j on classpath
Nov 11, 2024
f029b78
update Tomcat EmbeddedServer from 8.5 to 10.1.31
Nov 12, 2024
c98c654
change persistence from javax to jakarta
Nov 12, 2024
33f4df4
enable multiPartFeature for file requests
Nov 14, 2024
fc22305
configure Jasper compile variables
Nov 14, 2024
46bd592
fix usersync distro. Solve ClassNotFoundExceptions
Nov 25, 2024
e73f64e
disable bad constructed endpoints to avoid Jersey validation errors
Nov 26, 2024
26231ef
add transactionTemplate to fix store session in db.
Nov 26, 2024
dc3e2dc
disable security context explicit save
Nov 26, 2024
c60e734
javax -> jakarta upgrade
Nov 26, 2024
b9dc444
add jackson integration :tada:
Nov 27, 2024
cbe57be
move hadoop shaded scripts to download-archives.sh
Nov 27, 2024
5e3e38d
exclude some spring dependencies
Nov 27, 2024
1f80bf6
fix unit test in ranger-plugins-common
Nov 29, 2024
5c9b433
revert custom scripts commit: 365ee980de785b535e7d1759013d7b23e0f94e23
Nov 29, 2024
fbd0c78
kms: fix distro dependencies
Dec 3, 2024
43bba87
kms: fix util Base64 old version
Dec 3, 2024
9974728
tagsync: add dependencies. service working :tada:
Dec 9, 2024
e7d7f1a
kms: fix dependencies and update json provider
Dec 11, 2024
e30b7cf
kms: avoid accessing private constructor using reflexion
Dec 12, 2024
5f8d964
knox: not execute test if jre > 17. Knox gateway uses private constru…
Dec 12, 2024
e71ca81
hdfs: ignore test due to incompatibility of MiniHDFSCluster class wit…
Dec 12, 2024
930e8b1
change js engine options to allow host access
Dec 13, 2024
381b218
remove mock for RangerAtlasEntity due to mockito can´t access to fina…
Dec 13, 2024
bb4e459
EmbeddedServer: configurable ssl protocol
Dec 16, 2024
ae93ad4
ranger: fix the validations errors in Jersey's endpoint scanning
Dec 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
7 changes: 6 additions & 1 deletion agents-audit/src/main/resources/META-INF/persistence.xml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence version="3.0"
xmlns="https://jakarta.ee/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://jakarta.ee/xml/ns/persistence
https://jakarta.ee/xml/ns/persistence/persistence_3_0.xsd">

<persistence-unit name="xa_server">
<class>org.apache.ranger.audit.entity.AuthzAuditEventDbObj</class>

Expand Down
30 changes: 19 additions & 11 deletions agents-common/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,25 @@
</parent>
<dependencies>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-bundle</artifactId>
<version>${jersey-bundle.version}</version>
<exclusions>
<exclusion>
<groupId>javax.ws.rs</groupId>
<artifactId>jsr311-api</artifactId>
</exclusion>
</exclusions>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-binding</artifactId>
<version>${jersey-media.version}</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
<version>${jersey-core.version}</version>
</dependency>
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-json-jackson</artifactId>
<version>${jersey-media.version}</version>
</dependency>
<dependency>
<groupId>jakarta.servlet</groupId>
<artifactId>jakarta.servlet-api</artifactId>
<version>${jakarta.servlet.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
Expand Down Expand Up @@ -185,7 +195,6 @@
<artifactId>jackson-core</artifactId>
<version>${fasterxml.jackson.version}</version>
</dependency>
<!-- To be added for JDK15 and above
<dependency>
<groupId>org.graalvm.js</groupId>
<artifactId>js</artifactId>
Expand All @@ -196,7 +205,6 @@
<artifactId>js-scriptengine</artifactId>
<version>${graalvm.version}</version>
</dependency>
-->
<!-- Test -->
<dependency>
<groupId>org.junit.jupiter</groupId>
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import java.util.List;

import com.fasterxml.jackson.annotation.JsonInclude;
import jakarta.ws.rs.core.Response;
import org.apache.ranger.authorization.utils.StringUtil;
import org.apache.ranger.plugin.util.JsonUtilsV2;
import com.fasterxml.jackson.annotation.JsonAutoDetect;
Expand All @@ -29,8 +30,6 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.sun.jersey.api.client.ClientResponse;


@JsonAutoDetect(getterVisibility = Visibility.NONE, setterVisibility = Visibility.NONE, fieldVisibility = Visibility.ANY)
@JsonInclude(JsonInclude.Include.NON_EMPTY)
Expand Down Expand Up @@ -91,10 +90,10 @@ public String getMessage() {
return StringUtil.isEmpty(msgDesc) ? ("HTTP " + httpStatusCode) : msgDesc;
}

public static RESTResponse fromClientResponse(ClientResponse response) {
public static RESTResponse fromClientResponse(Response response) {
RESTResponse ret = null;

String jsonString = response == null ? null : response.getEntity(String.class);
String jsonString = response == null ? null : response.readEntity(String.class);
int httpStatus = response == null ? 0 : response.getStatus();

if(! StringUtil.isEmpty(jsonString)) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
*/
package org.apache.ranger.plugin.contextenricher;

import com.sun.jersey.api.client.ClientResponse;
import jakarta.ws.rs.core.Response;
import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.security.UserGroupInformation;
import org.apache.ranger.admin.client.datatype.RESTResponse;
Expand All @@ -34,7 +34,7 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.servlet.http.HttpServletResponse;
import jakarta.servlet.http.HttpServletResponse;
import java.io.File;
import java.io.Reader;
import java.io.Writer;
Expand All @@ -58,7 +58,7 @@ public class RangerUserStoreRefresher extends Thread {

private final String cacheFile;
private boolean hasProvidedUserStoreToReceiver;
private RangerRESTClient rangerRESTClient;
private final RangerRESTClient rangerRESTClient;

public RangerUserStoreRefresher(RangerUserStoreRetriever userStoreRetriever, RangerUserStoreEnricher userStoreEnricher,
RangerRESTClient restClient, long lastKnownVersion,
Expand Down Expand Up @@ -368,7 +368,7 @@ private RangerUserStore getUserStoreIfUpdated(long lastKnownUserStoreVersion, lo
final RangerUserStore ret;
final UserGroupInformation user = MiscUtil.getUGILoginUser();
final boolean isSecureMode = user != null && UserGroupInformation.isSecurityEnabled();
final ClientResponse response;
final Response response;

Map<String, String> queryParams = new HashMap<String, String>();
queryParams.put(RangerRESTUtils.REST_PARAM_LAST_KNOWN_USERSTORE_VERSION, Long.toString(lastKnownUserStoreVersion));
Expand All @@ -378,7 +378,7 @@ private RangerUserStore getUserStoreIfUpdated(long lastKnownUserStoreVersion, lo
if (LOG.isDebugEnabled()) {
LOG.debug("Checking UserStore updated as user : " + user);
}
response = MiscUtil.executePrivilegedAction((PrivilegedExceptionAction<ClientResponse>) () -> {
response = MiscUtil.executePrivilegedAction((PrivilegedExceptionAction<Response>) () -> {
try {
String relativeURL = RangerRESTUtils.REST_URL_SERVICE_SERCURE_GET_USERSTORE;

Expand Down Expand Up @@ -418,7 +418,7 @@ private RangerUserStore getUserStoreIfUpdated(long lastKnownUserStoreVersion, lo
+ ", response=" + response.getStatus()
+ ", " + "lastKnownUserStoreVersion=" + lastKnownUserStoreVersion
+ ", " + "lastActivationTimeInMillis=" + lastActivationTimeInMillis);
String exceptionMsg = response.hasEntity() ? response.getEntity(String.class) : null;
String exceptionMsg = response.hasEntity() ? response.readEntity(String.class) : null;
LOG.warn("Received 404 error code with body:[" + exceptionMsg + "], Ignoring");
} else {
RESTResponse resp = RESTResponse.fromClientResponse(response);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.sun.jersey.api.client.ClientResponse;
import jakarta.ws.rs.core.Response;

import java.io.Reader;
import java.io.Serializable;
Expand Down Expand Up @@ -88,14 +88,14 @@ static public String nonSerializableObjToJson(Object obj) throws Exception {
return getMapper().writeValueAsString(obj);
}

static public <T> T readResponse(ClientResponse response, Class<T> cls) throws Exception {
String jsonStr = response.getEntity(String.class);
static public <T> T readResponse(Response response, Class<T> cls) throws Exception {
String jsonStr = response.readEntity(String.class);

return jsonToObj(jsonStr, cls);
}

static public <T> T readResponse(ClientResponse response, TypeReference<T> cls) throws Exception {
String jsonStr = response.getEntity(String.class);
static public <T> T readResponse(Response response, TypeReference<T> cls) throws Exception {
String jsonStr = response.readEntity(String.class);

return jsonToObj(jsonStr, cls);
}
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.sun.jersey.core.util.Base64;
import java.util.Base64;
public class PasswordUtils {

private static final Logger LOG = LoggerFactory.getLogger(PasswordUtils.class);
Expand Down Expand Up @@ -78,7 +78,7 @@ private String encrypt() throws IOException {
SecretKey key = skf.generateSecret(keySpec);
engine.init(Cipher.ENCRYPT_MODE, key, new PBEParameterSpec(salt, iterationCount, new IvParameterSpec(iv)));
byte[] encryptedStr = engine.doFinal(strToEncrypt.getBytes());
ret = new String(Base64.encode(encryptedStr));
ret = new String(Base64.getEncoder().encodeToString(encryptedStr));
}
catch(Throwable t) {
LOG.error("Unable to encrypt password due to error", t);
Expand All @@ -101,7 +101,7 @@ private String encrypt() throws IOException {
SALT = crypt_algo_array[index++].getBytes(); // 2
iterationCount = Integer.parseInt(crypt_algo_array[index++]);// 3
if (needsIv(cryptAlgo)) {
iv = Base64.decode(crypt_algo_array[index++]);
iv = Base64.getDecoder().decode(crypt_algo_array[index++]);
} else {
iv = DEFAULT_INITIAL_VECTOR;
}
Expand Down Expand Up @@ -141,7 +141,7 @@ public static String decryptPassword(String aPassword) throws IOException {
private String decrypt() throws IOException {
String ret = null;
try {
byte[] decodedPassword = Base64.decode(password);
byte[] decodedPassword = Base64.getDecoder().decode(password);
Cipher engine = Cipher.getInstance(cryptAlgo);
PBEKeySpec keySpec = new PBEKeySpec(encryptKey);
SecretKeyFactory skf = SecretKeyFactory.getInstance(cryptAlgo);
Expand Down Expand Up @@ -185,7 +185,7 @@ public static String generateIvIfNeeded(String cryptAlgo) throws NoSuchAlgorithm
private static String generateBase64EncodedIV() throws NoSuchAlgorithmException {
byte[] iv = new byte[16];
SecureRandom.getInstance("NativePRNGNonBlocking").nextBytes(iv);
return new String(Base64.encode(iv));
return new String(Base64.getEncoder().encodeToString(iv));
}

public String getCryptAlgo() {
Expand Down Expand Up @@ -213,7 +213,7 @@ public byte[] getIv() {
}

public String getIvAsString() {
return new String(Base64.encode(getIv()));
return new String(Base64.getEncoder().encodeToString(getIv()));
}
public static String getDecryptPassword(String password) {
String decryptedPwd = null;
Expand Down
Loading