[Fix #3721] Reducing group event size #2122

fjtirado · 2024-10-18T17:26:49Z

Fix apache/incubator-kie-kogito-runtimes#3721
Merge wtih apache/incubator-kie-kogito-runtimes#3739

kie-ci3 · 2024-10-23T15:22:29Z

PR job #8 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer

build-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-apps -u #2122 --skipParallelCheckout

NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-apps-pr/job/PR-2122/8/display/redirect

Test results:

PASSED: 1712
FAILED: 3

Those are the test failures:

org.kie.kogito.index.ProcessDataIndexMongoDBIT.testProcessInstanceEvents

1 expectation failed.
JSON path data.UserTaskInstances[0].potentialGroups[0] doesn't match.
Expected: managers
Actual: null

org.kie.kogito.jobs.service.messaging.InfinispanMessagingApiTest.createJob

createJob() timed out after 10 minutes

org.kie.kogito.jobs.embedded.EmbeddedJobsServiceTest.testJobService

java.util.ConcurrentModificationException
at java.base/java.util.ArrayList.forEach(ArrayList.java:1513)
at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092)
at org.kie.kogito.jobs.embedded.EmbeddedJobsServiceTest.testJobService(EmbeddedJobsServiceTest.java:99)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at io.quarkus.test.junit.QuarkusTestExtension.runExtensionMethod(QuarkusTestExtension.java:1013)
at io.quarkus.test.junit.QuarkusTestExtension.interceptTestMethod(QuarkusTestExtension.java:827)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

kie-ci3 · 2024-10-23T20:05:21Z

PR job #9 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer

build-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-apps -u #2122 --skipParallelCheckout

NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-apps-pr/job/PR-2122/9/display/redirect

Test results:

PASSED: 1715
FAILED: 1

Those are the test failures:

org.kie.kogito.index.infinispan.ProcessDataIndexInfinispanKafkaIT.testProcessInstanceEvents

1 expectation failed.
JSON path errors doesn't match.
Expected: null
Actual: <[{message=Exception while fetching data (/UserTaskInstanceAttachmentUpdate) : Index 0 out of bounds for length 0, locations=[{line=1, column=12}], path=[UserTaskInstanceAttachmentUpdate], extensions={classification=DataFetchingException}}]>

gmunozfe

Looks good to me, just a few comments and questions.
Awesome work @fjtirado !

gmunozfe · 2024-10-24T10:11:05Z

data-audit/data-audit-common-service/src/main/java/org/kie/kogito/app/audit/json/JsonUtils.java

+        return objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false).registerModule(JsonFormat.getCloudEventJacksonModule())
+                .registerModule(new SimpleModule("Data Audit").addDeserializer(JobInstanceDataEvent.class, new JsonJobDataEventDeserializer())).findAndRegisterModules();


Is it not needed to register JavaTimeModule anymore?

findAndRegisterModules() automatically register all modules found in classpath

gmunozfe · 2024-10-24T10:16:03Z

...e-common/src/main/java/org/kie/kogito/index/service/messaging/KogitoIndexEventConverter.java

+            case MultipleProcessInstanceDataEvent.MULTIPLE_TYPE:
+                MultipleProcessInstanceDataEvent dataEvent = buildEvent(cloudEvent, objectMapper, MultipleProcessInstanceDataEvent::new);
+                if (cloudEvent.getData() != null) {
+                    String contentType = cloudEvent.getDataContentType();


If it's not binary, it's assumming that it's "application/json", isn't it? But potentially it could be other even null

if null, by cloudevent spec, application/json should be assumed.

gmunozfe · 2024-10-24T10:17:56Z

...e-common/src/main/java/org/kie/kogito/index/service/messaging/KogitoIndexEventConverter.java

-            case "ProcessInstanceErrorDataEvent":
+            case MultipleProcessInstanceDataEvent.MULTIPLE_TYPE:
+                MultipleProcessInstanceDataEvent dataEvent = buildEvent(cloudEvent, objectMapper, MultipleProcessInstanceDataEvent::new);
+                if (cloudEvent.getData() != null) {


If data is null, you could add a warn logger to trace it in an else clause, wdyt?:
LOGGER.warn("CloudEvent data is null for event of type: {}", cloudEvent.getType());

hmmmm, It was not there before, I do not think is needed now.

kie-ci3 · 2024-10-24T11:22:22Z

PR job #10 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer

build-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-apps -u #2122 --skipParallelCheckout

NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-apps-pr/job/PR-2122/10/display/redirect

Test results:

PASSED: 1721
FAILED: 1

Those are the test failures:

org.kie.kogito.jobs.embedded.EmbeddedJobsServiceTest.testJobService

java.util.ConcurrentModificationException
at java.base/java.util.ArrayList.forEach(ArrayList.java:1513)
at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092)
at org.kie.kogito.jobs.embedded.EmbeddedJobsServiceTest.testJobService(EmbeddedJobsServiceTest.java:99)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at io.quarkus.test.junit.QuarkusTestExtension.runExtensionMethod(QuarkusTestExtension.java:1013)
at io.quarkus.test.junit.QuarkusTestExtension.interceptTestMethod(QuarkusTestExtension.java:827)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

kie-ci3 · 2024-10-24T15:28:59Z

PR job #12 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer

build-chain build full_downstream -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-apps -u #2122 --skipParallelCheckout

NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-apps-pr/job/PR-2122/12/display/redirect

Test results:

PASSED: 1721
FAILED: 1

Those are the test failures:

org.kie.kogito.jobs.embedded.EmbeddedJobsServiceTest.testJobService

java.util.ConcurrentModificationException
at java.base/java.util.ArrayList.forEach(ArrayList.java:1513)
at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092)
at org.kie.kogito.jobs.embedded.EmbeddedJobsServiceTest.testJobService(EmbeddedJobsServiceTest.java:99)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at io.quarkus.test.junit.QuarkusTestExtension.runExtensionMethod(QuarkusTestExtension.java:1013)
at io.quarkus.test.junit.QuarkusTestExtension.interceptTestMethod(QuarkusTestExtension.java:827)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

fjtirado force-pushed the Fix_#3721 branch 2 times, most recently from cf8a7cf to 4b6bf88 Compare October 21, 2024 19:22

fjtirado mentioned this pull request Oct 21, 2024

[Fix #3721] Optimize event grouping apache/incubator-kie-kogito-runtimes#3739

Merged

fjtirado force-pushed the Fix_#3721 branch 3 times, most recently from bf180f1 to bd32fcb Compare October 22, 2024 12:59

fjtirado requested review from wmedvede, ricardozanini and gmunozfe October 22, 2024 15:30

fjtirado marked this pull request as ready for review October 23, 2024 11:41

ricardozanini approved these changes Oct 23, 2024

View reviewed changes

gmunozfe approved these changes Oct 24, 2024

View reviewed changes

fjtirado force-pushed the Fix_#3721 branch from bd32fcb to e0b0985 Compare October 24, 2024 13:20

[Fix #3721] Reducing group event size

8e20c97

fjtirado force-pushed the Fix_#3721 branch from e0b0985 to 8e20c97 Compare October 24, 2024 16:12

fjtirado merged commit c5a19dc into apache:main Oct 25, 2024
5 checks passed

rgdoliveira pushed a commit to rgdoliveira/kogito-apps that referenced this pull request Nov 7, 2024

[Fix #3721] Reducing group event size (apache#2122)

7ea7fa4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fix #3721] Reducing group event size #2122

[Fix #3721] Reducing group event size #2122

fjtirado commented Oct 18, 2024 •

edited

Loading

kie-ci3 commented Oct 23, 2024

kie-ci3 commented Oct 23, 2024

gmunozfe left a comment

gmunozfe Oct 24, 2024

fjtirado Oct 24, 2024

gmunozfe Oct 24, 2024

fjtirado Oct 24, 2024

gmunozfe Oct 24, 2024

fjtirado Oct 24, 2024

kie-ci3 commented Oct 24, 2024

kie-ci3 commented Oct 24, 2024

		return objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false).registerModule(JsonFormat.getCloudEventJacksonModule())
		.registerModule(new SimpleModule("Data Audit").addDeserializer(JobInstanceDataEvent.class, new JsonJobDataEventDeserializer())).findAndRegisterModules();

[Fix #3721] Reducing group event size #2122

[Fix #3721] Reducing group event size #2122

Conversation

fjtirado commented Oct 18, 2024 • edited Loading

kie-ci3 commented Oct 23, 2024

kie-ci3 commented Oct 23, 2024

gmunozfe left a comment

Choose a reason for hiding this comment

gmunozfe Oct 24, 2024

Choose a reason for hiding this comment

fjtirado Oct 24, 2024

Choose a reason for hiding this comment

gmunozfe Oct 24, 2024

Choose a reason for hiding this comment

fjtirado Oct 24, 2024

Choose a reason for hiding this comment

gmunozfe Oct 24, 2024

Choose a reason for hiding this comment

fjtirado Oct 24, 2024

Choose a reason for hiding this comment

kie-ci3 commented Oct 24, 2024

kie-ci3 commented Oct 24, 2024

fjtirado commented Oct 18, 2024 •

edited

Loading