Running a composed task as a schedule causes an NPE

[wqld]

Description:
We are configuring a composed task with the following settings

• Application 1: query-executor [type: Task, version: 3.x]
• Application 2: data-loader [type: Task, version: 3.x]
• Composed Task: xxx-statistics [query-executor && data-loader]

If we create the above composed task as a schedule and look at the log of the completed execution, we will see the following NullPointerException.

2024-06-18 00:30:07.944  INFO 1 --- [           main] .c.d.c.ComposedTaskStepExecutionListener : ComposedTaskStepExecutionListener supporting [query-executor, data-loader]

...

2024-06-18 00:30:15.377  INFO 1 --- [           main] o.s.c.d.c.ComposedTaskRunnerTaskListener : onTaskStartup:executionId=1534
2024-06-18 00:30:15.418  INFO 1 --- [           main] o.s.b.web.embedded.netty.NettyWebServer  : Netty started on port 8080
2024-06-18 00:30:15.483  INFO 1 --- [           main] o.s.c.d.c.ComposedTaskRunner             : Started ComposedTaskRunner in 13.685 seconds (JVM running for 14.578)
2024-06-18 00:30:15.490  INFO 1 --- [           main] o.s.b.a.b.JobLauncherApplicationRunner   : Running default command line with: []
2024-06-18 00:30:15.491  INFO 1 --- [           main] o.s.b.c.r.s.JobRepositoryFactoryBean     : No database type set, using meta data indicating: POSTGRES
2024-06-18 00:30:15.726  INFO 1 --- [           main] o.s.b.c.l.support.SimpleJobLauncher      : Job: [FlowJob: [name=xxx-statistics]] launched with the following parameters: [{run.id=2}]
2024-06-18 00:30:15.746  INFO 1 --- [           main] o.s.c.t.b.l.TaskBatchExecutionListener   : The job execution id 1297 was run within the task execution 1534
2024-06-18 00:30:15.799  INFO 1 --- [           main] o.s.batch.core.job.SimpleStepHandler     : Executing step: [xxx-query-executor_0]
2024-06-18 00:30:15.809  INFO 1 --- [           main] .c.d.c.ComposedTaskStepExecutionListener : beforeStep:xxx-query-executor_0:1297>>>>
2024-06-18 00:30:15.809  INFO 1 --- [           main] .c.d.c.ComposedTaskStepExecutionListener : beforeStep:xxx-query-executor_0:1297<<<
2024-06-18 00:31:30.964  INFO 1 --- [           main] .c.d.c.ComposedTaskStepExecutionListener : AfterStep processing for stepExecution xxx-query-executor_0:1297
2024-06-18 00:31:30.964  INFO 1 --- [           main] .c.d.c.ComposedTaskStepExecutionListener : AfterStep for xxx-query-executor_0:1297:xxx-query-executor:630:boot3
2024-06-18 00:31:30.965  WARN 1 --- [           main] o.s.c.d.c.TaskExplorerContainer          : Cannot find TaskExplorer for boot3. Using default
2024-06-18 00:31:30.972 ERROR 1 --- [           main] o.s.batch.core.step.AbstractStep         : Exception in afterStep callback in step xxx-query-executor_0 in job xxx-statistics

java.lang.NullPointerException: null
	at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskStepExecutionListener.afterStep(ComposedTaskStepExecutionListener.java:83) ~[classes/:na]
	at org.springframework.batch.core.listener.CompositeStepExecutionListener.afterStep(CompositeStepExecutionListener.java:64) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:247) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:152) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:68) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:68) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.flow.support.state.FlowState.handle(FlowState.java:56) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:167) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:139) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:320) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:149) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-5.3.29.jar:5.3.29]
	at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:140) ~[spring-batch-core-4.3.9.jar:4.3.9]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]

...

The good news is that the subtasks within that composed task complete without a problem.
Note that this error does not occur when the composed task is run via LAUNCH TASK. There is no difference in the settings between LAUNCH TASK and SCHEDULE TASK.

Release versions:

• K8S: v1.28.9
• SCDF: 2.11.2
• CTR: 2.11.0

[corneil]

@wqld Please try CTR 2.11.2 or even better upgrade both SCDF and CTR to 2.11.3

[wqld]

@corneil The same error occurs even with CTR 2.11.2. I will consider upgrading the SCDF version later.

[cppwfs]

Can you provide the task definition you are using?

[cppwfs]

Issue closed due to inactivity. If closed in error, please let us know.