Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tagV20210410 版本提交任务失败,但实际上成功的问题。 #46

Open
fzyejy opened this issue May 6, 2021 · 0 comments
Open

Comments

@fzyejy
Copy link

fzyejy commented May 6, 2021

试用最新tagV20210410 版本提交任务时,大概率出现失败(多次重试有小概率成功),实际去flink dashboard看,又是提交成功的,任务正确执行。貌似是小问题,请作者大神看下。
具体报错如下:
2021-05-06 11:57:27.640 [http-nio-9084-exec-9] [TxId : ] INFO c.f.s.web.controller.api.JobConfigApiController - [JobConfigApiController - getJobServerAO] 本地模式启动 LOCAL
2021-05-06 11:57:27.644 [http-nio-9084-exec-9] [TxId : ] INFO com.flink.streaming.web.common.util.FileUtils - [FileUtils - writeText] 写入文件 filePath=/data/flink-streaming-platform-web-new/sql/job_sql_2.sql
2021-05-06 11:57:27.666 [http-nio-9084-exec-9] [TxId : ] INFO com.flink.streaming.web.config.JobThreadPool - [JobThreadPool - getInstance] JobThreadPool threadPoolExecutor=java.util.concurrent.ThreadPoolExecutor@197dfb29[Running, pool size = 7, active threads = 0, queued tasks = 0, completed tasks = 7]
2021-05-06 11:57:27.668 [pool-2-thread-8] [TxId : ] INFO com.flink.streaming.web.common.util.CommandUtil - [CommandUtil - buildRunCommandForCluster] buildRunCommandForLocal runCommand=/data/flink/bin/flink run -d -c com.flink.streaming.core.JobApplication /data/flink-streaming-platform-web-new/lib/flink-streaming-core-1.3.0.RELEASE.jar -sql /data/flink-streaming-platform-web-new/sql/job_sql_2.sql
2021-05-06 11:57:27.668 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - submitJob] command =/data/flink/bin/flink run -d -c com.flink.streaming.core.JobApplication /data/flink-streaming-platform-web-new/lib/flink-streaming-core-1.3.0.RELEASE.jar -sql /data/flink-streaming-platform-web-new/sql/job_sql_2.sql
2021-05-06 11:57:27.669 [pool-2-thread-8] [TxId : ] INFO com.flink.streaming.web.config.WaitForPoolConfig - [WaitForPoolConfig - getInstance] WaitForPoolConfig threadPoolExecutor=java.util.concurrent.ThreadPoolExecutor@6c6d1805[Running, pool size = 7, active threads = 0, queued tasks = 0, completed tasks = 7]
2021-05-06 11:57:29.523 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - clearInfoLogStream]
#############CREATE_TABLE#############
CREATE TABLE test (
_time date,
price numeric(8,2),
id string,
primary key(id) not enforced
)WITH (
'connector' = 'postgres-cdc',
'hostname' = '192.168.3.158',
'port' = '5432',
'username' = 'postgres',
'password' = 'Fz123456!',
'database-name' = 'postgres',
'schema-name' = 'public',
'table-name' = 'test',
'decoding.plugin.name' = 'pgoutput',
'debezium.slot.name' = 'dbz_flink_test',
'debezium.publication.autocreate.mode'= 'filtered',
'debezium.publication.name'= 'dbz_flink_test'
)

2021-05-06 11:57:29.643 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - clearInfoLogStream]
#############CREATE_TABLE#############
CREATE TABLE test_01 (
_time date,
price numeric(8,2),
id string,
primary key(id) not enforced
)WITH (
'connector' = 'jdbc',
'url' = 'jdbc:postgresql://192.168.3.158:5432/postgres',
'table-name' = 'public.test_01',
'username' = 'postgres',
'password' = '123')
2021-05-06 11:57:29.654 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - clearInfoLogStream]

#############CREATE_TABLE#############
CREATE TABLE test_02 (
_time date,
price numeric(8,2),
id string,
primary key(id) not enforced
)WITH (
'connector' = 'jdbc',
'url' = 'jdbc:postgresql://192.168.3.158:5432/postgres',
'table-name' = 'public.test_02',
'username' = 'postgres',
'password' = '123')

2021-05-06 11:57:29.656 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - clearInfoLogStream]
#############INSERT_INTO#############
insert into test_01 select * from test

2021-05-06 11:57:32.163 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - clearInfoLogStream] Job has been submitted with JobID e999bef4f6484b7596154e71e473b444
job-submitted-success:e999bef4f6484b7596154e71e473b444

2021-05-06 11:57:32.201 [2021-05-06 11:57:27#startForLocal-error#LOCAL] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - close] [clearLogStream]关闭reader
2021-05-06 11:57:32.202 [2021-05-06 11:57:27#startForLocal-error#LOCAL] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - close] [clearLogStream]关闭stream
2021-05-06 11:57:32.202 [2021-05-06 11:57:27#startForLocal-error#LOCAL] [TxId : ] INFO com.flink.streaming.web.config.WaitForPoolConfig - [WaitForPoolConfig - getInstance] WaitForPoolConfig threadPoolExecutor=java.util.concurrent.ThreadPoolExecutor@6c6d1805[Running, pool size = 8, active threads = 1, queued tasks = 0, completed tasks = 7]
2021-05-06 11:57:32.202 [2021-05-06 11:57:27#startForLocal-error#LOCAL] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - close] 线程池状态: java.util.concurrent.ThreadPoolExecutor@6c6d1805[Running, pool size = 8, active threads = 1, queued tasks = 0, completed tasks = 7]
2021-05-06 11:57:32.203 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - close] [clearInfoLogStream]关闭reader
2021-05-06 11:57:32.207 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - close] [clearInfoLogStream]关闭stream
2021-05-06 11:57:32.207 [pool-2-thread-8] [TxId : ] INFO com.flink.streaming.web.config.WaitForPoolConfig - [WaitForPoolConfig - getInstance] WaitForPoolConfig threadPoolExecutor=java.util.concurrent.ThreadPoolExecutor@6c6d1805[Running, pool size = 8, active threads = 0, queued tasks = 0, completed tasks = 8]
2021-05-06 11:57:32.207 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.rpc.impl.CommandRpcClinetAdapterImpl - [CommandRpcClinetAdapterImpl - close] 线程池状态: java.util.concurrent.ThreadPoolExecutor@6c6d1805[Running, pool size = 8, active threads = 0, queued tasks = 0, completed tasks = 8]
2021-05-06 11:57:42.226 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.common.util.HttpServiceCheckerUtil - [HttpServiceCheckerUtil - checkUrlConnect] checkUrlConnect url is http://192.168.3.158:8081/
2021-05-06 11:57:42.239 [pool-2-thread-8] [TxId : ] INFO c.f.s.web.common.util.HttpServiceCheckerUtil - [HttpServiceCheckerUtil - checkUrlConnect] 网络检查正常 url=http://192.168.3.158:8081/
2021-05-06 11:57:42.240 [pool-2-thread-8] [TxId : ] INFO c.f.streaming.web.rpc.impl.FlinkRestRpcAdapterImpl - [FlinkRestRpcAdapterImpl - getJobInfoForStandaloneByAppId] [getJobInfoForStandaloneByAppId]请求参数 appId=Job has been submitted with JobID e999bef4f6484b7596154e71e473b444
e999bef4f6484b7596154e71e473b444 url=http://192.168.3.158:8081/jobs/Job has been submitted with JobID e999bef4f6484b7596154e71e473b444
e999bef4f6484b7596154e71e473b444
2021-05-06 11:57:42.255 [pool-2-thread-8] [TxId : ] ERROR c.f.streaming.web.rpc.impl.FlinkRestRpcAdapterImpl - [FlinkRestRpcAdapterImpl - getJobInfoForStandaloneByAppId] json 异常 res=null
org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request: [{"errors":["org.apache.flink.runtime.rest.handler.RestHandlerException: Bad request, could not parse parameters: Cannot resolve path parameter (jobid) from value "Job has been submitted with JobID e9... (6388 bytes)]
at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:101)
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:184)
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:125)
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:782)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:740)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:674)
at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:315)
at com.flink.streaming.web.rpc.impl.FlinkRestRpcAdapterImpl.getJobInfoForStandaloneByAppId(FlinkRestRpcAdapterImpl.java:44)
at com.flink.streaming.web.ao.impl.JobBaseServiceAOImpl$1.submitJobForStandalone(JobBaseServiceAOImpl.java:314)
at com.flink.streaming.web.ao.impl.JobBaseServiceAOImpl$1.run(JobBaseServiceAOImpl.java:195)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-05-06 11:57:42.256 [pool-2-thread-8] [TxId : ] ERROR c.flink.streaming.web.ao.impl.JobBaseServiceAOImpl - [JobBaseServiceAOImpl - submitJobForStandalone] [submitJobForYarn] is error jobStandaloneInfo=JobStandaloneInfo(jid=null, state=null, errors=400 Bad Request: [{"errors":["org.apache.flink.runtime.rest.handler.RestHandlerException: Bad request, could not parse parameters: Cannot resolve path parameter (jobid) from value "Job has been submitted with JobID e9... (6388 bytes)])
2021-05-06 11:57:42.256 [pool-2-thread-8] [TxId : ] ERROR c.flink.streaming.web.ao.impl.JobBaseServiceAOImpl - [JobBaseServiceAOImpl - run] exe is error
com.flink.streaming.web.exceptions.BizException: 任务失败
at com.flink.streaming.web.ao.impl.JobBaseServiceAOImpl$1.submitJobForStandalone(JobBaseServiceAOImpl.java:320)
at com.flink.streaming.web.ao.impl.JobBaseServiceAOImpl$1.run(JobBaseServiceAOImpl.java:195)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant