Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] There is no task when deploying yarn-application in jar package mode and running it. #3194

Closed
2 of 3 tasks
lliuql opened this issue Feb 28, 2024 · 2 comments
Closed
2 of 3 tasks
Assignees
Labels
Bug Something isn't working
Milestone

Comments

@lliuql
Copy link

lliuql commented Feb 28, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

EXECUTE JAR WITH ( 'uri'='rs:///flink_hello-1.0-SNAPSHOT.jar', 'main-class'='org.liu.StreamWordCount', 'parallelism'='1');
用applicaiton运行jar包后,提示部署成功
2024-02-28 13:35:32.614 INFO org.apache.flink.yarn.YarnClusterDescriptor(1136): YARN application has been deployed successfully. 2024-02-28 13:35:32.614 INFO org.apache.flink.yarn.YarnClusterDescriptor(1688): Found Web Interface hadoop167:38618 of application 'application_1709022070519_0014'. 2024-02-28 13:36:03.139 INFO org.dinky.service.impl.TaskServiceImpl(190): execute job finished,status is SUCCESS Process Step SUBMIT_EXECUTE exit with status:FINISHED
运维中心没有任务:
image

yarn启动了容器:
image

但是flink没有任何任务:
image

What you expected to happen

应该能够在运维中心找到任务。且flink应该有任务

How to reproduce

jar包的内容
`
public class StreamWordCount {
public static void main(String[] args) throws Exception {
// 1. 创建流式执行环境
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
// 2. 读取文本流:hadoop102表示发送端主机名、7777表示端口号
DataStreamSource lineStream = env.socketTextStream("hadoop165", 7777);

    // 3. 转换、分组、求和,得到统计结果
    SingleOutputStreamOperator<Tuple2<String, Long>> sumStream = lineStream.flatMap(new FlatMapFunction<String, Tuple2<String, Long>>() {
        @Override
        public void flatMap(String line, Collector<Tuple2<String, Long>> out) throws Exception {
            String[] words = line.split(" ");
            for (String word : words) {
                out.collect(Tuple2.of(word, 1L));
            }
        }
    }).keyBy(new KeySelector<Tuple2<String, Long>, Object>() {
        @Override
        public Object getKey(Tuple2<String, Long> data) throws Exception {
            return data.f0;
        }
    }).sum(1);
    // 4. 打印
    sumStream.print();

    // 5. 执行
    env.execute();

}

}
pom内容:
`


4.0.0

<groupId>org.liu</groupId>
<artifactId>flink_hello</artifactId>
<version>1.0-SNAPSHOT</version>

<properties>
    <maven.compiler.source>8</maven.compiler.source>
    <maven.compiler.target>8</maven.compiler.target>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <flink.version>1.17.0</flink.version>
</properties>
<dependencies>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-java</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-clients</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
    </dependency>
</dependencies>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.2.4</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <artifactSet>
                            <excludes>
                                <exclude>com.google.code.findbugs:jsr305</exclude>
                                <exclude>org.slf4j:*</exclude>
                                <exclude>log4j:*</exclude>
                            </excludes>
                        </artifactSet>
                        <filters>
                            <filter>
                                <!-- Do not copy the signatures in the META-INF folder.
                                Otherwise, this might cause SecurityExceptions when using the JAR. -->
                                <artifact>*:*</artifact>
                                <excludes>
                                    <exclude>META-INF/*.SF</exclude>
                                    <exclude>META-INF/*.DSA</exclude>
                                    <exclude>META-INF/*.RSA</exclude>
                                </excludes>
                            </filter>
                        </filters>
                        <transformers combine.children="append">
                            <transformer
                                    implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer">
                            </transformer>
                        </transformers>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>
`

`

Anything else

No response

Version

1.0.0

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@lliuql lliuql added Bug Something isn't working Waiting for reply Waiting for reply labels Feb 28, 2024
Copy link

Hello @lliuql, this issue is about web, so I assign it to @Zzm0809. If you have any questions, you can comment and reply.

你好 @lliuql, 这个 issue 是关于 web 的,所以我把它分配给了 @Zzm0809。如有任何问题,可以评论回复。

@github-actions github-actions bot changed the title [Bug] jar包方式部署yarn-application,运行起来,没有任务。 [Bug] There is no task when deploying yarn-application in jar package mode and running it. Feb 28, 2024
@Zzm0809 Zzm0809 assigned gaoyan1998 and zackyoungh and unassigned Zzm0809 Feb 28, 2024
@aiwenmo aiwenmo removed the Waiting for reply Waiting for reply label Feb 28, 2024
@Zzm0809 Zzm0809 added this to the 1.1.0 milestone Mar 13, 2024
@Zzm0809 Zzm0809 moved this to ToDo in Dinky Roadmap Mar 13, 2024
@Zzm0809
Copy link
Contributor

Zzm0809 commented Mar 13, 2024

无法复现

@Zzm0809 Zzm0809 closed this as completed Mar 13, 2024
@Zzm0809 Zzm0809 moved this from ToDo to Done in Dinky Roadmap Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
Status: Done
Development

No branches or pull requests

5 participants