Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

报kerberos认证失败,直接用hadoop distcp没问题 #123

Open
736008081 opened this issue May 28, 2024 · 2 comments
Open

报kerberos认证失败,直接用hadoop distcp没问题 #123

736008081 opened this issue May 28, 2024 · 2 comments

Comments

@736008081
Copy link

736008081 commented May 28, 2024

集群采用ha和keytab,直接使用hadoop distcp没问题,命令如下:hadoop jar jindo-distcp-tool-4.6.12.jar --bandWidth 4096 --parallelism 80 --disableChecksum --src hdfs://test1 --dest hdfs://test2 --codec keep --hadoopConf hadoop.security.keytab.file=test1.keytab --hadoopConf hadoop.security.principal=test1 --hadoopConf dfs.nameservices=a1
image

@736008081 736008081 reopened this May 28, 2024
@adrian-wang
Copy link
Collaborator

跑hadoop distcp的时候,是怎么指定keytab的呢?

@736008081
Copy link
Author

跑hadoop distcp的时候,是怎么指定keytab的呢?

跑hadoop的机器当前用户已经做了kerberos认证,所以不需要指定keytab,我看了下,怀疑是jindo架包在使用hdfs rest api操作yarn的api接口的时候报了这个错,之前我自己用java的distcp架包也有这个错,看了java的distcp源码,是在通过api查询yarn任务状态的时候报的这个kerberos认证失败

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants