Hive Tez 실행 오류 (DIGEST-MD5: No common protection layer between client and server at )

 

오류 내용 :

Caused by: java.io.IOException: javax.security.sasl.SaslException: DIGEST-MD5: No common protection layer between client and server at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:755) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899) at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:709) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:813) at org.apache.hadoop.ipc.Client$Connection.access$3800(Client.java:363) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1649) at org.apache.hadoop.ipc.Client.call(Client.java:1474)

 

 

For more detailed output, check the application tracking page: http://호스트:38088/cluster/app/application_1727638811672_0119 Then click on links to logs of each attempt. . Failing the application. at org.apache.tez.client.TezClientUtils.getAMProxy(TezClientUtils.java:945) at org.apache.tez.client.FrameworkClient.getProxy(FrameworkClient.java:187) at org.apache.tez.client.FrameworkClient.shutdownSession(FrameworkClient.java:176) at org.apache.tez.client.TezClient.stop(TezClient.java:738) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.closeAndIgnoreExceptions(TezSessionState.java:480) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:473) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.access$100(TezSessionState.java:101) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState$1.call(TezSessionState.java:376) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState$1.call(TezSessionState.java:371) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) 2024-10-17 15:03:35,913 INFO client.TezClient: Could not connect to AM, killing session via YARN, sessionName=HIVE-4141fb36-7eda-485a-964d-2e41d72d815f, applicationId=application_1727638811672_0119 2024-10-17 15:03:35,917 INFO impl.YarnClientImpl: Killed application application_1727638811672_0119 2024-10-17 15:03:35,918 ERROR tez.TezSessionState: Failed to start Tez session org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1727638811672_0119 failed 2 times due to AM Container for appattempt_1727638811672_0119_000002 exited with exitCode: 1 Failing this attempt.Diagnostics: [2024-10-17 15:03:35.842]Exception from container-launch. Container id: container_e16_1727638811672_0119_02_000001 Exit code: 1

 

 

 

 

 

 

 

그 전에 Hadoop 에서 고객사가 요청한 보안 사항을 적용 후 Hadoop 재실행을 했는데, 

Yarn 은 하지 않았다. 

 

 

Yarn 을 재실행하니 해결됐는데, 그 이유는 

 

YARN이 재시작되면서 클러스터 내 모든 노드 간의 통신과 인증이 다시 설정되었기 때문.
YARN의 NodeManager와 ResourceManager 간의 통신이나 인증 문제가 초기화되고, Hive와의 SASL 연결이 정상적으로 설정되었을 가능성이 크다.

 

 

 

또, Tez가 실행될 때 HDFS를 읽는 것이 일반적이며, Tez는 대규모 데이터 처리 엔진으로, 주로 HDFS나 YARN과 통합되어 동작하며, 데이터를 처리하는 과정에서 HDFS에 저장된 데이터를 읽고 쓸 수 있다.