flink job running in Hadoop YARN, got a error "AccessDeniedException"

16 Views Asked by At

flink 1.17

I think the possible reasons are: 1 some programs access the file concurrency 2 one program is writing or deleting the file and the operation is ongoing , another program is trying to access this file at this time

2024-03-06 14:19:09,262 WARN  org.apache.flink.configuration.Configuration                 [] - Config uses deprecated configuration key 'web.port' instead of proper key 'rest.bind-port'
2024-03-06 14:19:09,277 INFO  org.apache.flink.runtime.clusterframework.BootstrapTools     [] - Overriding Flink's temporary file directories with those specified in the Flink config: /opt/software/flink/tempJars
2024-03-06 14:19:09,377 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Starting YarnApplicationClusterEntryPoint.
2024-03-06 14:19:09,382 WARN  org.apache.flink.core.plugin.PluginConfig                    [] - The plugins directory [plugins] does not exist.
2024-03-06 14:19:09,383 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Install default filesystem.
2024-03-06 14:19:09,427 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Install security context.
2024-03-06 14:19:09,465 INFO  org.apache.flink.runtime.security.modules.HadoopModule       [] - Hadoop user set to root (auth:SIMPLE)
2024-03-06 14:19:09,465 INFO  org.apache.flink.runtime.security.modules.HadoopModule       [] - Kerberos security is disabled.
2024-03-06 14:19:09,474 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Shutting YarnApplicationClusterEntryPoint down with application status FAILED. Diagnostics java.lang.RuntimeException: unable to generate a JAAS configuration file
    at org.apache.flink.runtime.security.modules.JaasModule.generateDefaultConfigFile(JaasModule.java:183)
    at org.apache.flink.runtime.security.modules.JaasModule.install(JaasModule.java:92)
    at org.apache.flink.runtime.security.SecurityUtils.installModules(SecurityUtils.java:76)
    at org.apache.flink.runtime.security.SecurityUtils.install(SecurityUtils.java:57)
    at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.installSecurityContext(ClusterEntrypoint.java:274)
    at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:226)
    at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:729)
    at org.apache.flink.yarn.entrypoint.YarnApplicationClusterEntryPoint.main(YarnApplicationClusterEntryPoint.java:112)
Caused by: java.nio.file.AccessDeniedException: /opt/software/flink/tempJars/jaas-1158202180153204055.conf
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
    at java.nio.file.Files.newByteChannel(Files.java:361)
    at java.nio.file.Files.createFile(Files.java:632)
    at java.nio.file.TempFileHelper.create(TempFileHelper.java:138)
    at java.nio.file.TempFileHelper.createTempFile(TempFileHelper.java:161)
    at java.nio.file.Files.createTempFile(Files.java:852)
    at org.apache.flink.runtime.security.modules.JaasModule.generateDefaultConfigFile(JaasModule.java:173)

hope for solutions to fix this bug and get this flink job finished successfully

0

There are 0 best solutions below