Failed: k3s.service won't start - Air Gapped install of k3s in Rocky 9 VM
I'm trying to install k3s in a disconnected environment on a VM running Rocky 9.
The k3s.service fails to start. It mentions permission denied.
As part of troubleshooting I did the following:
- Disabled SELinux
- Disabled Swap memory
Install media:
Tar file https://github.com/k3s-io/k3s/releases/download/v1.24.3%2Bk3s1/k3s-airgap-images-amd64.tar -> /var/lib/rancher/k3s/agent/images/k3s-airgap-images-amd64.tar
K3S Binary https://github.com/k3s-io/k3s/releases/download/v1.24.3%2Bk3s1/k3s -> /usr/local/bin/k3s
Install Script https://get.k3s.io/ -> /usr/local/install/k3s/install.sh
Install CMD using install script:
sudo INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh
I noticed the following in: /etc/systemd/system/
-rw-r--r-- 1 root root 836 Aug 4 15:14 k3s.service -rw------- 1 root root 0 Aug 4 15:14 k3s.service.env
The install script is meant to set permissions to 755 on the service. That doesn't happen. Doing chmod 755 and rebooting the VM makes no difference to k3s.service starting
Errors:
Job for k3s.service failed because the control process exited with error code. See "systemctl status k3s.service" and "journalctl -xeu k3s.service" for details.
[admin@demolab01 k3s]$ systemctl status k3s.service
k3s.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: exit-code) since Thu 2022-08-04 15:30:22 UTC; 3s ago
Docs: https://k3s.io
Process: 4247 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
Process: 4249 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Process: 4250 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Process: 4251 ExecStart=/usr/local/bin/k3s server (code=exited, status=1/FAILURE)
Main PID: 4251 (code=exited, status=1/FAILURE)
CPU: 18ms
[admin@demolab01 k3s]$ journalctl -xeu k3s.service
Subject: A start job for unit k3s.service has failed
Defined-By: systemd
Support: https://access.redhat.com/support
A start job for unit k3s.service has finished with a failure.
The job identifier is 38821 and the job result is failed.
Aug 04 15:31:24 demolab01.****<fqdn> systemd[1]: k3s.service: Scheduled restart job, restart counter is at 267.
Subject: Automatic restarting of a unit has been scheduled
Defined-By: systemd
Support: https://access.redhat.com/support
Automatic restarting of the unit k3s.service has been scheduled, as the result for
the configured Restart= setting for the unit.
Aug 04 15:31:24 demolab01.****<fqdn> systemd[1]: Stopped Lightweight Kubernetes.
Subject: A stop job for unit k3s.service has finished
Defined-By: systemd
Support: https://access.redhat.com/support
A stop job for unit k3s.service has finished.
The job identifier is 38959 and the job result is done.
Aug 04 15:31:24 demolab01.****<fqdn> systemd[1]: Starting Lightweight Kubernetes...
Subject: A start job for unit k3s.service has begun execution
Defined-By: systemd
Support: https://access.redhat.com/support
A start job for unit k3s.service has begun execution.
The job identifier is 38959.
Aug 04 15:31:24 demolab01.****<fqdn> sh[4359]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 04 15:31:24 demolab01.****<fqdn> sh[4360]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 04 15:31:25 demolab01.****<fqdn> k3s[4363]: time="2022-08-04T15:31:25Z" level=fatal msg="permission denied"
Aug 04 15:31:25 demolab01.****<fqdn> systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Subject: Unit process exited
Defined-By: systemd
Support: https://access.redhat.com/support
An ExecStart= process belonging to unit k3s.service has exited.
The process' exit code is 'exited' and its exit status is 1.
Aug 04 15:31:25 demolab01.****<fqdn> systemd[1]: k3s.service: Failed with result 'exit-code'.
Subject: Unit failed
Defined-By: systemd
Support: https://access.redhat.com/support
The unit k3s.service has entered the 'failed' state with result 'exit-code'.
Aug 04 15:31:25 demolab01.****<fqdn> systemd[1]: Failed to start Lightweight Kubernetes.
Subject: A start job for unit k3s.service has failed
Defined-By: systemd
Support: https://access.redhat.com/support
A start job for unit k3s.service has finished with a failure.
Any ideas welcome. I am no linux expert :-(