How to start a KVM-QEMU VM once we connect to the VNC port by systemd sockets?

Question

I am trying to keep my virtual machines in a non started state until they are needed. And then keep them on until the users or I turn them off. I'm trying to auto trigger the connection to the VNC server in port 5900. (example #1) or pipe it to another port that proxies it to port 5900 later (example #2). Right now I have Cloudfare tunnels that render the VNC directly on a browser. I tried the two distinct approaches:

First idea kvm1.socket (port 5900) kvm1.service (starting the VM with VNC in port 5900)

and

Second idea with: kvm1-proxy.socket (port 5888) kvm1-proxy.service (transfer to port 5900) kvm1.service (starting the VM with VNC in port 5900)

Config files for the first idea kvm1.socket

[Unit]
Description=KVM1 VM socket

[Socket]
ListenStream=5900
NoDelay=true
DeferAcceptSec=15
[Install]
WantedBy=sockets.target
     

kvm1.service

[Unit]
Description = kvm1
After=network.target
[Service]
Type=notify
ExecStart=virsh start ubuntu23.10-kvm1
#ExecStartPost=/usr/local/bin/waitport 127.0.0.1 5900 #This is somewhat a script to do "sleep every 0.1 until port is open"
#ExecStop=virsh shutdown ubuntu23.10-kvm1

[Install]
WantedBy=multi-user.target

Results from: $ sudo systemctl enable kvm1.socket $ sudo systemctl start kvm1.socket

socket listens to port 5900 As soon as I connect to it:

Feb 26 00:16:46 hostname systemd[1]: Listening on kvm1.socket - KVM1 VM socket. Feb 26 00:17:53 hostname systemd[1]: kvm1.socket: Failed with result 'service-start-limit-hit'.
Feb 26 00:17:52 hostname systemd[1]: Starting kvm1.service - kvm1...
Feb 26 00:17:53 hostname virsh[2120685]: error: Failed to start domain 'ubuntu23.10-kvm1' Feb 26 00:17:53 hostname virsh[2120685]: error: internal error: QEMU unexpectedly closed the monitor (vm='ubuntu23.>
Feb 26 00:17:53 hostname systemd[1]: kvm1.service: Main process exited, code=exited, status=1/FAILURE Feb 26 00:17:53 hostname
systemd[1]: kvm1.service: Failed with result 'exit-code'. 
Feb 26 00:17:53 hostname systemd[1]: Failed to start kvm1.service - kvm1. 
Feb 26 00:17:53 hostname systemd[1]: kvm1.service: Start request repeated too quickly.
Feb 26 00:17:53 hostname systemd[1]: kvm1.service: Failed with result 'exit-code'.    
Feb 26 00:17:53 hostname systemd[1]: Failed to start kvm1.service - kvm1.

Is there anyway I can do a socket-start KVM while using the same port?

Test On the second idea connect to 5888 (Via cloudflare in a browser) and proxy it to port 5900.

kvm1-proxy.socket

[Unit]
Description=KVM1 VM socket

[Socket]
ListenStream=5888
NoDelay=true
DeferAcceptSec=15
[Install]
WantedBy=sockets.target

kvm1-proxy.service

[Unit]
Requires=kvm1.service
After=kvm1.service

[Service]
ExecStart=/lib/systemd/systemd-socket-proxyd 127.0.0.1:5900

kvm1.service

[Unit]
Description=kvm1

[Service]
ExecStart=virsh start ubuntu23.10-kvm1
ExecStartPost=/bin/sleep 1
ExecStop=virsh shutdown ubuntu23.10-kvm1

[Install]
WantedBy=multi-user.target

Results from: $ sudo systemctl enable kvm1-proxy.socket $ sudo systemctl start kvm1-proxy.socket

$ sudo systemctl status kvm1-proxy.socket

Active: active (listening) since Mon 2024-02-26 00:28:13 CST; 1min 48s ago    
Triggers: ● kvm1-proxy.service
Listen: [::]:5888 (Stream)

Feb 26 00:28:13 hostname systemd[1]: Listening on kvm1-proxy.socket - KVM1 VM socket.

KVM1.service starts and the virtual machine works.

Now, how can I make kvm1-proxy.socket deactivate when I shutdown the VM? Basically, like this it remains forever active.

kvm1-proxy.service  Loaded: loaded (/etc/systemd/system/kvm1-proxy.service; static)  Active: active (running) since Mon 2024-02-26 00:31:39 CST; 4min 12s ago  TriggeredBy: ● kvm1-proxy.socket    Main PID: 2138322 (systemd-socket-)  Tasks: 2 (limit: 154233)  Memory: 1.0M    CPU: 6ms  CGroup: /system.slice/kvm1-proxy.service
       └─2138322 /lib/systemd/systemd-socket-proxyd 127.0.0.1:5900

Feb 26 00:31:39 hostname systemd[1]: Started kvm1-proxy.service.   
Feb 26 00:31:39 hostname systemd-socket-proxyd[2138322]: Failed to connect to remote host: Connection refused

The idea is that it remains on, until the user turns it off (in the VM poweroff or with virsh shutdown ubuntu23.10-kvm1). systemd status sees it in inactive state (whether running or shutdown). Therefore kvm1-proxy.service keeps "active" and will never start the machine for another cycle. Any way I can do systemd know when the VM is off and the proxy.service goes down?

kvm1.service - kvm1
  Loaded: loaded (/etc/systemd/system/kvm1.service; disabled; preset: enabled)
  Active: inactive (dead) since Mon 2024-02-26 00:31:40 CST; 4min 37s ago    Duration: 664ms
  Process: 2138228 ExecStart=virsh start ubuntu23.10-kvm1 (code=exited, status=0/SUCCESS)
  Process: 2138229 ExecStartPost=/bin/sleep 1 (code=exited, status=0/SUCCESS)
  Process: 2138370 ExecStop=virsh shutdown ubuntu23.10-kvm1 (code=exited, status=0/SUCCESS)    Main PID: 2138228 (code=exited,
status=0/SUCCESS CPU: 46ms

Answer 1

You need to keep the ExecStart= process running for as long as the VM is running.

If the actual VM process is started indirectly (through libvirt), that means you'll need to make your ExecStart into a command that starts the VM and waits forever for the VM to stop. More precisely:

1. issues the virsh command,
2. waits for libvirt to indicate the VM is up (you can use your waitport here, but connecting to libvirt would be nicer)
3. sends a READY=1 message to systemd's $NOTIFY_SOCKET,
4. waits forever for libvirt to indicate the VM is no longer up,
5. exits.

Step 3 needs Type=notify; it lets systemd's dependency mechanism delay the startup of proxyd until the VM is actually ready to accept the connection.