What is the predictable behavior of changing SOL_SOCKET, SO_RCVBUF on the fly on a UDP socket?

Question

What should be expected to happen if we resize the input buffer of a UDP server socket on the fly on a Linux system?

setsockopt(sock, SOL_SOCKET, SO_RCVBUF, ...)

I am particularly interested in these questions:

• If I shrink below what is currently in the buffer, would this simply drop the oldest/newest? datagrams properly, or could it flush everything that's there, or worse could it corrupt data such as truncating a datagram?
• Would shrinking the buffer even save memory or something prevents that memory from being reused by the system?
• Is the behavior predictable or could it behave randomly at times?

Answer 1

First and foremost: the term "buffer" might be confusing: the kernel does not actually save packets in buffers of fixed size, but rather in queues called "backlogs" (see include/net/sock.h:400). The size set through SO_RCVBUF and SO_SNDBUF only limits the maximum size of the backlog.

If I shrink below what is currently in the buffer, would this simply drop the oldest/newest?

No, what was already received is kept. No datagrams are thrown away. The only thing that happens when you do setsockopt(SO_RECVBUF) is that the value of the sk_rcvbuf field of the socket is changed. No other action is performed.

The real effect of this is only seen when receiving more packets: all subsequent datagrams that are received will be immediately dropped, and will continue being dropped until the queue shrinks under the set size (i.e. userspace receives enough datagrams).

Would shrinking the buffer even save memory or something prevents that memory from being reused by the system?

As I said earlier, since the "buffer" is not really a buffer and does not have a fixed size, changing SO_RECVBUF will not change anything immediately.

There are two scenarios:

1. If the backlog size was below (or equal to) the specified size: the new maximum size will be limited, so it will save memory in the future at the cost of potentially losing packets.
2. If the backlog size was above the specified size: memory will eventually be freed when userspace receives the buffered packets, and it will not grow again over the set value. This will again save memory in the future.

Is the behavior predictable or could it behave randomly at times?

Looking at the kernel code, I would say 100% predictable in the way I described above. However I am not entirely sure where this might be documented. I would say it's kind of intuitive if you think about send and receive "buffers" as queues (which they actually are).