Closing a TcpStream / not reading from the socket

Question

I'm working on a Rust application where I need to send an HTTPS request to a server but intentionally not read the response.

My primary goal is to minimize resource utilization, processing time and mainly to not consume bytes since I'm using a proxy and I want to cut costs -- by avoiding the handling of response data entirely. I understand that the HTTP protocol inherently expects a request-response pattern and that the operating system buffers incoming data for a socket until it's read by the application. However, for my specific use case, I'd like to either close the socket immediately after sending the request or somehow instruct the OS not to consume the data on the socket.

I'm currently using tokio and tokio-rustls for asynchronous networking and TLS support, respectively. Here is a simplified version of my current approach:

let mut root_cert_store = RootCertStore::empty();
root_cert_store.extend(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());
let config = ClientConfig::builder()
    .with_root_certificates(root_cert_store)
    .with_no_client_auth();

let connector = TlsConnector::from(Arc::new(config));

let host = "www.rust-lang.org";

let stream = TcpStream::connect(format!("{}:443", host))
    .await
    .expect("Failed to connect");

let domain = pki_types::ServerName::try_from(host.to_string())
    .unwrap()
    .to_owned();

let mut stream = connector
    .connect(domain, stream)
    .await
    .expect("Failed to connect");

let request = format!(
    "GET / HTTP/1.1\r\nHost: {}\r\nConnection: close\r\n\r\n",
    host
);

stream
    .write_all(request.as_bytes())
    .await
    .expect("Failed to write");

stream.shutdown().await.unwrap();

I'm using tcpdump to monitor if there's data received, and I see that I get the website back, even without reading from the stream.

In Python I'd achieve this capability by explicitly closing the socket I assume.

To my understanding, there's no way of closing the Read stream, only the Write stream, using stream.shutdown, however, this still results in bytes being received.

I also tried being creative with sending close_notify on the ClientConnection, still consumed bytes.

let client_conn = stream.get_mut().1;
client_conn.send_close_notify();

Any insights or suggestions on how to achieve this in Rust would be greatly appreciated.

Answer 1

There is really no way in TCP to send a message from client to server, and tell the other side (at the TCP layer) that you don't want to consume the bytes sent in response. However, you can simply close the socket and not receive them. The trick is: how can you minimize the number of bytes sent by the server?

By reducing the TCP session's "receive window" below its usual/default value, you can minimize the number of bytes sent per segment (segment == a block of data sent in a single packet by TCP and the number of segments the sender will send before waiting for acknowledgment). The second bit of this is to close the socket or equivalently do a shutdown-for-read so that your own kernel sends a RST packet back as quickly as possible (the kernel won't accept more data if it knows the client has already closed/shutdown the socket; instead it resets the connection).

So, as the client, you:

• create a socket
• configure the receive buffer size on the socket to the smallest accepted value
• connect to the server
• send your request to the server
• close/shutdown the socket

Demonstration toy program (written in python because it's quicker for me to prototype with). Here's the server, which is just accepting a connection, reading a single "request" and attempting to send back a 50K block of data.

import socket

s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(("", 5555))
s.listen(5)

conn, ca = s.accept()
x = conn.recv(1000)
print("GOT", x)
conn.sendall(b'12345' * 10000)
x = conn.recv(1000)

Here's the client (which I run on a different system):

import socket

s = socket.socket()
x = s.getsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF)
print("RCVBUF BEFORE", x)
s.setsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF, 1)
x = s.getsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF)
print("RCVBUF AFTER", x)

s.connect(("192.168.1.48", 5555))
s.shutdown(socket.SHUT_RD)
s.sendall(b"xyzzy")

Both systems are linux. The smallest effective SO_RCVBUF it will let me configure is actually 2304. The resulting TCP segment size appears to be 1/4 that (576).

If you start server, then run client and watch tcpdump output in another window, I see that two 576 byte segments are sent in response before the server recognizes the RST.

If I comment out the s.setsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF, 1), then around ten 1448 byte segments are typically sent before the RST is recognized.

(Not sure but some of the details are likely dependent on my particular network setup where the two boxes are connected by a wifi network.)