A simple Java HTTP server fails with ApacheBench but works fine on a browser

Question

As part of a concurrency blog series, I was building the simplest HTTP server in different languages (Java, Kotlin, Rust, Go, JS, TS) and everything works fine for everything except Java/Kotlin, aka on the JVM. All the code can be found here. The below is the server code in Java, I tried a traditional Thread based one and an AsynchronousServerSocketChannel based one, but regardless when I run a benchmark with ApacheBench it fails with Broken pipe and apr_socket_recv: Connection reset by peer (104) this is weird as similar setup in other languages works fine. The problem here happens only with ApacheBench, coz when I access the URL in a browser it just works fine. SO I'm banging my head to figure out what is going on. I tried to play with keep-alive etc but doesn't seem to help. I looked at a bunch of examples of something similar and I don't see anything special being done anywhere. I'm hoping someone can figure out what is going wrong here as it definitely seems to be something to do with JVM + APacheBench. I have tried this with Java 11 and 15 but it's the same result.

Java Thread Sample (hello.html can be any HTML file)

import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;

public class JavaHTTPServerCopy {
    public static void main(String[] args) {
        int port = 8080;
        try (ServerSocket serverSocket = new ServerSocket(port)) {
            System.out.println("Server is listening on port " + port);
            while (true) {
                new ServerThreadCopy(serverSocket.accept()).start();
            }
        } catch (IOException ex) {
            System.out.println("Server exception: " + ex.getMessage());
        }
    }
}

class ServerThreadCopy extends Thread {

    private final Socket socket;

    public ServerThreadCopy(Socket socket) {
        this.socket = socket;
    }

    @Override
    public void run() {
        var file = new File("hello.html");
        try (
                // we get character output stream to client (for headers)
                var out = new PrintWriter(socket.getOutputStream());
                // get binary output stream to client (for requested data)
                var dataOut = new BufferedOutputStream(socket.getOutputStream());
                var fileIn = new FileInputStream(file)
        ) {
            var fileLength = (int) file.length();
            var fileData = new byte[fileLength];
            int read = fileIn.read(fileData);
            System.out.println("Responding with Content-length: " + read);
            var contentMimeType = "text/html";
            // send HTTP Headers
            out.println("HTTP/1.1 200 OK");
            out.println("Connection: keep-alive");
            out.println("Content-type: " + contentMimeType);
            out.println("Content-length: " + fileLength);
            out.println(); // blank line between headers and content, very important !
            out.flush(); // flush character output stream buffer

            dataOut.write(fileData, 0, fileLength);
            dataOut.flush();
        } catch (Exception ex) {
            System.err.println("Error with exception : " + ex);
        } finally {
            try {
                socket.close(); // we close socket connection
            } catch (Exception e) {
                System.err.println("Error closing stream : " + e.getMessage());
            }
        }
    }
}

Error on console

Responding with Content-length: 176
Error with exception : java.net.SocketException: Broken pipe (Write failed)
Error with exception : java.net.SocketException: Broken pipe (Write failed)
Error with exception : java.net.SocketException: Broken pipe (Write failed)
Error with exception : java.net.SocketException: Broken pipe (Write failed)
Error with exception : java.net.SocketException: Broken pipe (Write failed)

ApacheBench output

ab -c 100 -n 1000 http://localhost:8080/ 

This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
apr_socket_recv: Connection reset by peer (104)

Java Async Sample

import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.StandardSocketOptions;
import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousServerSocketChannel;
import java.nio.channels.AsynchronousSocketChannel;
import java.nio.channels.CompletionHandler;

public class JavaAsyncHTTPServer {

    public static void main(String[] args) throws Exception {
        new JavaAsyncHTTPServer().go();
        Thread.currentThread().join();//Wait forever
    }

    private void go() throws IOException {
        AsynchronousServerSocketChannel server = AsynchronousServerSocketChannel.open();
        InetSocketAddress hostAddress = new InetSocketAddress("localhost", 8080);
        server.bind(hostAddress);
        server.setOption(StandardSocketOptions.SO_REUSEADDR, true);
        System.out.println("Server channel bound to port: " + hostAddress.getPort());

        if (server.isOpen()) {
            server.accept(null, new CompletionHandler<>() {
                @Override
                public void completed(final AsynchronousSocketChannel result, final Object attachment) {
                    if (server.isOpen()) {
                        server.accept(null, this);
                    }
                    handleAcceptConnection(result);
                }

                @Override
                public void failed(final Throwable exc, final Object attachment) {
                    if (server.isOpen()) {
                        server.accept(null, this);
                        System.out.println("Connection handler error: " + exc);
                    }
                }
            });
        }
    }

    private void handleAcceptConnection(final AsynchronousSocketChannel ch) {
        var content = "Hello Java!";
        var message = ("HTTP/1.0 200 OK\n" +
                "Connection: keep-alive\n" +
                "Content-length: " + content.length() + "\n" +
                "Content-Type: text/html; charset=utf-8\r\n\r\n" +
                content).getBytes();
        var buffer = ByteBuffer.wrap(message);
        ch.write(buffer);
        try {
            ch.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

No error on console

ApacheBench output

❯ ab -c 100 -n 1000 http://localhost:8080/

This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
apr_socket_recv: Connection reset by peer (104)

ApacheBench output with keep-alive

 ab -k -c 100 -n 1000 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
apr_socket_recv: Connection reset by peer (104)
Total of 37 requests completed

Answer 1

So, thanks to the comments and answers here and on Twitter, the first code sample is fixed now. The issue was writing to the TCP stream before reading it. Thanks to Ganesh for the original solution on this here and the explanation is on this SO answer

So here is the updated code that works for Java Thread Sample

import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;

public class JavaHTTPServer {
    public static void main(String[] args) {
        var count = 0;
        var port = 8080;
        try (var serverSocket = new ServerSocket(port, 100)) {
            System.out.println("Server is listening on port " + port);
            while (true) {
                count++;
                new ServerThread(serverSocket.accept(), count).start();
            }
        } catch (IOException ex) {
            System.out.println("Server exception: " + ex.getMessage());
        }
    }
}

class ServerThread extends Thread {

    private final Socket socket;
    private final int count;

    public ServerThread(Socket socket, int count) {
        this.socket = socket;
        this.count = count;
    }

    @Override
    public void run() {
        var file = new File("hello.html");
        try (
                var in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
                // we get character output stream to client (for headers)
                var out = new PrintWriter(socket.getOutputStream());
                // get binary output stream to client (for requested data)
                var dataOut = new BufferedOutputStream(socket.getOutputStream());
                var fileIn = new FileInputStream(file)
        ) {
            // add 2 second delay to every 10th request
            if (count % 10 == 0) {
                System.out.println("Adding delay. Count: " + count);
                Thread.sleep(2000);
            }

            // read the request fully to avoid connection reset errors and broken pipes
            while (true) {
                String requestLine = in.readLine();
                if (requestLine == null || requestLine.length() == 0) {
                    break;
                }
            }

            var fileLength = (int) file.length();
            var fileData = new byte[fileLength];
            fileIn.read(fileData);

            var contentMimeType = "text/html";
            // send HTTP Headers
            out.println("HTTP/1.1 200 OK");
            out.println("Content-type: " + contentMimeType);
            out.println("Content-length: " + fileLength);
            out.println("Connection: keep-alive");

            out.println(); // blank line between headers and content, very important !
            out.flush(); // flush character output stream buffer

            dataOut.write(fileData, 0, fileLength); // write the file data to output stream
            dataOut.flush();
        } catch (Exception ex) {
            System.err.println("Error with exception : " + ex);
        }
    }
}

and apacheBench output

ab -r -c 100 -n 1000 http://127.0.0.1:8080/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /
Document Length:        176 bytes

Concurrency Level:      100
Time taken for tests:   2.385 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      260000 bytes
HTML transferred:       176000 bytes
Requests per second:    419.21 [#/sec] (mean)
Time per request:       238.546 [ms] (mean)
Time per request:       2.385 [ms] (mean, across all concurrent requests)
Transfer rate:          106.44 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   1.8      0       8
Processing:     0  221 600.7     21    2058
Waiting:        0  220 600.8     21    2057
Total:          0  221 600.8     21    2058

Percentage of the requests served within a certain time (ms)
  50%     21
  66%     33
  75%     38
  80%     43
  90%   2001
  95%   2020
  98%   2036
  99%   2044
 100%   2058 (longest request)

I'm gonna try and fix the second Async sample the same way

EDIT: Fixed the Async sample as well

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.StandardSocketOptions;
import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousServerSocketChannel;
import java.nio.channels.AsynchronousSocketChannel;
import java.nio.channels.CompletionHandler;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.ExecutionException;

public class JavaAsyncHTTPServer {

    public static void main(String[] args) throws Exception {
        new JavaAsyncHTTPServer().start();
        Thread.currentThread().join(); // Wait forever
    }

    private void start() throws IOException {
        // we shouldn't use try with resource here as it will kill the stream
        var server = AsynchronousServerSocketChannel.open();
        var hostAddress = new InetSocketAddress("127.0.0.1", 8080);
        server.bind(hostAddress, 100);   // bind listener
        server.setOption(StandardSocketOptions.SO_REUSEADDR, true);
        System.out.println("Server is listening on port 8080");

        final int[] count = {0}; // count used to introduce delays

        // listen to all incoming requests
        server.accept(null, new CompletionHandler<>() {
            @Override
            public void completed(final AsynchronousSocketChannel result, final Object attachment) {
                if (server.isOpen()) {
                    server.accept(null, this);
                }
                count[0]++;
                handleAcceptConnection(result, count[0]);
            }

            @Override
            public void failed(final Throwable exc, final Object attachment) {
                if (server.isOpen()) {
                    server.accept(null, this);
                    System.out.println("Connection handler error: " + exc);
                }
            }
        });
    }

    private void handleAcceptConnection(final AsynchronousSocketChannel ch, final int count) {
        var file = new File("hello.html");
        try (var fileIn = new FileInputStream(file)) {
            // add 2 second delay to every 10th request
            if (count % 10 == 0) {
                System.out.println("Adding delay. Count: " + count);
                Thread.sleep(2000);
            }
            if (ch != null && ch.isOpen()) {
                // Read the first 1024 bytes of data from the stream
                final ByteBuffer buffer = ByteBuffer.allocate(1024);
                // read the request fully to avoid connection reset errors
                ch.read(buffer).get();

                // read the HTML file
                var fileLength = (int) file.length();
                var fileData = new byte[fileLength];
                fileIn.read(fileData);

                // send HTTP Headers
                var message = ("HTTP/1.1 200 OK\n" +
                        "Connection: keep-alive\n" +
                        "Content-length: " + fileLength + "\n" +
                        "Content-Type: text/html; charset=utf-8\r\n\r\n" +
                        new String(fileData, StandardCharsets.UTF_8)
                ).getBytes();

                // write the to output stream
                ch.write(ByteBuffer.wrap(message)).get();

                buffer.clear();
                ch.close();
            }
        } catch (IOException | InterruptedException | ExecutionException e) {
            System.out.println("Connection handler error: " + e);
        }
    }
}

Answer 2

Wanted to ask if tried with setting 127.0.0.1 instead of localhost

InetAddress addr = InetAddress.getByName("127.0.0.1"); ServerSocket sock = new ServerSocket(1234, 50, addr);

Answer 3

I took your "Java Thread Sample" code and ran it on macOS 11.1 in IntelliJ IDEA and then ran the default ab binary installed on my machine - the output of the test run is similar to yours:

➜ ab -c 100 -n 1000 http://127.0.0.1:8080/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
apr_socket_recv: Connection reset by peer (54)

After researching for a bit, the dominant info I gathered pointed me in the direction that the ab binary on macOS might be broken. I went down the road of running the ab binary in a docker container.

➜ docker run --rm jordi/ab -k -c 100 -n 1000 http://host.docker.internal:8080/
This is ApacheBench, Version 2.3 <$Revision: 1826891 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking host.docker.internal (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:
Server Hostname:        host. docker.internal
Server Port:            8080

Document Path:          /
Document Length:        7 bytes

Concurrency Level:      100
Time taken for tests:   0.343 seconds
Complete requests:      1000
Failed requests:        495
   (Connect: 0, Receive: 0, Length: 495, Exceptions: 0)
Keep-Alive requests:    505
Total transferred:      45657 bytes
HTML transferred:       3591 bytes
Requests per second:    2913.39 [#/sec] (mean)
Time per request:       34.324 [ms] (mean)
Time per request:       0.343 [ms] (mean, across all concurrent requests)
Transfer rate:          129.90 [Kbytes/sec] received

Connection Times (ms)
               min  mean[+/-sd] median   max
Connect:        0   15  15.5     20      48
Processing:     0   15  11.1     11      43
Waiting:        0   13  13.6     10      43
Total:          0   31  25.4     33      75

Percentage of the requests served within a certain time (ms)
  50%     33
  66%     46
  75%     57
  80%     59
  90%     65
  95%     68
  98%     73
  99%     74
 100%     75 (longest request)

Might be worth trying a different version/revision of ApacheBench in your tests.

EDIT: When omitting the -k (keep-alive flag) while running ApacheBench, the output is

➜ docker run --rm jordi/ab -c 100 -n 1000 
http://host.docker.internal:8080/
This is ApacheBench, Version 2.3 <$Revision: 1826891 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, 
http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking host.docker.internal (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:
Server Hostname:        host.docker.internal
Server Port:            8080

Document Path:          /
Document Length:        7 bytes

Concurrency Level:      100
Time taken for tests:   0.934 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      90157 bytes
HTML transferred:       7091 bytes
Requests per second:    1070.17 [#/sec] (mean)
Time per request:       93.444 [ms] (mean)
Time per request:       0.934 [ms] (mean, across all concurrent requests)
Transfer rate:          94.22 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        2   40  15.1     35      85
Processing:    26   48  14.2     44     110
Waiting:       13   36  12.0     33      79
Total:         31   88  20.1     86     144

Percentage of the requests served within a certain time (ms)
  50%     86
  66%     92
  75%    101
  80%    105
  90%    123
  95%    129
  98%    131
  99%    138
 100%    144 (longest request)

So no failed requests.

Answer 4

Looks like a bug on your side to me.

Your response is HTTP/1.0 + "Connection: keep-alive", meaning you're advertising the client that it can reuse the connection for performing other requests. And yet, you're closing the socket right after writing the response.

As a result, as network is not instantaneous, the client is trying to reuse the socket and write a second request, just to get the door slammed on its nose.

Either stop closing the socket on each response, or stop enforcing "Connection: keep-alive" (close is the default on HTTP/1.0).