Now I want to build a middleware to process requests sent from browser, filter some of them then pass other to server(e.g. the middleware receive stream 1 and stream 3, it blocks 1 then pass 3 to the server).
I try to use GOAWAY to implement this but not ideal. It is expected that browser receives GOAWAY with last stream ID 1 and establish a new connection to send original stream 3. However general circumstance is that it will skip the stream 3 and send following requests

Another problem is that the browser will stop establishing new connection after sending requests for several times
Please give me some advice
HTTP/2 is multiplexed, and one connection carries several HTTP/2 streams, each representing a request/response.
For this reason, you do not want to use GOAWAY, because that tears down the whole connection.
HTTP/2 has another way of canceling a specific stream, via the RST_STREAM frame.
If your middleware application wants to filter out stream #1, it can send back to the browser a RST_STREAM frame for stream #1, and the browser will stop sending data (if any) for that stream.
Stream #3 will be unaffected, as future streams (on the same connection) will also be unaffected.
Your middleware can forward stream #3, #5, filter out #7 with another RST_STREAM, but forward #9, etc.