Golang request.Body.Close() returns an empty Document

Question

I have 2 methods in 2 different packages, where func B() takes a url reads the web page and returns *html.Tokenizer. But the problem is, it is working fine Only when I comment the defer r.Body.Close(), If I enable it this doc returned from func B is empty.

And it also works if both the functions are merged in single function. but I need them in 2 different package.

Any suggestion or Idea that What am I missing here ? shoudn't the res.Body be closed ?

func  (s ParserService) A(u string) (*domain.Result, error) {
    doc, err := s.B("https://www.google.com/")
    if err != nil {
        fmt.Println(err.Error())
    }
    for tokenType := doc.Next(); tokenType != html.ErrorToken; {
        token := doc.Token()
        fmt.Println(token)
        tokenType = doc.Next()
    }
}

func (c Downloader) B(url string) (*html.Tokenizer, error) {
    r, err := c.httpClient.Get(url)
    if err != nil {
        return nil, err
    }
//    defer r.Body.Close()
    doc := html.NewTokenizer(r.Body)
    return doc, nil
}

Answer 1

tl;dr

The html.Tokenier‘s Next method reads directly from the reader. Don’t close the body until you’ve finished processing it through the tokenizer. In your example, you should perform the HTTP request and tokenize the body in the same function, then you can uncomment your deferred close.

Details

html.Tokenizer accepts an io.Reader from which the tokenizer will read until it receives an io.EOF error. This "error" indicates that there is nothing left to be read and the tokenizer source is completed.

http.Request.Body is an io.ReadCloser which is a combination of an io.Reader and an io.Closer. What happens after a call to Close is implementation specific, however for the http.Request.Body, no more bytes can be read from the reader after close is called.

Your problem is ultimately caused by prematurely closing the http.Request.Body (io.ReadCloser).