Convert boost::beast::multibuffer to std::istream

Question

I am getting boost::beast::multibuffer object from http::response<http::dynamic_body>::body() method. Then, I want to parse json content from it like this:

boost::property_tree::read_json(requestBodyStream, propertyTree);

Should I use boost::beast::buffers_to_string and std::stringstream to get requestBodyStream or is it possible to do without spending so much memory on copying the contents of the buffer?

Answer 1

In general, don't program the specific implementation, but program to the concept. Here, dynamic_body documents:

This body uses a DynamicBuffer as a memory-based container for holding message payloads. Messages using this body type may be serialized and parsed.

You don't need that concept, as you will be consuming this entirely in-memory anyways, but if you did, you would go about it like:

net::streambuf sb;
sb.commit(net::buffer_copy(sb.prepare(body.size()), body.cdata()));

std::istream is(&sb);
ptree doc;
read_json(is, doc);

See it Live On Coliru

#include <boost/asio.hpp>
#include <boost/asio/posix/stream_descriptor.hpp>

#include <boost/beast/http.hpp>
#include <boost/property_tree/json_parser.hpp>
#include <iostream>
namespace net = boost::beast::net;
namespace http = boost::beast::http;
using boost::property_tree::ptree;

int main() {
    net::posix::stream_descriptor input(net::system_executor{}, 0); // stdin

    http::response<http::dynamic_body> res;
    {
        net::streambuf readbuf;
        http::read(input, readbuf, res);
    }

    auto& body = res.body();

    net::streambuf sb;
    sb.commit(net::buffer_copy(sb.prepare(body.size()), body.cdata()));

    std::istream is(&sb);
    ptree doc;
    read_json(is, doc);

    write_json(std::cout << "Parsed body: ", doc);
}

It reads a sample response from stdin, let's use

HTTP/1.1 200 OK
Content-Length: 50

{"this":{"is":[1,2,3], "a":"sample"}, "object":42}

Like so:

g++ -std=c++20 -O2 -Wall -pedantic -pthread main.cpp && ./a.out <<< $'HTTP/1.1 200 OK\r\nContent-Length: 50\r\n\r\n{\"this\":{\"is\":[1,2,3], \"a\":\"sample\"}, \"object\":42}'

Prints

Parsed body: {
    "this": {
        "is": [
            "1",
            "2",
            "3"
        ],
        "a": "sample"
    },
    "object": "42"
}

HOWEVER

Now that we've answered the question, let's add context:

• Don't use Boost Property Tree (unless you need Property Trees. Hint: you do not). Look at the output: Property Tree is NOT a JSON library

• Don't use dynamic body unless you need it. In this case you're reading the entire message in memory, copying it in memory (to convert to a streambuf), reading from it using locale-aware istream (slow) and the result lives as another copy in memory.

Instead, use the simplest model you can and use a JSON library, like, you know, Boost.JSON:

Live On Coliru

#include <boost/asio.hpp>
#include <boost/asio/posix/stream_descriptor.hpp>

#include <boost/beast/http.hpp>
#include <boost/json/src.hpp> // for header-only
#include <iostream>
namespace net  = boost::beast::net;
namespace http = boost::beast::http;
namespace json = boost::json;

int main() {
    net::posix::stream_descriptor input(net::system_executor{}, 0); // stdin

    http::response<http::string_body> res;
    {
        net::streambuf readbuf;
        http::read(input, readbuf, res);
    }

    auto doc = json::parse(res.body());
    std::cout << "Parsed body: " << doc << "\n";
}

It's less code, more efficient and most importantly, correct handling of the JSON!