How can I prevent the memory from growing by using the boost log?

Question

Running the sample code of boost log in this link will continue to increase memory usage. I think it is a memory leak.

boost lib version : 1.64.0

This is the piece of code I'm checking more carefully
/*
 *          Copyright Andrey Semashev 2007 - 2013.
 * Distributed under the Boost Software License, Version 1.0.
 *    (See accompanying file LICENSE_1_0.txt or copy at
 *          http://www.boost.org/LICENSE_1_0.txt)
 */
#include <stdexcept>
#include <stdio.h>
#include <string>
#include <iostream>


#include <fstream>
#include <boost/ref.hpp>
#include <boost/bind.hpp>
#include <boost/thread/thread.hpp>
#include <boost/thread/barrier.hpp>

#include <boost/smart_ptr/shared_ptr.hpp>
#include <boost/date_time/posix_time/posix_time.hpp>

#include <boost/log/common.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/attributes.hpp>
#include <boost/log/sources/logger.hpp>
#include <boost/log/sinks.hpp>
//#include <boost/log/sinks/sync_frontend.hpp>
//#include <boost/log/sinks/text_file_backend.hpp>

namespace logging = boost::log;
namespace attrs = boost::log::attributes;
namespace src = boost::log::sources;
namespace sinks = boost::log::sinks;
namespace expr = boost::log::expressions;
namespace keywords = boost::log::keywords;

using boost::shared_ptr;

enum
{
    LOG_RECORDS_TO_WRITE = 10000,
    THREAD_COUNT = 19
};


//BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT(tmp_logger, src::logger_mt)
BOOST_LOG_INLINE_GLOBAL_LOGGER_DEFAULT(test_lg, src::logger_mt)

//! This function is executed in multiple threads
void thread_fun(boost::barrier& bar)
{
    // Wait until all threads are created
    bar.wait();

    // Now, do some logging
    // modify code start
    while(1) {
        BOOST_LOG(test_lg::get()) << "Log record Log record Log record Log record Log record Log record Log record Log record ";
    }
    // modify code end
}

int main(int, char*[])
{
    try
    {
        // Open a rotating text file
        shared_ptr< std::ostream > strm(new std::ofstream("test.log"));

        if (!strm->good()) {
            throw std::runtime_error("Failed to open a text log file");
        }

        // Create a text file sink
        shared_ptr< sinks::synchronous_sink< sinks::text_ostream_backend > > sink(
                new sinks::synchronous_sink< sinks::text_ostream_backend >
                );

        sink->locked_backend()->add_stream(strm);

        sink->set_formatter(
             expr::format("%1%: [%2%] [%3%] - %4%")
             % expr::attr< unsigned int >("RecordID")
             % expr::attr< boost::posix_time::ptime >("TimeStamp")
             % expr::attr< attrs::current_thread_id::value_type >("ThreadID")
             % expr::smessage
            );

        // Add it to the core
        logging::core::get()->add_sink(sink);

        // Add some attributes too
        logging::core::get()->add_global_attribute("TimeStamp", attrs::local_clock());
        logging::core::get()->add_global_attribute("RecordID", attrs::counter< unsigned int >());
        logging::core::get()->add_global_attribute("ThreadID", attrs::current_thread_id());

        // Create logging threads
        boost::barrier bar(THREAD_COUNT);
        boost::thread_group threads;

        for (unsigned int i = 0; i < THREAD_COUNT; ++i)
            threads.create_thread(boost::bind(&thread_fun, boost::ref(bar)));

        // Wait until all action ends
        threads.join_all();
    }
    catch (std::exception& e)
    {
        std::cout << "FAILURE: " << e.what() << std::endl;
        return 1;
    }
    return 0;
}

The amount of free size continues to decrease while the while loop continues. The free capacity at the start time (18:50:56) is 28,331KB and after 55 minutes(19:45:14) is 13,449KB, so a difference of about 15MB

EDIT : /proc/meminfo result of "MemFree" line

It continues to shrink, causing oom, eventually kernel panic, and rebooting.

How to fix this issue? I'm looking forward to hearing your comments and advices

Answer 1

Memory verification was done using "MemFree" value in "cat /proc/meminfo"

That's not a reliable way to measure memory usage of a process.

Instead, run your code under a memory profiler, like valgrind --tool=massif.

I ran your modified example, but with 20 threads, under Massif for 10 minutes, and it clearly indicates no growth:

Notes:

1. you might have another process leaking memory
2. you might have the logfile itself (implicitly) in tmpfs, making available memory shrink
3. there might have been a bug in older versions of boost (I tested with boost 1.67)