Huge memory leak #323
Comments
|
What do your message handler and tls_init_handler look like? |
|
tls_init is almost a copy-paste from an echo_server_tls example: context_ptr WebSocketServer::on_tls_init(websocketpp::connection_hdl /*hdl*/)
{
context_ptr ctx(new boost::asio::ssl::context(boost::asio::ssl::context::tlsv1));
try
{
ctx->set_options(boost::asio::ssl::context::default_workarounds |
boost::asio::ssl::context::no_sslv2 |
boost::asio::ssl::context::single_dh_use);
ctx->use_certificate_chain_file(Config().GetCertChainFile());
ctx->set_password_callback(std::bind(&WebSocketServer::get_password, this));
ctx->use_private_key_file(Config().GetKeyFile(), boost::asio::ssl::context::pem);
}
catch (const std::exception& e)
{
g_log.log(L_ERROR, __FUNCTION__" %s", e.what());
}
return ctx;
}
void WebSocketServer::on_message(connection_hdl hdl, server::message_ptr msg)
{
std::error_code ec;
connection_ptr con = m_server.get_con_from_hdl(hdl, ec);
if (ec)
return;
if (msg->get_opcode() == websocketpp::frame::opcode::text)
{
const std::string& response = HandleClientMsg(msg->get_payload());
if (!response.empty())
con->send(response, websocketpp::frame::opcode::text);
}
} |
|
oops, missclicked trying to find a 'code beautifyer' |
|
for code beautification, look at https://help.github.com/articles/github-flavored-markdown. I edited your comment to add a code block + syntax highlighting as an example. WebSocket++ connections do not store the TLS certificate or the contents of individual messages. Looking at your handlers, can you confirm that the |
|
Yes, [2014-02-15 15:46:07] [fatal] error in handle_read_handshake: End of File |
|
Additional logging experiments, I added global atomic_int32 which I increase in connection ctor and decrease in dtor (postfix increment is used so numbers are little odd) [2014-02-15 16:59:44] [application] It seems strange that connection dtor wasn't called for 16:59:54 disconnect. |
|
those logs definitely look strange, I'll see what I can dig up |
|
I thought I'd chip in on the issue with my experience: asio_tls configuration does seem to have this problem, but after a lot of trying to get it to work better I also tried the example boost ssl server/client (no websockets), and while not as bad as ws which naturally adds some overheads, it still ends up with ~600MB for 10k connections, so I don't think the issue is with websocketpp... An option I found to work a lot better in terms of memory consumption was to use stud (https://github.com/bumptech/stud - a really rather thin layer on top of openssl) in front of non-tls websocket - I end up with ~257.5MB used by stud and another 93MB used by websocketpp (example echo server) for 7.5k connections. Re-running echo_server_tls with 7.5k connections gets me up to 780MB total. |
|
Thanks for a hint for a good library. The issue with websocketpp library memory consumption is that a connection is not freed under some circumnstances. The one I'm already familiar with - no connection destructor is called when handle_read_handshake fails. This is log file with added 'connection dtor' logging to visualise the issue (app is build using the latest websocketpp from a trunk) : [2014-03-02 09:00:25] [connect] asio con transport constructor As you can see there were two handle_read_handshake errors but connection destructors are not invoked. This causes memory and handles to be leaked slowly. My app consumes up to 2gb of memory after a 24 hours of uptime because of connections intensity. I had to schedule a daily service restart as a temporarily solution for the issue. |
|
I've reproduced and fixed the memory leak described by droppy here. Specifically the one that caused failed connections to never get destroyed. This would affect both regular and TLS versions, although TLS versions would leak more memory because TLS uses more memory per connection. I'll continue looking into TLS related memory usage. I've found few articles about memory optimizations for asio's TLS client. |
I tried to update old websocket library used in my project to the latest version from 'master' branch (SHA-1: ce2c1d6).
My app uses asio_tls config 'out of the box'. When the app starts it consumes ~170mb, but after 9000 total websocket client connects/disconnects it consumes ~860 mb.
I dumped app's memory to a file and viewed different parts using a binary viewer. It mostly consist of TLS certificates + JSON message bodies my app send to the clients + some binary garbage. I suppose connection_ptrs aren't being destroyed and the memory used is not freed.
My code doesn't store any connection_ptrs on its own. I use VIsual Studio x64 build mode using Intel Compiler 14.1, but I don't think this info is related to the issue.
The text was updated successfully, but these errors were encountered: