#1376 Memory leak

Reporter Joerg Morbitzer
Owner Nobody
Created
Updated
Stars ★ (1)
Tags
  • Status-Accepted
  • Priority-Medium
  • Type-Defect
  1. Joerg Morbitzer on

    Hi, I am running Prosody 0.11.2 on Debian Stretch 64 Bit., and the memory usage increases over time, I have to restart the server once a week to save my server from crashing (after a week the Prosody server consumes around 4 to 6 GB). So... I entered the debug mode, ran a small script to trace the memory usage and waited for the next incidence when the process used more memory again, here we go: In exact this second the memory usage went up from 256084 to 256612 KB: Jun 10 13:45:48 s2sin55e5691c3880 debug Received[s2sin]: <presence xml:lang='en' from='ABC@xmpp.jp/1056441226866303795374674' to='XYZ@jabber-germany.de'> Jun 10 13:45:48 rostermanager debug load_roster: asked for: XYZ@jabber-germany.de Jun 10 13:45:48 rostermanager debug load_roster: loading for offline user: XYZ@jabber-germany.de Please find the enabled modules below, storage is configured ad: storage = { archive2 = "xmlarchive" } archive_store = "archive2" archive_expires_after = "2w" In case the logfile snippet does not help, what other chance do I have to find the "memory eating" culprit? Kind regards, and thanks in advance, Joerg. modules_enabled = { "roster"; -- Allow users to have a roster. Recommended ;) "saslauth"; -- Authentication for clients and servers. Recommended if you want to log in. "tls"; -- Add support for secure TLS on c2s/s2s connections "dialback"; -- s2s dialback support "disco"; -- Service discovery "private"; -- Private XML storage (for room bookmarks, etc.) "vcard4"; -- Allow users to set vCards "vcard_legacy"; "version"; -- Replies to server version requests "uptime"; -- Report how long server has been running "time"; -- Let others know the time here on this server "ping"; -- Replies to XMPP pings with pongs "pep"; -- Enables users to publish their mood, activity, playing music and more "pep_simple"; "register"; -- Allow users to register on this server using a client and change passwords "smacks"; -- XEP-0198 "smacks_offline"; "carbons"; -- XEP-0280 "lastactivity"; -- XEP-0012 "cloud_notify"; -- XEP-0357 "csi"; -- XEP-0352 "blocking"; -- XEP-0191 "http_upload"; -- XEP-0363 "mam"; -- XEP-0313 "track_muc_joins"; "lastlog"; "pep_vcard_avatar"; "csi_battery_saver"; "log_auth"; "admin_adhoc"; -- Allows administration via an XMPP client that supports ad-hoc commands "admin_telnet"; -- Opens telnet console interface on localhost port 5582 "posix"; -- POSIX functionality, sends server to background, enables syslog, etc. "announce"; -- Send announcement to all online users "welcome"; -- Welcome users who register accounts "legacyauth"; -- Legacy authentication. Only used by some old clients and bots. "storage_xmlarchive"; "pubsub_hub"; "privacy_lists"; -- XEP-0016 "server_contact_info"; -- XEP-0157 "e2e_policy"; "http"; "bookmarks"; "register_web"; "conversejs"; };

  2. Zash on

    Thanks for the report! We are currently investigating better ways to gather the info needed to debug memory related issues. This additional info may help.

    Changes
    • tags Status-Accepted
  3. Joerg Morbitzer on

    Hi Zash, saw that new module "mod_log_events_by_memory", which I loaded straight away. So I have another example, exactly within this second 07:12:42h the operating system showed an inreased memory usage, went up from by 528 KB, and shortly after by another 956 KB, and the logfile is showing this: Jun 12 07:12:41 s2sin55ac5fbb5d20 debug Received[s2sin]: <presence type='unavailable' to='ABC@jabber-germany.de' from='DEF@jabbim.com/Psi+'> Jun 12 07:12:42 s2sin55ac5fbb5d20 debug Received[s2sin]: <presence type='unavailable' to='JKL@jabber-germany.de' from='DEF@jabbim.com/Psi+'> Jun 12 07:12:42 socket debug try to start ssl at client id: 55ac63213780 Jun 12 07:12:42 socket debug accepted incoming client connection from: 205.201.55.121 11045 to 5223 Jun 12 07:12:42 socket debug starting handshake... Jun 12 07:12:42 socket debug ssl handshake of client with id:table: 0x55ac63213780, attempt:1 Jun 12 07:12:42 socket debug ssl handshake of client with id:table: 0x55ac63213780, attempt:2 Jun 12 07:12:42 socket debug ssl handshake done Jun 12 07:12:42 socket debug try to start ssl at client id: 55ac60b1bb20 Jun 12 07:12:42 socket debug accepted incoming client connection from: 109.40.66.227 12615 to 5223 Jun 12 07:12:42 c2s55ac63067c90 info Client connected Jun 12 07:12:42 socket debug starting handshake... Jun 12 07:12:42 socket debug ssl handshake of client with id:table: 0x55ac60b1bb20, attempt:1 Jun 12 07:12:42 socket debug ssl handshake of client with id:table: 0x55ac60b1bb20, attempt:2 Jun 12 07:12:42 s2sin55ac6095b120 debug Received[s2sin]: <presence xml:lang='en' from='XYZ@sj.ms/Psi+' to='GHI@jabber-germany.de'> Jun 12 07:12:42 rostermanager debug load_roster: asked for: GHI@jabber-germany.de Jun 12 07:12:42 rostermanager debug load_roster: loading for offline user: GHI@jabber-germany.de Jun 12 07:12:42 mod_log_events_by_memory warn Memory increased by 1.10406e+06 bytes while processing event 'presence/bare' Jun 12 07:12:42 c2s55ac63067c90 debug Client sent opening <stream:stream> to jabber-germany.de Jun 12 07:12:42 c2s55ac63067c90 debug Sending[c2s_unauthed]: <?xml version='1.0'?> Jun 12 07:12:42 c2s55ac63067c90 debug Sending[c2s_unauthed]: <stream:stream xmlns:stream='http://etherx.jabber.org/streams' xml:lang='en' from='jabber-germany.de' id='91e051ff-284b-4111-9f0e-984ad36a6b3e' version='1 .0' xmlns='jabber:client'> Jun 12 07:12:42 c2s55ac63067c90 debug Sent reply <stream:stream> to client Jun 12 07:12:42 c2s55ac63067c90 debug Offering mechanism PLAIN Jun 12 07:12:42 c2s55ac63067c90 debug Offering mechanism SCRAM-SHA-1 Jun 12 07:12:42 c2s55ac63067c90 debug Offering mechanism SCRAM-SHA-1-PLUS Jun 12 07:12:42 c2s55ac63067c90 debug Sending[c2s_unauthed]: <stream:features> Jun 12 07:12:43 socket debug ssl handshake of client with id:table: 0x55ac60b1bb20, attempt:3 And I think this mod_log_events_by_memory output is very output triggered by event "'presence/bare', which is near "load_roster: loading for offline user ..." Kind regards, Joerg.

  4. Zash on

    Just keep in mind that garbage collection is used so it's very hard to know if these short term allocations are "leaked".

  5. Joerg Morbitzer on

    Hi Zash, I just happened to see that the memory leak seems to be gone on my installation (running 0.11.4 in the meantime, since a while), so at some point - between the time when I opened the ticket and today - a module or whatever must have changed that kind of "fixed" the memory leak issue. Kind regards, Joerg.

New comment

Not published. Used for spam prevention and optional update notifications.