#1545 mod_mam and mod_muc_mam lag significantly on internal storage

Reporter ge0rg
Owner Zash
Stars ★ (1)
  • Priority-Medium
  • Status-Started
  • Milestone-0.12
  • Type-Defect
  1. ge0rg on

    Running prosody-trunk with mod_mam and mod_muc_mam (both built-in), the server will lag significantly (read: multiple seconds) each time a user or MUC hits the MAM quota, which causes the history to be "shifted" by one item, causing a full rewrite of the backend file. One of the affected files on my server is 52MB, and it takes 3~5s of blocking wall-clock time to load, deserialize, remove the first item, serialize, and to store it back. On *each* *new* *message* to that MUC: May 05 16:53:13 chat.yax.im:storage_internal debug someroom reached or over quota, not adding to store May 05 16:53:13 chat.yax.im:muc_mam debug User 'someroom' over quota, truncating archive May 05 16:53:15 chat.yax.im:storage_internal debug someroom has 9999 items out of 10000 limit in store muc_log This is a very nice DoS to the server, which can't do anything else during that activity. While I understand that storage_internal is not your favorite child, it looks like it's the default (which it shouldn't be), and the modules don't refuse to load on it (which they probably should). A quick workaround would be to change the quota enforcement from one-message-at-a-time to something like 1%-a-time or 5%-a-time, here: https://hg.prosody.im/trunk/file/tip/plugins/mod_muc_mam.lua#l400 and in the respective place in mod_mam. Thank you.

  2. Zash on

    Thanks. https://hg.prosody.im/trunk/rev/62794e065e33 sets the MAM modules to delete 1% of messages. I'll leave this as Started while we evaluate how that performs.

    • owner Zash
    • tags Status-Started Milestone-0.12

New comment

Not published. Used for spam prevention and optional update notifications.