#1299 PEP data does not fit into current storage abstraction

Reporter ge0rg
Owner Nobody
Created
Updated
Stars ★★★ (5)
Tags
  • Priority-Medium
  • Status-Blocked
  • Type-Defect
  1. ge0rg on

    With the new mod_pep module, prosody will persist PEP items on disk. The directory structure used for that is not optimal. The below paths are all relative to `/var/lib/prosody/SERVERNAME/` The list of a user's PEP nodes is stored in `pep/USER.dat` which is perfect. Individual nodes, however, are stored under `pep_NODE/USER.dat` leading to a cluttering of the prosody storage directory. This is not only impractical because it will end up with a huge number of different PEP usage cases in the top-level directory, but especially bad with OMEMO, which creates a devicelist node per user and a bundles node *per device*. With just a single account, I have the following files and directories on my server: ``` pep pep/georg.dat pep_eu%2esiacs%2econversations%2eaxolotl%2ebundles%3a1399321121 pep_eu%2esiacs%2econversations%2eaxolotl%2ebundles%3a1399321121/georg.list pep_eu%2esiacs%2econversations%2eaxolotl%2edevicelist pep_eu%2esiacs%2econversations%2eaxolotl%2edevicelist/georg.list ``` Please change the structure to be `pep/USER/NODE.dat` so that it is cleanly scoped, doesn't clutter the top-level storage directory and can be manually inspected per-user. Thank you very much.

  2. ge0rg on

    According to https://prosody.im/doc/storage you can store PEP node configuration and explicitly named nodes into SQL, but there is no obvious way to store _all_ PEP nodes regardless of their name into SQL without the big storage="sql" hammer.

  3. Zash on

    I agree that this is awkward, however this is the corner we've backed into in order to have persistent PEP data at all without having do drastic changes to the internal storage API. Each PEP node needs to be its own logical archive. The current archive API is modeled as an ordered key-value where the key is (user, host, store, archive-id). The current file system data manager library does not support more deeply nested directory trees. There is some in-progress work on this, see #881.

  4. ge0rg on

    For future readers of this, #1320 does provide a workaround for me (move all PEP nodes into SQLite).

  5. Zash on

    Changes
    • tags Status-Blocked
    • title prosody 0.11 mod_pep: non-optimal persistent directory structure PEP data does not fit into current storage abstraction

New comment

Not published. Used for spam prevention and optional update notifications.