#1212 MUC does not recover from unavailable SQL server during init

Reporter Dennis Schridde
Owner Zash
Created
Updated
Stars ★ (1)
Tags
  • Milestone-0.11
  • Priority-Medium
  • Status-Fixed
  • Type-Defect
  1. Dennis Schridde on

    What steps will reproduce the problem? 1. Configure `sql` as `default_storage` 2. Have the MySQL server be unavailable during Prosody startup What is the expected output? All Prosody modules should use the same database connection, which is being reestablished with exponential backoff, if it fails. All modules will be reactivated once the database connection is reestablished. This is also and especially true for initialisation. What do you see instead? MUC (and possibly others) become unavailable, if establishing a database connection during initialisation fails. What version of the product are you using? On what operating system? 0.10.2 on Linux Please provide any additional information below. sql: Error in SQL transaction: Failed to connect to database: Can't connect to MySQL server on '$DATABASE_HOSTNAME' (111) $JVUD_HOSTNAME:storage_sql: Unable to read from database user_index store for <host>: Failed to connect to database: Can't connect to MySQL server on '$DATABASE_HOSTNAME' (111) mod_log_slow_events: Slow event 'host-activated' took 0.71s: no recognised data sql: Error in SQL transaction: Failed to connect to database: Can't connect to MySQL server on '$DATABASE_HOSTNAME' (111) $MUC_HOSTNAME:storage_sql: Unable to read from database persistent store for <host>: Failed to connect to database: Can't connect to MySQL server on '$DATABASE_HOSTNAME' (111) $MUC_HOSTNAME:muc: Error loading list of persistent rooms from storage. Reload mod_muc or restart to recover. modulemanager: Error initializing module 'muc' on '$MUC_HOSTNAME': /usr/lib64/prosody/modules/muc/mod_muc.lua:44: Storage error: Failed to connect to database: Can't connect to MySQL server on '$DATABASE_HOSTNAME' (111) stack traceback: /usr/lib64/prosody/core/modulemanager.lua:31: in function </usr/lib64/prosody/core/modulemanager.lua:31> [C]: in function 'error' /usr/lib64/prosody/modules/muc/mod_muc.lua:44: in main chunk [C]: in function 'pcall' /usr/lib64/prosody/core/modulemanager.lua:177: in function 'do_load_module' /usr/lib64/prosody/core/modulemanager.lua:255: in function 'load' /usr/lib64/prosody/core/modulemanager.lua:74: in function </usr/lib64/prosody/core/modulemanager.lua:49> /usr/lib64/prosody/util/events.lua:78: in function 'handlers' ...sody/modules/mod_log_slow_events/mod_log_slow_events.lua:12: in function 'fire_event' /usr/lib64/prosody/core/hostmanager.lua:116: in function 'activate' /usr/lib64/prosody/core/hostmanager.lua:59: in function </usr/lib64/prosody/core/hostmanager.lua:50> /usr/lib64/prosody/util/events.lua:78: in function 'fire_event' /usr/bin/prosody:358: in function 'prepare_to_start' /usr/bin/prosody:427: in main chunk [C]: at 0x56367c6aa3d0 sql: Error in SQL transaction: Failed to connect to database: Can't connect to MySQL server on '$DATABASE_HOSTNAME' (111) $MUC_HOSTNAME:storage_sql: Unable to read from database persistent store for <host>: Failed to connect to database: Can't connect to MySQL server on '$DATABASE_HOSTNAME' (111) $MUC_HOSTNAME:muc: Error loading list of persistent rooms from storage. Reload mod_muc or restart to recover. modulemanager: Error initializing module 'muc' on '$MUC_HOSTNAME': /usr/lib64/prosody/modules/muc/mod_muc.lua:44: Storage error: Failed to connect to database: Can't connect to MySQL server on '$DATABASE_HOSTNAME' (111) stack traceback: /usr/lib64/prosody/core/modulemanager.lua:31: in function </usr/lib64/prosody/core/modulemanager.lua:31> [C]: in function 'error' /usr/lib64/prosody/modules/muc/mod_muc.lua:44: in main chunk [C]: in function 'pcall' /usr/lib64/prosody/core/modulemanager.lua:177: in function 'do_load_module' /usr/lib64/prosody/core/modulemanager.lua:255: in function 'load' /usr/lib64/prosody/core/moduleapi.lua:170: in function 'depends' /usr/lib64/prosody/modules/mod_mam_muc/mod_mam_muc.lua:27: in main chunk [C]: in function 'pcall' /usr/lib64/prosody/core/modulemanager.lua:177: in function 'do_load_module' /usr/lib64/prosody/core/modulemanager.lua:255: in function 'load' /usr/lib64/prosody/core/modulemanager.lua:77: in function </usr/lib64/prosody/core/modulemanager.lua:49> /usr/lib64/prosody/util/events.lua:78: in function 'handlers' ...sody/modules/mod_log_slow_events/mod_log_slow_events.lua:12: in function 'fire_event' /usr/lib64/prosody/core/hostmanager.lua:116: in function 'activate' /usr/lib64/prosody/core/hostmanager.lua:59: in function </usr/lib64/prosody/core/hostmanager.lua:50> /usr/lib64/prosody/util/events.lua:78: in function 'fire_event' /usr/bin/prosody:358: in function 'prepare_to_start' /usr/bin/prosody:427: in main chunk [C]: at 0x56367c6aa3d0 modulemanager: Error initializing module 'mam_muc' on '$MUC_HOSTNAME': /usr/lib64/prosody/modules/mod_mam_muc/mod_mam_muc.lua:27: Unable to load required module, mod_muc: /usr/lib64/prosody/modules/muc/mod_muc.lua:44: Storage error: Fail> stack traceback: /usr/lib64/prosody/core/modulemanager.lua:31: in function </usr/lib64/prosody/core/modulemanager.lua:31> [C]: in function 'error' /usr/lib64/prosody/modules/muc/mod_muc.lua:44: in main chunk [C]: in function 'pcall' /usr/lib64/prosody/core/modulemanager.lua:177: in function 'do_load_module' /usr/lib64/prosody/core/modulemanager.lua:255: in function 'load' /usr/lib64/prosody/core/moduleapi.lua:170: in function 'depends' /usr/lib64/prosody/modules/mod_mam_muc/mod_mam_muc.lua:27: in main chunk [C]: in function 'pcall' /usr/lib64/prosody/core/modulemanager.lua:177: in function 'do_load_module' /usr/lib64/prosody/core/modulemanager.lua:255: in function 'load' /usr/lib64/prosody/core/modulemanager.lua:77: in function </usr/lib64/prosody/core/modulemanager.lua:49> /usr/lib64/prosody/util/events.lua:78: in function 'handlers' ...sody/modules/mod_log_slow_events/mod_log_slow_events.lua:12: in function 'fire_event' /usr/lib64/prosody/core/hostmanager.lua:116: in function 'activate' /usr/lib64/prosody/core/hostmanager.lua:59: in function </usr/lib64/prosody/core/hostmanager.lua:50> /usr/lib64/prosody/util/events.lua:78: in function 'fire_event' /usr/bin/prosody:358: in function 'prepare_to_start' /usr/bin/prosody:427: in main chunk [C]: at 0x56367c6aa3d0 stack traceback: /usr/lib64/prosody/core/modulemanager.lua:31: in function </usr/lib64/prosody/core/modulemanager.lua:31> [C]: in function 'depends' /usr/lib64/prosody/modules/mod_mam_muc/mod_mam_muc.lua:27: in main chunk [C]: in function 'pcall' /usr/lib64/prosody/core/modulemanager.lua:177: in function 'do_load_module' /usr/lib64/prosody/core/modulemanager.lua:255: in function 'load' /usr/lib64/prosody/core/modulemanager.lua:77: in function </usr/lib64/prosody/core/modulemanager.lua:49> /usr/lib64/prosody/util/events.lua:78: in function 'handlers' ...sody/modules/mod_log_slow_events/mod_log_slow_events.lua:12: in function 'fire_event' /usr/lib64/prosody/core/hostmanager.lua:116: in function 'activate' /usr/lib64/prosody/core/hostmanager.lua:59: in function </usr/lib64/prosody/core/hostmanager.lua:50> /usr/lib64/prosody/util/events.lua:78: in function 'fire_event' /usr/bin/prosody:358: in function 'prepare_to_start' /usr/bin/prosody:427: in main chunk [C]: at 0x56367c6aa3d0 [...] sql: Error in SQL transaction: Failed to connect to database: Can't connect to MySQL server on '$DATABASE_HOSTNAME' (111)

  2. Zash on

    This should already be the case in trunk/0.11, where rooms are loaded on-demand instead of all at once on startup. What you are seeing here is the fix for #1091. Read and follow the error message: > Error loading list of persistent rooms from storage. Reload mod_muc or restart to recover. We decided against backporting the from trunk due to how different that code base has become, instead opting to this hard failure with the above suggestion to reload/restart.

    Changes
    • tags Milestone-0.11 Status-Fixed
    • owner Zash

New comment

Not published. Used for spam prevention and optional update notifications.