#598 net.adns stops working if it runs into NOFILE limit

Reporter Zash
Owner Zash
Created
Updated
Stars ★ (1)  
Tags
  • Milestone-0.9
  • Type-Defect
  • Status-Fixed
  • Priority-Medium
  1. Zash on

    debug Records for _xmpp-server._tcp.remote.example. not in cache, sending query (thread: 0x18802f0)... error Error sending DNS query: Too many open files debug remote.example has no SRV records, falling back to A/AAAA debug Records for remote.example not in cache, sending query (thread: 0x17eb3e0)... error Error sending DNS query: Too many open files debug Records for remote.example not in cache, sending query (thread: 0x16fbb70)... error Error sending DNS query: Too many open files debug DNS lookup failed to get a response for remote.exampleinfo Out of connection options, can't connect to remote.example debug No other records to try for remote.example - destroying debug Destroying outgoing session example.local->remote.example: DNS resolution failed: Too many open files debug First attempt to connect to remote.example, starting with SRV lookup... debug Records for _xmpp-server._tcp.remote.example. not in cache, sending query (thread: 0x1817df0)... ... then nothing until s2s_timeout

  2. Zash on

    Changes
    • tags Milestone-0.9
  3. Zash on

    This appears to be because net.dns adds an entry into the "waiting for response" set before getting a socket and sending the query, which can fail. The next query sees that it is already waiting for an answer and writes itself up there too. But since the query packet was never sent, no answer is ever received.

    Changes
    • owner Zash
    • tags Status-Started
  4. Zash on

    Fixed in 388281be2bc7

    Changes
    • tags Status-Fixed
  5. s on

    Error still present: May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:47 adns error Error sending DNS query: Too many open files [...] May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files use_libevent = true # prosodyctl about Prosody 0.10.0 # Prosody directories Data directory: /var/lib/prosody Config directory: /etc/prosody Source directory: /usr/lib64/prosody Plugin directories: /opt/prosody-modules - prosody-modules rev: 0fb95dc11bc8 /usr/lib64/prosody/modules/ # Lua environment Lua version: Lua 5.1 Lua module search paths: /usr/lib64/prosody/?.lua /usr/share/lua/5.1/?.lua /usr/share/lua/5.1/?/init.lua /usr/lib64/lua/5.1/?.lua /usr/lib64/lua/5.1/?/init.lua /root/.luarocks/share/lua/5.1/?.lua /root/.luarocks/share/lua/5.1/?/init.lua Lua C module search paths: /usr/lib64/prosody/?.so /usr/lib64/lua/5.1/?.so /usr/lib64/lua/5.1/loadall.so /root/.luarocks/lib64/lua/5.1/?.so LuaRocks: Installed (2.3.0) # Lua module versions lfs: LuaFileSystem 1.6.2 libevent: 2.0.21-stable luaevent: 0.4.4 lxp: LuaExpat 1.3.0 socket: LuaSocket 3.0-rc1 ssl: 0.6 May 17 05:42:25 c2s8a57410 debug Received[c2s]: <iq id='mLgIs-3703' type='get' to='XXX@gabber-germany.de'> May 17 05:42:25 stanzarouter debug Routing to remote... May 17 05:42:25 mod_s2s debug opening a new outgoing connection for this stanza May 17 05:42:25 mod_s2s debug stanza [iq] queued until connection complete May 17 05:42:25 mod_s2s debug First attempt to connect to gabber-germany.de, starting with SRV lookup... May 17 05:42:25 adns debug Records for _xmpp-server._tcp.gabber-germany.de. not in cache, sending query (thread: 0x1e4ff540)... May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 mod_s2s debug gabber-germany.de has no SRV records, falling back to A/AAAA May 17 05:42:25 adns debug Records for gabber-germany.de not in cache, sending query (thread: 0x16c3a2d0)... May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 mod_s2s debug Error in DNS lookup: Too many open files May 17 05:42:25 adns debug Records for gabber-germany.de not in cache, sending query (thread: 0x1de81300)... May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 mod_s2s debug Error in DNS lookup: Too many open files May 17 05:42:25 mod_s2s debug DNS lookup failed to get a response for gabber-germany.de May 17 05:42:25 s2sout156c8720 info Failed in all attempts to connect to gabber-germany.de May 17 05:42:25 mod_s2s debug No other records to try for gabber-germany.de - destroying May 17 05:42:25 s2sout156c8720 debug Destroying outgoing session XXX->gabber-germany.de: DNS resolution failed: Too many open files [...] May 17 07:10:06 mod_s2s debug opening a new outgoing connection for this stanza May 17 07:10:06 mod_s2s debug stanza [db:verify] queued until connection complete May 17 07:10:06 mod_s2s debug First attempt to connect to tlabs.de, starting with SRV lookup... May 17 07:10:06 adns debug Records for _xmpp-server._tcp.tlabs.de. not in cache, sending query (thread: 0x6f39b30)... May 17 07:10:06 adns error Error sending DNS query: Too many open files May 17 07:10:06 mod_s2s debug tlabs.de has no SRV records, falling back to A/AAAA May 17 07:10:06 adns debug Records for tlabs.de not in cache, sending query (thread: 0xf19aaf0)... May 17 07:10:06 adns error Error sending DNS query: Too many open files May 17 07:10:06 mod_s2s debug Error in DNS lookup: Too many open files May 17 07:10:06 adns debug Records for tlabs.de not in cache, sending query (thread: 0x37855430)... May 17 07:10:06 adns error Error sending DNS query: Too many open files May 17 07:10:06 mod_s2s debug Error in DNS lookup: Too many open files May 17 07:10:06 mod_s2s debug DNS lookup failed to get a response for tlabs.de May 17 07:10:06 s2sout3bdf25e0 info Failed in all attempts to connect to tlabs.de May 17 07:10:06 mod_s2s debug No other records to try for tlabs.de - destroying May 17 07:10:06 s2sout3bdf25e0 debug Destroying outgoing session XXX->tlabs.de: DNS resolution failed: Too many open files

  6. s on

    Error still present: May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:47 adns error Error sending DNS query: Too many open files [...] May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files May 17 07:11:12 adns error Error sending DNS query: Too many open files use_libevent = true # prosodyctl about Prosody 0.10.0 # Prosody directories Data directory: /var/lib/prosody Config directory: /etc/prosody Source directory: /usr/lib64/prosody Plugin directories: /opt/prosody-modules - prosody-modules rev: 0fb95dc11bc8 /usr/lib64/prosody/modules/ # Lua environment Lua version: Lua 5.1 Lua module search paths: /usr/lib64/prosody/?.lua /usr/share/lua/5.1/?.lua /usr/share/lua/5.1/?/init.lua /usr/lib64/lua/5.1/?.lua /usr/lib64/lua/5.1/?/init.lua /root/.luarocks/share/lua/5.1/?.lua /root/.luarocks/share/lua/5.1/?/init.lua Lua C module search paths: /usr/lib64/prosody/?.so /usr/lib64/lua/5.1/?.so /usr/lib64/lua/5.1/loadall.so /root/.luarocks/lib64/lua/5.1/?.so LuaRocks: Installed (2.3.0) # Lua module versions lfs: LuaFileSystem 1.6.2 libevent: 2.0.21-stable luaevent: 0.4.4 lxp: LuaExpat 1.3.0 socket: LuaSocket 3.0-rc1 ssl: 0.6 May 17 05:42:25 c2s8a57410 debug Received[c2s]: <iq id='mLgIs-3703' type='get' to='XXX@gabber-germany.de'> May 17 05:42:25 stanzarouter debug Routing to remote... May 17 05:42:25 mod_s2s debug opening a new outgoing connection for this stanza May 17 05:42:25 mod_s2s debug stanza [iq] queued until connection complete May 17 05:42:25 mod_s2s debug First attempt to connect to gabber-germany.de, starting with SRV lookup... May 17 05:42:25 adns debug Records for _xmpp-server._tcp.gabber-germany.de. not in cache, sending query (thread: 0x1e4ff540)... May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 mod_s2s debug gabber-germany.de has no SRV records, falling back to A/AAAA May 17 05:42:25 adns debug Records for gabber-germany.de not in cache, sending query (thread: 0x16c3a2d0)... May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 mod_s2s debug Error in DNS lookup: Too many open files May 17 05:42:25 adns debug Records for gabber-germany.de not in cache, sending query (thread: 0x1de81300)... May 17 05:42:25 adns error Error sending DNS query: Too many open files May 17 05:42:25 mod_s2s debug Error in DNS lookup: Too many open files May 17 05:42:25 mod_s2s debug DNS lookup failed to get a response for gabber-germany.de May 17 05:42:25 s2sout156c8720 info Failed in all attempts to connect to gabber-germany.de May 17 05:42:25 mod_s2s debug No other records to try for gabber-germany.de - destroying May 17 05:42:25 s2sout156c8720 debug Destroying outgoing session XXX->gabber-germany.de: DNS resolution failed: Too many open files [...] May 17 07:10:06 mod_s2s debug opening a new outgoing connection for this stanza May 17 07:10:06 mod_s2s debug stanza [db:verify] queued until connection complete May 17 07:10:06 mod_s2s debug First attempt to connect to tlabs.de, starting with SRV lookup... May 17 07:10:06 adns debug Records for _xmpp-server._tcp.tlabs.de. not in cache, sending query (thread: 0x6f39b30)... May 17 07:10:06 adns error Error sending DNS query: Too many open files May 17 07:10:06 mod_s2s debug tlabs.de has no SRV records, falling back to A/AAAA May 17 07:10:06 adns debug Records for tlabs.de not in cache, sending query (thread: 0xf19aaf0)... May 17 07:10:06 adns error Error sending DNS query: Too many open files May 17 07:10:06 mod_s2s debug Error in DNS lookup: Too many open files May 17 07:10:06 adns debug Records for tlabs.de not in cache, sending query (thread: 0x37855430)... May 17 07:10:06 adns error Error sending DNS query: Too many open files May 17 07:10:06 mod_s2s debug Error in DNS lookup: Too many open files May 17 07:10:06 mod_s2s debug DNS lookup failed to get a response for tlabs.de May 17 07:10:06 s2sout3bdf25e0 info Failed in all attempts to connect to tlabs.de May 17 07:10:06 mod_s2s debug No other records to try for tlabs.de - destroying May 17 07:10:06 s2sout3bdf25e0 debug Destroying outgoing session XXX->tlabs.de: DNS resolution failed: Too many open files

New comment