#1778 iPhone push notification not working due to bug in mod_cloud_notify_encrypted on line 132: 'final' method getting light userdata rather than string
Reporter
Chris
Owner
MattJ
Created
Updated
Stars
★ (1)
Tags
Status-Fixed
Type-Defect
Priority-Medium
Chris
on
What steps will reproduce the problem?
1. The receiver's XMPP account is registered on a Prosody server.
2. The server of the receiving XMPP account has the cloud_notify_extensions module installed and active (which includes the mod_cloud_notify_encrypted and some other modules needed for iPhone push notification).
3. The receiving client is installed on iPhone (e.g. Siskin app).
4. Send a message to the XMPP account on the iPhone.
5. Check prosody.log and you will see the error log entry below.
What is the expected output?
If a message is sent to the iPhone, a notification should show up on the iPhone about the new message, and no error log entry in prosody.log on the server.
What do you see instead?
No notification on receiver's iPhone. Only until opening Siskin the message (and notifications) get retrieved.
Error log entry in prosody.log:
error Traceback[c2s]: /var/lib/prosody/custom_plugins/share/lua/5.4/mod_cloud_notify_encrypted.lua:132: bad argument #2 to 'final' (string expected, got light userdata)
stack traceback:
[C]: in method 'final'
/var/lib/prosody/custom_plugins/share/lua/5.4/mod_cloud_notify_encrypted.lua:132: in field '?'
/lib64/prosody/util/events.lua:81: in function </lib64/prosody/util/events.lua:77>
(...tail calls...)
/var/lib/prosody/custom_plugins/share/lua/5.4/mod_cloud_notify.lua:409: in upvalue 'handle_notify_request'
/var/lib/prosody/custom_plugins/share/lua/5.4/mod_cloud_notify.lua:440: in field '?'
/lib64/prosody/util/events.lua:81: in function </lib64/prosody/util/events.lua:77>
(...tail calls...)
/lib64/prosody/modules/mod_message.lua:60: in function </lib64/prosody/modules/mod_message.lua:18>
(...tail calls...)
/lib64/prosody/util/events.lua:81: in function </lib64/prosody/util/events.lua:77>
(...tail calls...)
/lib64/prosody/core/stanza_router.lua:188: in upvalue 'core_post_stanza'
/lib64/prosody/core/stanza_router.lua:128: in upvalue 'core_process_stanza'
/lib64/prosody/modules/mod_c2s.lua:332: in upvalue 'func'
/lib64/prosody/util/async.lua:144: in function </lib64/prosody/util/async.lua:142>
What version of the product are you using? On what operating system?
Product: Prosody 0.12 (hg:5d4957c8a972)
Operating System: Fedora 37, Linux 5.19.15-301.fc37.x86_64
Lua version: Lua 5.4
LuaExpat: 1.3.0
LuaFileSystem: 1.8.0
LuaSec: 1.2.0
LuaSocket: 3.0.0
luaunbound: 1.0.0
# library versions
libcrypto: OpenSSL 3.0.5 5 Jul 2022
libunbound: 1.16.3
Please provide any additional information below.
The root of this issue might not be in the mod_cloud_notify_encrypted module, but it renders the notification on iPhones unusable. From my knowledge, it should be technically possible to work around at least, to make iPhone notifications function again.
Chris
on
As additional information:
I tested that if the receiver's XMPP server is eJabberd, the push notification on the receiver's iPhone works as expected, having the rest of the setup the same.
Setup that does work:
sender's server -> eJabbered (receiver server) -> iPhone/Siskin (receiver client)
Setup that doesn't work (error on Prosody server's side in mod_cloud_notify_encrypted):
sender's server -> Prosody (receiver server) -> iPhone/Siskin (receiver client)
Chris
on
PARTLY SOLVED - Can the exception on server side be caused by the client?
After the receiving client switched from iPhone/Siskin to iPhone/Monal, the Prosody server doesn't produce an error anymore, and Push Notification works perfectly on the client end.
This is when the account on Siskin was deactivated, and then activated in Monal:
Oct 27 11:12:24 info Push notifications disabled (push.tigase.im[...])
Oct 27 11:12:25 info Client disconnected: unexpected eof while reading
Oct 27 11:12:42 info Client connected
Oct 27 11:12:44 info Stream encrypted (TLSv1.2 with ECDHE-RSA-AES256-GCM-SHA384)
Oct 27 11:12:44 info Authenticated as [...]
Oct 27 11:12:48 info Push notifications enabled for [...] (eu.prod.push.monal-im.org[...])
While I'm relieved to have something working now for iOS, it's not understandable to me how one client can cause an exception on the server, and another doesn't. If it's a wrong implementation on the client side (Siskin), then why would an exception show up on the server, - I would expect the server to handle mal-formed data in a way that doesn't escalate in an exception, as that makes me think of a bug on the server side. Or is throwing an exception on wrong client behavior the expected way and I'm wrong in my point of view here?
MattJ
on
Hi Chris,
Many thanks for your report, and sorry for the delay responding.
I've looked into your issue (thanks for providing all the details). It appears to be a bug in a dependency called 'luaossl' when used with Lua 5.4. They already fixed it in this commit earlier this year: https://github.com/wahern/luaossl/commit/1054d12f23ecd177881b1bcb6ade92f247cc01e9 and the fix was released in luaossl version 20220711: https://github.com/wahern/luaossl/releases/tag/rel-20220711
As for your question about clients triggering exceptions: we always consider it a bug if a client is able to trigger an exception in Prosody. And yes, clients all behave differently - e.g. using different features, or using them in different ways, that may expose bugs that other clients did not experience.
In this case the bug is in one of our dependencies, which is only used when encrypting push notifications. Only Siskin and Snikket iOS use that feature, Monal does not use that feature which is why it does not trigger the bug.
If you wish to use Siskin on your server, upgrading luaossl to the latest version should fix the problem for you.
I'll close the issue now, but if you have any further questions or comments, you are welcome to continue commenting.
Changes
tags Status-Fixed
owner MattJ
Chris
on
Thank you! luaossl version 20220711 is in Debian bookworm which is in testing right now. So many servers running Debian stable are having the same problem with Prosody + Siskin currently. Same counts for Fedora 37.
Thanks to your detailed reply I could also help out someone else with the exact same problem.
MattJ
on
Yeah, this is just one of multiple issues that affect Lua 5.4 packages in Debian 11/stable, which is why we don't recommend Lua 5.4 on that version.
We've been gradually fixing everything so that Debian 12 should be a well-supported base for Lua 5.4 applications.
Glad this information was helpful!
What steps will reproduce the problem? 1. The receiver's XMPP account is registered on a Prosody server. 2. The server of the receiving XMPP account has the cloud_notify_extensions module installed and active (which includes the mod_cloud_notify_encrypted and some other modules needed for iPhone push notification). 3. The receiving client is installed on iPhone (e.g. Siskin app). 4. Send a message to the XMPP account on the iPhone. 5. Check prosody.log and you will see the error log entry below. What is the expected output? If a message is sent to the iPhone, a notification should show up on the iPhone about the new message, and no error log entry in prosody.log on the server. What do you see instead? No notification on receiver's iPhone. Only until opening Siskin the message (and notifications) get retrieved. Error log entry in prosody.log: error Traceback[c2s]: /var/lib/prosody/custom_plugins/share/lua/5.4/mod_cloud_notify_encrypted.lua:132: bad argument #2 to 'final' (string expected, got light userdata) stack traceback: [C]: in method 'final' /var/lib/prosody/custom_plugins/share/lua/5.4/mod_cloud_notify_encrypted.lua:132: in field '?' /lib64/prosody/util/events.lua:81: in function </lib64/prosody/util/events.lua:77> (...tail calls...) /var/lib/prosody/custom_plugins/share/lua/5.4/mod_cloud_notify.lua:409: in upvalue 'handle_notify_request' /var/lib/prosody/custom_plugins/share/lua/5.4/mod_cloud_notify.lua:440: in field '?' /lib64/prosody/util/events.lua:81: in function </lib64/prosody/util/events.lua:77> (...tail calls...) /lib64/prosody/modules/mod_message.lua:60: in function </lib64/prosody/modules/mod_message.lua:18> (...tail calls...) /lib64/prosody/util/events.lua:81: in function </lib64/prosody/util/events.lua:77> (...tail calls...) /lib64/prosody/core/stanza_router.lua:188: in upvalue 'core_post_stanza' /lib64/prosody/core/stanza_router.lua:128: in upvalue 'core_process_stanza' /lib64/prosody/modules/mod_c2s.lua:332: in upvalue 'func' /lib64/prosody/util/async.lua:144: in function </lib64/prosody/util/async.lua:142> What version of the product are you using? On what operating system? Product: Prosody 0.12 (hg:5d4957c8a972) Operating System: Fedora 37, Linux 5.19.15-301.fc37.x86_64 Lua version: Lua 5.4 LuaExpat: 1.3.0 LuaFileSystem: 1.8.0 LuaSec: 1.2.0 LuaSocket: 3.0.0 luaunbound: 1.0.0 # library versions libcrypto: OpenSSL 3.0.5 5 Jul 2022 libunbound: 1.16.3 Please provide any additional information below. The root of this issue might not be in the mod_cloud_notify_encrypted module, but it renders the notification on iPhones unusable. From my knowledge, it should be technically possible to work around at least, to make iPhone notifications function again.
As additional information: I tested that if the receiver's XMPP server is eJabberd, the push notification on the receiver's iPhone works as expected, having the rest of the setup the same. Setup that does work: sender's server -> eJabbered (receiver server) -> iPhone/Siskin (receiver client) Setup that doesn't work (error on Prosody server's side in mod_cloud_notify_encrypted): sender's server -> Prosody (receiver server) -> iPhone/Siskin (receiver client)
PARTLY SOLVED - Can the exception on server side be caused by the client? After the receiving client switched from iPhone/Siskin to iPhone/Monal, the Prosody server doesn't produce an error anymore, and Push Notification works perfectly on the client end. This is when the account on Siskin was deactivated, and then activated in Monal: Oct 27 11:12:24 info Push notifications disabled (push.tigase.im[...]) Oct 27 11:12:25 info Client disconnected: unexpected eof while reading Oct 27 11:12:42 info Client connected Oct 27 11:12:44 info Stream encrypted (TLSv1.2 with ECDHE-RSA-AES256-GCM-SHA384) Oct 27 11:12:44 info Authenticated as [...] Oct 27 11:12:48 info Push notifications enabled for [...] (eu.prod.push.monal-im.org[...]) While I'm relieved to have something working now for iOS, it's not understandable to me how one client can cause an exception on the server, and another doesn't. If it's a wrong implementation on the client side (Siskin), then why would an exception show up on the server, - I would expect the server to handle mal-formed data in a way that doesn't escalate in an exception, as that makes me think of a bug on the server side. Or is throwing an exception on wrong client behavior the expected way and I'm wrong in my point of view here?
Hi Chris, Many thanks for your report, and sorry for the delay responding. I've looked into your issue (thanks for providing all the details). It appears to be a bug in a dependency called 'luaossl' when used with Lua 5.4. They already fixed it in this commit earlier this year: https://github.com/wahern/luaossl/commit/1054d12f23ecd177881b1bcb6ade92f247cc01e9 and the fix was released in luaossl version 20220711: https://github.com/wahern/luaossl/releases/tag/rel-20220711 As for your question about clients triggering exceptions: we always consider it a bug if a client is able to trigger an exception in Prosody. And yes, clients all behave differently - e.g. using different features, or using them in different ways, that may expose bugs that other clients did not experience. In this case the bug is in one of our dependencies, which is only used when encrypting push notifications. Only Siskin and Snikket iOS use that feature, Monal does not use that feature which is why it does not trigger the bug. If you wish to use Siskin on your server, upgrading luaossl to the latest version should fix the problem for you. I'll close the issue now, but if you have any further questions or comments, you are welcome to continue commenting.
ChangesThank you! luaossl version 20220711 is in Debian bookworm which is in testing right now. So many servers running Debian stable are having the same problem with Prosody + Siskin currently. Same counts for Fedora 37. Thanks to your detailed reply I could also help out someone else with the exact same problem.
Yeah, this is just one of multiple issues that affect Lua 5.4 packages in Debian 11/stable, which is why we don't recommend Lua 5.4 on that version. We've been gradually fixing everything so that Debian 12 should be a well-supported base for Lua 5.4 applications. Glad this information was helpful!