#361 Message stanza body element getting extra xmlns attribute when passing trough BOSH
Reporter
marko.martinovic00
Owner
Waqas
Created
Updated
Stars
★★ (3)
Tags
Priority-Medium
Type-Defect
Status-Accepted
marko.martinovic00
on
*What steps will reproduce the problem?*
1. Enable and setup BOSH module
2. My environment is MUC and anonymous authentication if that matters
3. Send following message stanza trough BOSH:
<message type="chat" from="93a2bcd1-3d0c-4796-b8e4-2e8770c216a2@community.loc/a9fee2b7-1314-4148-80dc-b140d57fa5ba" to="support@conference.operators.community.loc/Operator 1"><body>Test</body></message>
*What is the expected output? What do you see instead?*
Expected stanza received by client should be:
<message type='chat' to='operator1@operators.community.loc/Gajim' from='support@conference.operators.community.loc/127.0.0.1'>
<body>Test</body>
</message>
Instead client receives following stanza:
<message type='chat' to='operator1@operators.community.loc/Gajim' from='support@conference.operators.community.loc/127.0.0.1'>
<body xmlns='http://jabber.org/protocol/httpbind'>Test</body>
</message>
Please note the extra xmlns='http://jabber.org/protocol/httpbind' on message body element. This behaviour isn't standards compliant and causes most clients to reject message.
*What version of the product are you using? On what operating system?*
This was tested on prosody 0.8.2 and 0.9
*Please provide any additional information below.*
I ain't good with Lua but I've isolated the problem to modules/mod_bosh.lua stream_callbacks.handlestanza() and worked around it. For example here's function in question for 0.8.2 version:
function stream_callbacks.handlestanza(request, stanza)
if request.ignore then return; end
log("debug", "BOSH stanza received: %s\n", stanza:top_tag());
local session = sessions[request.sid];
if session then
if stanza.attr.xmlns == xmlns_bosh then
stanza.attr.xmlns = nil;
end
core_process_stanza(session, stanza);
end
end
It appears that:
if stanza.attr.xmlns == xmlns_bosh then
stanza.attr.xmlns = nil;
end
code somehow moves the xmlns attribute to message body instead of removing it. This was tested by logging full stanza into log before and after this code block.
My workaround is to remove the extra xmlns attribute using:
local body = stanza:get_child("body", xmlns_bosh);
if body then
body.attr.xmlns = nil;
end
right bellow the first trouble making if. Patch for 0.8.2 version is attached to this error report. Thank you for your attention.
This is a bug in the client. Prosody has a workaround for this bug which apparently isn't working correctly.
The bug is this: The BOSH top level body element has xmlns='http://jabber.org/protocol/httpbind'. Any stanzas inside the body should have xmlns='jabber:client'. Some BOSH clients are buggy and don't use the jabber:client namespaces on stanzas, instead using the BOSH namespace. The code removing the namespace is trying to work around that, but only does so for the stanza element itself and not its children.
Your fix is incomplete. It does the job for the body element, but not for any other children of messages (e.g., subject element), or for other types of stanzas (e.g., status element in presence). We'll figure out how to get this properly fixed and released, thanks for the detailed report!
Changes
tags Status-Accepted
owner Waqas
marko.martinovic00
on
Thanks for such quick response, this all makes sense. Allow me to give a few additional observations, mostly because I'm curious about internals of prosody and Lua as programming language and it's libraries.
What's strange to me is that xmlns appended to message elements isn't jabber:client as you've said, instead http://jabber.org/protocol/httpbind is used. Also it appears that line
stanza.attr.xmlns = nil;
triggers the behaviour of adding xmlns to message body, but to my Lua-untrained eye it looks like it should just remove xmlns attribute from stanza. I've checked this by adding logging code like this:
log("debug", "Stanza BEFORE: %s\n", stanza:__tostring());
if stanza.attr.xmlns == xmlns_bosh then
stanza.attr.xmlns = nil;
end
log("debug", "Stanza AFTER: %s\n", stanza:__tostring());
This produces following inside prosody.log:
Sep 14 20:54:21 mod_bosh debug Stanza BEFORE: <message xmlns='http://jabber.org/protocol/httpbind' type='chat' to='support@conference.operators.community.loc/Operator 1' from='0df7a8a5-1f4c-4f45-9a74-c53e4caeee36@community.loc/eb0afd23-4711-4928-aa40-a5a3e8ce2ce3'><body>Test</body></message>
Sep 14 20:54:21 mod_bosh debug Stanza AFTER: <message type='chat' to='support@conference.operators.community.loc/Operator 1' from='0df7a8a5-1f4c-4f45-9a74-c53e4caeee36@community.loc/eb0afd23-4711-4928-aa40-a5a3e8ce2ce3'><body xmlns='http://jabber.org/protocol/httpbind'>Test</body></message>
Also to point out that Gajim is the only client I found that receives message with http://jabber.org/protocol/httpbind namespaced body.
Thanks again and best regards,
Marko
Waqas
on
A rough explanation of what's going on, just to satisfy your curiosity:
Every element has a namespace. Elements which are without an xmlns are just inheriting their parent's, but you can assume that they too have the namespace. In XMPP, the standard namespace is 'jabber:client'. Now, when XMPP was first being defined, 'jabber:client' was used on client-to-server streams, but 'jabber:server' was used on server-to-server, and 'http://jabber.org/protocol/httpbind' was used for BOSH. This was a design mistake, because now it means servers have to translate namespaces when reading from BOSH and forwarding to a client, etc. It can't be fixed for compatibility reasons.
So, when reading from any type of stream, Prosody normalizes namespaces to nil, and when writing, the elements inherit whatever is the content namespace of the stream. So any jabber:server elements we receive from a server, turn in jabber:client when sending to clients. In BOSH's case, we apparently did the normalization wrong in that it was only being done for top level elements, so child elements still had the BOSH namespace. Note that in BOSH's case this is a client bug, because the BOSH spec actually tells clients to put xmlns='jabber:client' on the elements, but many clients don't do this.
So, the fix is to just do for BOSH what we are doing for client and server streams already, i.e., convert elements (including child elements) with the BOSH namespace to have the nil (i.e., default=jabber:client) namespace.
The fix will go in the 0.9 branch by the way, which also has tons of other BOSH improvements.
marko.martinovic00
on
Thanks for detailed explanation and all efforts regarding Prosody.
falter
on
Hi there,
This bug was affecting my users on 0.9.2 (and 0.9.3). I put a patch together for 0.9.3 that nil's namespace for any of the stanza's children that are already set to 'http://jabber.org/protocol/httpbind', which I believe it fixes the problem. I believe this satisfies waqas20's request in comment #1.
Mike
Can we ask what clients are sending these broken stanzas? I'd at least like to know they will be fixed at some point, before we start bloating our code with workarounds for them... :)
falter
on
Here are the clients that I've found, so far. Swift is the only decent BOSH capable XMPP client that I've been able to find for OSX, thus far. Adium/pidgin/etc just don't work at all.
* Swift 2.0: http://swift.im/swift/ - Doesn't include jabber:client xmlns on <message..>.
POST /http-bind/ HTTP/1.1
Host: example.com:5280
Content-Type: text/xml; charset=utf-8
Content-Length: 244
<body rid='284684036641121' sid='30383fb1-47aa-402a-afa1-257a6b1215ad' xmlns='http://jabber.org/protocol/httpbind'><message id="8f4eb7d2-20e3-4643-aef8-14d74f4eff7e" to="mike@example.com" type="chat"><body>THIS IS A TEST</body></message></body>
* xmpp4r 0.5.6 - https://github.com/xmpp4r/xmpp4r - Doesn't include jabber:client xmlns on <message..>.
POST /http-bind/ HTTP/1.1
Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept: */*
User-Agent: Ruby
Content-Length: 191
Content-Type: text/xml; charset=utf-8
Host: example.com:5280
<body rid='1171' sid='641dec49-3e70-4996-9559-0199ce0725c6' xmlns='http://jabber.org/protocol/httpbind'><message to='mike@example.com' type='chat'><body>THIS IS A TEST</body></message></body>
*What steps will reproduce the problem?* 1. Enable and setup BOSH module 2. My environment is MUC and anonymous authentication if that matters 3. Send following message stanza trough BOSH: <message type="chat" from="93a2bcd1-3d0c-4796-b8e4-2e8770c216a2@community.loc/a9fee2b7-1314-4148-80dc-b140d57fa5ba" to="support@conference.operators.community.loc/Operator 1"><body>Test</body></message> *What is the expected output? What do you see instead?* Expected stanza received by client should be: <message type='chat' to='operator1@operators.community.loc/Gajim' from='support@conference.operators.community.loc/127.0.0.1'> <body>Test</body> </message> Instead client receives following stanza: <message type='chat' to='operator1@operators.community.loc/Gajim' from='support@conference.operators.community.loc/127.0.0.1'> <body xmlns='http://jabber.org/protocol/httpbind'>Test</body> </message> Please note the extra xmlns='http://jabber.org/protocol/httpbind' on message body element. This behaviour isn't standards compliant and causes most clients to reject message. *What version of the product are you using? On what operating system?* This was tested on prosody 0.8.2 and 0.9 *Please provide any additional information below.* I ain't good with Lua but I've isolated the problem to modules/mod_bosh.lua stream_callbacks.handlestanza() and worked around it. For example here's function in question for 0.8.2 version: function stream_callbacks.handlestanza(request, stanza) if request.ignore then return; end log("debug", "BOSH stanza received: %s\n", stanza:top_tag()); local session = sessions[request.sid]; if session then if stanza.attr.xmlns == xmlns_bosh then stanza.attr.xmlns = nil; end core_process_stanza(session, stanza); end end It appears that: if stanza.attr.xmlns == xmlns_bosh then stanza.attr.xmlns = nil; end code somehow moves the xmlns attribute to message body instead of removing it. This was tested by logging full stanza into log before and after this code block. My workaround is to remove the extra xmlns attribute using: local body = stanza:get_child("body", xmlns_bosh); if body then body.attr.xmlns = nil; end right bellow the first trouble making if. Patch for 0.8.2 version is attached to this error report. Thank you for your attention.
AttachmentsThis is a bug in the client. Prosody has a workaround for this bug which apparently isn't working correctly. The bug is this: The BOSH top level body element has xmlns='http://jabber.org/protocol/httpbind'. Any stanzas inside the body should have xmlns='jabber:client'. Some BOSH clients are buggy and don't use the jabber:client namespaces on stanzas, instead using the BOSH namespace. The code removing the namespace is trying to work around that, but only does so for the stanza element itself and not its children. Your fix is incomplete. It does the job for the body element, but not for any other children of messages (e.g., subject element), or for other types of stanzas (e.g., status element in presence). We'll figure out how to get this properly fixed and released, thanks for the detailed report!
ChangesThanks for such quick response, this all makes sense. Allow me to give a few additional observations, mostly because I'm curious about internals of prosody and Lua as programming language and it's libraries. What's strange to me is that xmlns appended to message elements isn't jabber:client as you've said, instead http://jabber.org/protocol/httpbind is used. Also it appears that line stanza.attr.xmlns = nil; triggers the behaviour of adding xmlns to message body, but to my Lua-untrained eye it looks like it should just remove xmlns attribute from stanza. I've checked this by adding logging code like this: log("debug", "Stanza BEFORE: %s\n", stanza:__tostring()); if stanza.attr.xmlns == xmlns_bosh then stanza.attr.xmlns = nil; end log("debug", "Stanza AFTER: %s\n", stanza:__tostring()); This produces following inside prosody.log: Sep 14 20:54:21 mod_bosh debug Stanza BEFORE: <message xmlns='http://jabber.org/protocol/httpbind' type='chat' to='support@conference.operators.community.loc/Operator 1' from='0df7a8a5-1f4c-4f45-9a74-c53e4caeee36@community.loc/eb0afd23-4711-4928-aa40-a5a3e8ce2ce3'><body>Test</body></message> Sep 14 20:54:21 mod_bosh debug Stanza AFTER: <message type='chat' to='support@conference.operators.community.loc/Operator 1' from='0df7a8a5-1f4c-4f45-9a74-c53e4caeee36@community.loc/eb0afd23-4711-4928-aa40-a5a3e8ce2ce3'><body xmlns='http://jabber.org/protocol/httpbind'>Test</body></message> Also to point out that Gajim is the only client I found that receives message with http://jabber.org/protocol/httpbind namespaced body. Thanks again and best regards, Marko
A rough explanation of what's going on, just to satisfy your curiosity: Every element has a namespace. Elements which are without an xmlns are just inheriting their parent's, but you can assume that they too have the namespace. In XMPP, the standard namespace is 'jabber:client'. Now, when XMPP was first being defined, 'jabber:client' was used on client-to-server streams, but 'jabber:server' was used on server-to-server, and 'http://jabber.org/protocol/httpbind' was used for BOSH. This was a design mistake, because now it means servers have to translate namespaces when reading from BOSH and forwarding to a client, etc. It can't be fixed for compatibility reasons. So, when reading from any type of stream, Prosody normalizes namespaces to nil, and when writing, the elements inherit whatever is the content namespace of the stream. So any jabber:server elements we receive from a server, turn in jabber:client when sending to clients. In BOSH's case, we apparently did the normalization wrong in that it was only being done for top level elements, so child elements still had the BOSH namespace. Note that in BOSH's case this is a client bug, because the BOSH spec actually tells clients to put xmlns='jabber:client' on the elements, but many clients don't do this. So, the fix is to just do for BOSH what we are doing for client and server streams already, i.e., convert elements (including child elements) with the BOSH namespace to have the nil (i.e., default=jabber:client) namespace. The fix will go in the 0.9 branch by the way, which also has tons of other BOSH improvements.
Thanks for detailed explanation and all efforts regarding Prosody.
Hi there, This bug was affecting my users on 0.9.2 (and 0.9.3). I put a patch together for 0.9.3 that nil's namespace for any of the stanza's children that are already set to 'http://jabber.org/protocol/httpbind', which I believe it fixes the problem. I believe this satisfies waqas20's request in comment #1. Mike
AttachmentsCan we ask what clients are sending these broken stanzas? I'd at least like to know they will be fixed at some point, before we start bloating our code with workarounds for them... :)
Here are the clients that I've found, so far. Swift is the only decent BOSH capable XMPP client that I've been able to find for OSX, thus far. Adium/pidgin/etc just don't work at all. * Swift 2.0: http://swift.im/swift/ - Doesn't include jabber:client xmlns on <message..>. POST /http-bind/ HTTP/1.1 Host: example.com:5280 Content-Type: text/xml; charset=utf-8 Content-Length: 244 <body rid='284684036641121' sid='30383fb1-47aa-402a-afa1-257a6b1215ad' xmlns='http://jabber.org/protocol/httpbind'><message id="8f4eb7d2-20e3-4643-aef8-14d74f4eff7e" to="mike@example.com" type="chat"><body>THIS IS A TEST</body></message></body> * xmpp4r 0.5.6 - https://github.com/xmpp4r/xmpp4r - Doesn't include jabber:client xmlns on <message..>. POST /http-bind/ HTTP/1.1 Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3 Accept: */* User-Agent: Ruby Content-Length: 191 Content-Type: text/xml; charset=utf-8 Host: example.com:5280 <body rid='1171' sid='641dec49-3e70-4996-9559-0199ce0725c6' xmlns='http://jabber.org/protocol/httpbind'><message to='mike@example.com' type='chat'><body>THIS IS A TEST</body></message></body>