As of writing, Prosody supports LibIDN (which only supports IDNA 2003) and LibICU for handling IDNA domain support. Is there any chance for experimental LibIDN2 support (and thus for IDNA 2008)?
Be aware that LibIDN2 doesn't support a way from ACE to Unicode, just from Unicode to ACE - however the idna_to_unicode() functionality doesn't seem to be used in the core of Prosody anyway (this is why I would declare this experimental). The core of Prosody only uses idna_to_ascii() as far as I can see.
IDNA 2003 vs. IDNA 2008 make differences once it comes to e.g. straße.de: IDNA 2003 makes it strasse.de, IDNA 2008 makes it xn--strae-oqa.de.
This has gotten even easier, I think: Libidn2 >= 2.0 supports now both, ACE to Unicode and the other way round. Thus existing libidn support could be switched to libidn2 (or support both, if desired), same API just different includes or so.
It appears to be possible to make ICU use IDNA 2008 now, but it is unclear how this interacts with stringprep.
Zash
on
Trunk nightlies have been build with ICU for a while now and it seems mostly fine.
I also just figured out what flags to give ICU to get proper IDNA 2008, as it apparently has some transitional IDNA 2003 compat mode as default.
So how about this plan:
Switch to ICU as default
Switch to IDNA 2008 with transitional compat enabled
Test for a period
Disable compat
Relevant discussion about lack of stringprep/precis in libidn2:
https://gitlab.com/libidn/libidn2/-/issues/28
Still have not heard about any problems with ICU from users of trunk nightlies,
so maybe it's time to make it the official default for 0.12?
As of writing, Prosody supports LibIDN (which only supports IDNA 2003) and LibICU for handling IDNA domain support. Is there any chance for experimental LibIDN2 support (and thus for IDNA 2008)? Be aware that LibIDN2 doesn't support a way from ACE to Unicode, just from Unicode to ACE - however the idna_to_unicode() functionality doesn't seem to be used in the core of Prosody anyway (this is why I would declare this experimental). The core of Prosody only uses idna_to_ascii() as far as I can see. IDNA 2003 vs. IDNA 2008 make differences once it comes to e.g. straße.de: IDNA 2003 makes it strasse.de, IDNA 2008 makes it xn--strae-oqa.de.
Yes, we should upgrade to follow https://tools.ietf.org/html/rfc7622 which says to use IDNA 2008
ChangesThis has gotten even easier, I think: Libidn2 >= 2.0 supports now both, ACE to Unicode and the other way round. Thus existing libidn support could be switched to libidn2 (or support both, if desired), same API just different includes or so.
Prosody needs normalization functions for XMPP addresses. Previously this was the STRINGPREP framework. The newer RFC7622 uses the PRECIS framework instead. libidn2 supports neither of these, so we can't use it: https://libidn.gitlab.io/libidn2/manual/libidn2.html#Stringprep-and-libidn2 libicu doesn't either at the time of this writing, but has a feature request here: https://unicode-org.atlassian.net/browse/ICU-11981
ChangesIt appears to be possible to make ICU use IDNA 2008 now, but it is unclear how this interacts with stringprep.
Trunk nightlies have been build with ICU for a while now and it seems mostly fine. I also just figured out what flags to give ICU to get proper IDNA 2008, as it apparently has some transitional IDNA 2003 compat mode as default. So how about this plan: Switch to ICU as default Switch to IDNA 2008 with transitional compat enabled Test for a period Disable compat
ChangesICU binding changed to IDNA 2008 (compat mode) in https://hg.prosody.im/trunk/rev/010c67532ed0
ChangesRelevant discussion about lack of stringprep/precis in libidn2: https://gitlab.com/libidn/libidn2/-/issues/28 Still have not heard about any problems with ICU from users of trunk nightlies, so maybe it's time to make it the official default for 0.12?