Not sure, if there is something wrong:
I was sending a PM by using the preview a few times, and as I believe each of the previews was sent as a PM.
I could not send the final PM, because the limit of 4 PMs per hour was reached.
yep, I was the one who got those PMs.. definitely a bug.. Also: german umlauts are cluttered, not sure if you copied your text from another software to ElkArte?!?
Yes...there were some quotes from my "broken" SMF-Forum in the PM, TE. :-[ Sorry.
(Die Umlaute sind da doch teilweise kaputt, die fehlerhafte Anzeige kann auch daher kommen)
No problem, just wanted to make sure this is not another ElkArte related bug.. those broken characters are ISO-8859-1 coded umlauts (and repairable)..
/me whistles innocently
/me would like to send more than 4 PMs a hour
TE, maybe we can talk about the further things about the migration here?
http://support.ElkArte-hilfe.de/index.php/topic,167.0.html
@TE I did not copy the begin of my PM, just the quotes.
So I think, that ä, ö, ü and ß will get broken in PM, if you use the preview first.
Edit: And if I use the preview in a reply in a topic I will see them broken too:
2. Edit:
My account-settings say, that my favorite language is english ;) it has been german before. So maybe the umlaut-mistake is just because all profile settings are now in default?
I cannot change my profile, I have forgotten my password.
Bug can happen if you click preview more than once..
for reference: öäüß
result after preview: öäüÃ
Looks like the strings (body and subject) are not properly encoded before being sent to the server before the preview.
I didn't know it was set... :-[
But in the meantime I caught a problem with the site, apparently it won't let me change settings in the admin panel, even though locally works fine.
There are few other settings to tweak, but at the moment I can't.
I fixed the preview in fcf89327935fb0031e291b6c20e8585bf0cdf81a
The problem with the special characters seems to be the result of the "usual" mixed status the AJAX calls are all around the code.
In these AJAX calls (PM previews and posts previews, plus few other places), the texts are converted to 8bit strings before being sent to the server. In 1.0, that string was somehow processed and returned properly to the client, now it is not and the characters result broken.
If I understand correctly the problem, I would be tempted to drop entirely the usage of php_to8bit from the codebase, but TBH it's something I'm a little worried about...
What do you think? Scrap during beta and if something is broken let's find a solution, or let's find a solution to the preview before facing the "real" issue?
Would you have to update all the xml to json ? I don't know if xml is going to handle all of utf8 stuff so I thought that was why the utf8_encode (on the js side via php_to8bit was done).
Anyway for a 1.1 I don't think we should be redoing that, IMO All of that was fine in 1.0 ... I do remember having to update things for some of the 4 byte utf8 items.
Well I least I said it may be breaking :P
That update to add encodeURIComponent should have taken care of everything .. hummm
I guess we can put it live here and see if it breaks anything. O:-)
Done, hard-refresh is mandatory, though I changed the CACHE_STALE to try force it.
ETA: looks good for the moment. :D
Still getting several PMs with the preview button but the charset problem seems to be fixed :)
me may have forgot to upload the code. O:-)
Uh, XML is like
the (initial) UTF-8 stronghold, lol. I reckon it's far more likely that some old version of PHP had issues with it. ;)
Edit: I just got nothing (white page) trying to submit this from preview. Could've been that a new reply had been posted while I was typing this.
Then I look forward to your PR where all that encoding is removed ;)
I thought
@emanuele did that already. :P
Nope ... I had updated the code in 1.1 to use some built in "php to 8 bit" JS functions since we did not have to concern ourselves with older browser support.
A couple of those calls were missed causing double encoding. However if we feel that no encoding is required for 1/2/3/4 byte utf8 then cool, but that was not a step I took in my original PR.
It supposedlly shouldn't be for PHP 5.recent, but I don't know enough about strings and characters in 5.older to say. It's just not XML that has the issues. ;)
Edit: posting from mobile and finally saw the loading... during quick edit!
Next time I'll be sure to read all the specifications before I ask a question O:-) XML is the one.
My original idea (the one that I pushed) was to remove all the calls to php_to8bit from the js, and now I feel it was the way to go since from what I understood, Spuds added this to the urlencode function, and it seems to work! :)
I wouldn't touch for now anything else, just to avoid break something in some odd manner, though in 2.0 it may even be beneficial to consolidate all the AJAX calls passing through jQuery, so that we have to care only about how the jQuery developers decide to change the methods to call. LOL
I assume jQuery uses fetch with a fallback?
Note to self: jQuery.get() doesn't use fetch, probably nor should it.
@Spuds I don't really see any places where php_urlencode should be dropped? I mean, it's applied to URI components after all. And on the PHP side of things all I see is the correct filtering of a couple of disallowed characters:
function cleanXml($string)
{
// http://www.w3.org/TR/2000/REC-xml-20001006#NT-Char
return preg_replace('~[\x00-\x08\x0B\x0C\x0E-\x19\x{FFFE}\x{FFFF}]~u', '', $string);
}
Long story short, I set out to perform the cleaning you suggested and all I found was the php_urlencode() function itself.