Skip to main content ↑ ↓
EMail Replies to the EA forum Started by Steeley · · Read 51444 times 0 Members and 1 Guest are viewing this topic. previous topic - next topic

Re: EMail Replies to the EA forum

Reply #15 –

Well as I did a code:beers::takeout_box:review, I did an experiment that looks to fix this issue (and the previous), and best of all it removes some code :D

Going way back,  improvements were added to properly (cough) work with Multipart/Related messages.  However some old conditions were left in place that had previously been used to, somewhat, work around/circumvent them. 

So when I come around in the AM I'll do a bit more testing, but right now it kind of looks good, at least with @Steeley email's

Re: EMail Replies to the EA forum

Reply #16 –

Hopefully it opened up the acceptance criteria rather than narrowed it to Pegasus 4.80!

Little story: We had a bunch of old Sundstrand low-speed ARINC 429 data loggers that one of our engineers hacked so they would also work on the high-speed 429 buses. Worked great until a few years later when we built the first HF Data Link Radios,. We updated the HF's on a 28 day cycle with new HF Ground Stations as they came on line, and/or changed frequencies. Only they couldn't update in the aircraft, only on the bench for some reason. I found the reason.. the 429 bus had a programming error and was running at "Half-High-speed", in between low-speed and high speed. Our modified data-loggers didn't care, they worked with anything in between 10khz and 100khz. Those that were "compliant with the standards" (low-speed 12-14.5Khz, high speed 100Khz +/- 1%) however, wouldn't work when the bus is running at 50Khz.

I guess as long as the parser will work with  anything coming at it (good bad or ugly), it's all good (and low maintenance to boot). 

Can you shoot the code changes to me that I can plug into the appropriate 1.1.6 php files?

(BTW I just discovered something else with 1.16 that I'll bounce off you a little later - probably via PM.)
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #17 –

Quote from: Steeley – Hopefully it opened up the acceptance criteria rather than narrowed it to Pegasus 4.80!
It should have! ... Found a couple of other bugs but getting "out there" in terms of what the PBE function would expect to get in a reply.   The parser handles a lot of different cases but not all (by design)  TBH I wrote most of that 10 years ago so its showing its patina ;)
Quote from: Steeley – Can you shoot the code changes to me that I can plug into the appropriate 1.1.6 php files?
Yup ... I'll get this all in 1.1.9 and the specific files (I think there are two) should be drop in replacements.

And as a test I released (3) of your reply's back (the ones that the parser used to fail)
 

Re: EMail Replies to the EA forum

Reply #19 –

Well alrighty then.  I went up on the board and deleted those three emails to "clean things up".

I'll be looking for the fixes, and hopefully Google doesn't decide in the meantime to implement some new "security/spam/pfishing" Gmail format protocol and try to drive it to the rest of the known universe.

If this posts, you'll know to check your PM for that "other issue" I mentioned.

-Steeley 

Did you exchange, a walk-on part in the war for a lead role in a cage?

[~ Pink Floyd: "Wish You Were Here"]

--
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #20 –

Quote from: ahrasis –
Quote from: Spuds – I'll get this all in 1.1.9
I like.
Would be an interesting opportunity. In fact, it is easier for me to swap my test forum from 1.1.8 to 1.1.9 instead of swapping to the 2.0 code base. In the latter case, a missing database table is complained about.
However, until now git fetch origin does not result in updates...

Re: EMail Replies to the EA forum

Reply #21 –

I've been working on, somewhat sporadically, the 1.1.x => 2.0 update script to take care of any DB changes and updates.  Without that it will not "like" a 1.1.x db.

I've been doing more work on the parser and found some more edge cases that I fixed, all were mainly all around malformed email,  headers or body text but there were some ways to deal with those with out to much drama.  All the testing was on 2.0,  so up next it to test those changes on 1.1.x and make sure they work as expected there.

Once that is done, I'll post the updates for folks to test out.

Re: EMail Replies to the EA forum

Reply #22 –

Well hells bells...  I went to post a question on the Pegasus Community board about the email formatting with multipart turned off, when I noticed the IERenderer part was updated this past week.

(IERenderer is the Pegasus Mail HTML renderer)

So I downloaded and installed it, and guess what...

NOW emails post to v. 1.1.6 just fine, either way..(multipart on or off...)

And instead of this with multipart deselected (copied from an earlier post):

X-mailer: Pegasus Mail for Windows (4.80.1028)
Content-type: Multipart/Related; boundary="Message-Boundary-12625"
X-PMFLAGS: 570949760 0 1 P68TF1SD.CNM                     

--Message-Boundary-12625
Content-type: text/html; charset=ISO-8859-1
Content-transfer-encoding: Quoted-printable
Content-description: Mail message body

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
          "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html  xmlns=3D"http://www.w3.org/1999/xhtml" xml:lang=3D"en" lang=3D"en">=
<head>
<title></title>
<meta http-equiv=3D"content-type" content=3D"text/html;charset=3Dutf-8"/>
<meta http-equiv=3D"Content-Style-Type" content=3D"text/css"/>
</head>
<body>

{snip}

</body>
</html>

--Message-Boundary-12625--

Instead, now I get this:

Priority: normal
X-mailer: Pegasus Mail for Windows (4.80.1028)
Content-type: text/html; charset=US-ASCII
Content-transfer-encoding: 7BIT
Content-description: Mail message body
X-PMFLAGS: 34079360 0 1 P6A14D28.CNM                       

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
          "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html  xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title></title>
<meta http-equiv="content-type" content="text/html;charset=utf-8"/>
<meta http-equiv="Content-Style-Type" content="text/css"/>
</head>
<body>

{snip}

</body>
</html>

No boundary markers at all,  just "html"  (settings have "rich text" enabled).

Now THAT looks sane, and it posts just fine using the existing parser in 1.1.6.

It appears the previous version(s) of IR hosed it all up. Pegasus itself hasn't been changed

(Of course, there's no mention of this in the latest version IR release notes, natch..)

In the course of posting that in the Pegasus community, I also asked if there shouldn't be at least SOME plain text in there somewhere?
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #23 –

That is nice and simple :smiley: Interesting they make no mention of it in the changelog.

I've added that latest example to my testcase.mbox just so I have it.  Been trying to build a nice set of tests that stress the parser and then check its output  -> forum post.  Overall this has been a good exercise in finding a few odd bugs!

Re: EMail Replies to the EA forum

Reply #24 –

Maybe Michael was hoping nobody noticed. Why plead guilty before you've been charged? My post in the community at least amounts to someone saying "I see what you did there.." (For the record, I installed the previous version less than a month ago, and I have no idea what version the malformation began in. And I have little curiosity to start loading earlier versions to find out).

We did both question the RFC compliance of that earlier version of "non-multipart", but at least for me the malformation wasn't as clear until I saw the 'corrected' version.  In hindsight "Content-transfer-encoding: Quoted-printable" and multipart still being  presented with it "off" should have been a clue, but I've not spent much time looking at email headers before, and, perhaps more pertinent, I'm probably getting too old for this stuff.

More and more I can see the progression from "I know what's wrong" to  "Something's not right but I don't know why". and I'm looking forward to the day when I get to "Something's wrong? I didn't notice." Then there will be people to feed and take care of me and I can just watch TV..  ;D  
Last Edit: August 07, 2022, 12:19:11 pm by Steeley
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #25 –

And just a note - the multipart enabled formatting did not change with the IER version update as the multipart disabled format did.

To be honest, I don't know why that multipart enable/disable switch even exists - my guess it's to allow compatibility with various mail-servers, at least during network "standards transitions", but I can't imagine that 95% of their userbase (and that includes me) would understand the need to "customize" their outbound mail for various server implementations, never mind knowing to switch multipart on/off depending on who they're sending to.  Pegasus also has a switch for adding "Attachment information" to multipart messages which I had to disable sometime back - I don't recall the problem it created now, but so far it appears that not having it enabled causes no grief anywhere. Indeed, programming for email is a dark and lonely place to grope around in.
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #26 –

multipart should mean that there is both a html and text version or that an additional attachment is sent along with the text.

However mail clients don’t tend to follow this and ignore the rfc. I tend to strictly adhere to the RFC and just drop anything outside of that, which can cause issues with some clients. However surprisingly gmail, outlook and the other larger ones tend to conform pretty much.

Re: EMail Replies to the EA forum

Reply #27 –

Quote from: Steeley – If it's any consolation, Spuds, if I turn off multipart in Pegasus, it doesn't post in my forum either..  (but it seems to recognize that I tried emailing a reply to the same message that failed earlier and thusly won't post a "properly formatted" reply attempt that follows).

I decided to dig into this a bit further, and dug out the emails in the cache that failed to post  on my server..

(In my best Maxwell Smart voice: ) Would you believe.....

...the reason it didn't post in my forum is because I don't use the same "identity" to log into it, as I use to log in here. So I was merrily switching "identities" in Pegasus as I sent emails to each forum, and "got out of sync". You got at least two emails from an unregistered email account and I got one.
The failure to post had nothing to do with the email format on 1.1.6 at all..  just a slipped nut on my keyboard.  :P

Pegasus and IER, and the parser on 1.1.6 are (and were) fine.

(Note- @Spuds - that should have nothing to do with the quoting thing I PM'd you about, however..)
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #28 –

Quote from: tino – multipart should mean that there is both a html and text version or that an additional attachment is sent along with the text.

That was both Spuds and my assumption also... but Pegasus for one won't include plain text in "some" versions of multipart, if you have "rich text" (which is their plain text/html switch) selected. Other aspects of multipart, it will include both.  No idea what, if any of it, is RFC compliant.  But if it works, well, I haven't heard the FBI is investigating RFC standards violations ... yet anyway.

Sometimes I wonder if there is email traffic out their being sent around so intentionally "malformed" they get rejected by every standards-compliant or semi-compliant parser except the clandestine servers they're intended for, so hardly anyone knows about them. An Email "Dark Web" so to speak.  Nah, I'm sure that's just "crazy talk"..  :(
Last Edit: August 09, 2022, 02:02:53 am by Steeley
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #29 –

Quote from: Spuds –
I've added that latest example to my testcase.mbox just so I have it.  Been trying to build a nice set of tests that stress the parser and then check its output  -> forum post.  Overall this has been a good exercise in finding a few odd bugs!

@Spuds - reply to this so I get an email notification, and I'll throw another email reply at you with yet different formatting to stress-test the parser.. Pegasus has an "Attachment Information" switch that, if enabled, will give you this..

Code: [Select]

X-mailer: Pegasus Mail for Windows (4.80.1028)
X-PMFLAGS: 570949760 0 1 PIMHW5EA.CNM                       


--Message-Boundary-24840
Content-type: Multipart/Alternative; boundary="Alt-Boundary-9241.139177750"

--Alt-Boundary-9241.139177750
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Content-description: Mail message body

this is a test with an attachment

--Alt-Boundary-9241.139177750
Content-type: text/html; charset=US-ASCII
Content-transfer-encoding: 7BIT
Content-description: Mail message body

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
          "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html  xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"><head>
<title></title>
<meta http-equiv="content-type" content="text/html;charset=utf-8"/>
<meta http-equiv="Content-Style-Type" content="text/css"/>
</head>
<body>

<snipped html>

</body>
</html>

--Alt-Boundary-9241.139177750--

--Message-Boundary-24840
Content-type: text/plain; charset=US-ASCII
Content-disposition: inline
Content-description: Attachment information.

The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

   ---- File information -----------
     File:  butts.jpg
     Date:  22 Feb 2009, 9:32
     Size:  33794 bytes.
     Type:  JPEG-image

--Message-Boundary-24840
Content-type: Image/JPEG; name="butts.jpg"
Content-disposition: attachment; filename="butts.jpg"
Content-transfer-encoding: BASE64

/9j/4AAQSkZJRgABAQEBLAEsAAD/2wBDAA0JCgsKCA0LCgsODg0PEyAVExISEyccHhcgLikx


<snip>


If I recall correctly, two years ago the parser would interpret it as two separate attachments 

and the second part is stored as raw (unencoded) alphanumeric, which of course gives an
"unrecognized format" response.

In your reply to this, if you ask me nicely to not to blow up the parser, I won't do an email reply to it with an attachment and "Attachment Information" enabled ..  :)

// Deep inside every dilemma lies a solution that involves explosives //