Skip to main content ↑ ↓
EMail Replies to the EA forum Started by Steeley · · Read 40144 times 0 Members and 1 Guest are viewing this topic. previous topic - next topic

EMail Replies to the EA forum

In the topic "how to "Subscribe to boards and get the full text of the posts on that board"?", in addition to posting a reply directly to the thread,  https://www.elkarte.net/community/index.php?topic=6150.msg43946#msg43946 , I also sent in a (different) reply via email, to the same message as the board-posted reply..

As I start this topic some 45 minutes later it has not appeared, either gone off to bit heaven, or stuck in moderation.

" I shot an arrow into the air, where it's gone I know not where. "  Would someone like to take a look around and see if EA is actually working as intended?
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #1 –

Of course, a few problems.

First was simply that the site uses Gmail / IMap, but was using the regular credentials.  Looks like the allow unsecure app access was removed at some point so instead had to change to an app password for allowing this site to access the email via IMap.  Of course that also required turning on f2a for the Gmail account and:yawning_face:you get the drift.

So once that was done it did pull in two emails:tada:

One from you that the system complained had no message body (I'm checking why, probably some filter) and one from @rjm which had no key in the message, so of course that is never going to post.

Re: EMail Replies to the EA forum

Reply #2 –

Quote from: Spuds – Of course, a few problems.

First was simply that the site uses Gmail / IMap, but was using the regular credentials.  Looks like the allow unsecure app access was removed at some point so instead had to change to an app password for allowing this site to access the email via IMap.  Of course that also required turning on f2a for the Gmail account and:yawning_face:you get the drift.

So once that was done it did pull in two emails:tada:

One from you that the system complained had no message body (I'm checking why, probably some filter) and one from @rjm which had no key in the message, so of course that is never going to post.

I replied via email about 20 minutes ago and....  my guess is you're still unclogging that filter.. as that one didn't show up either..

Or maybe gmail detected my despise of it... ::)
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #3 –

Think I have this traced down ... now I just need to be sure about the fix, I have something on my local but need to do some more testing.

The problem is caused by Pegasus Mail for Windows (4.80.1028)  presenting the message as Multipart/Related  and the way our parser deals with that.

QuoteThe multipart/related content type is used for compound documents (that is, messages in which the separate body parts are intended to work together to provide the full meaning of the message).

In this case there is only 1 part of that Multipart/Related, and its just a standard Multipart/Alternative.  Multipart/Alternative is also multiple sections but they contain the same message but encoded in different ways, like plain, html, base64, such that the client can present the best one.

Anyway since there is only one section in that Multipart/Related, the parser fails to pull it together so to speak.  Email is truly a dark and lonely place, probably filled with Cobol programmers !

Re: EMail Replies to the EA forum

Reply #4 –

Quote from: Spuds – Think I have this traced down ... now I just need to be sure about the fix, I have something on my local but need to do some more testing.

The problem is caused by Pegasus Mail for Windows (4.80.1028)  presenting the message as Multipart/Related  and the way our parser deals with that.

QuoteThe multipart/related content type is used for compound documents (that is, messages in which the separate body parts are intended to work together to provide the full meaning of the message).

In this case there is only 1 part of that Multipart/Related, and its just a standard Multipart/Alternative.  Multipart/Alternative is also multiple sections but they contain the same message but encoded in different ways, like plain, html, base64, such that the client can present the best one.

Anyway since there is only one section in that Multipart/Related, the parser fails to pull it together so to speak.  Email is truly a dark and lonely place, probably filled with Cobol programmers !


My guess is that the parser is looking for text after the Message Boundary before encountering the Alt-Boundary, and not seeing it, thinking the message is "empty"..

That is...

Content-type: Multipart/Related; boundary="Message-Boundary-13115"
X-PMFLAGS: 570949760 0 1 PRRPJEKN.CNM                      

--Message-Boundary-13115
Content-type: Multipart/Alternative; boundary="Alt-Boundary-9315.904040406"

Nothing here

--Alt-Boundary-9315.904040406 -(Start of Alt message 1)
Content-type: text/plain; charset=US-ASCII
Content-transfer-encoding: 7BIT
Content-description: Mail message body

Text here

--Alt-Boundary-9315.904040406 - (end of Alt message 1, start of Alt message 2)
Content-type: text/html; charset=US-ASCII
Content-transfer-encoding: 7BIT
Content-description: Mail message body

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
          "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

html here

--Alt-Boundary-9315.904040406--(end of alt message 2)

--Message-Boundary-13115-- (End of message)

I think that's RFC1521 compliant, but I could be wrong, and/or my opinion is irrelevant.

I can turn off multipart formatting I suppose, it was good enough back in the 70's.  :D 

FWIW, it works fine with 1.16 on my own server.. (did I mention my distaste for gmail? Why yes, yes I did..)

::)
Last Edit: August 04, 2022, 02:21:38 pm by Steeley
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #5 –

Well alrighty then, I'll reply to this post in email and see if it works now..

BTW, did I ever mention that I totally despise Google Mail? In fact, gmail address accounts are not permitted on my forum - I see it as the equivalent of having Google Inc. as a lurking member of the forum.

On a "public forum" it's not as big of a concern, I suppose.

OK, so I draw back the bow and yell "incomming"..

-Steeley

Did you exchange, a walk-on part in the war for a lead role in a cage? [~ Pink Floyd: "Wish You Were Here"]

--
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #6 –

Should be working now, I ran the (2) failed emails again so they should post.

Yes the email is compliant AFAIK, it was just that our parser did not deal with it properly.

Normally when you get a Multipart/Related you will find at least two sections in that boundary.  The first is (normally) a typical Multipart/Alternative section with its own boundaries that include a plain text and html section.  Then, for example, there would be another part of the Multipart/Related such as an image that would used in the first part, and then another section, maybe with another image or whatever.

Anyway the parser was not dealing with only one section, so I put in a fix that should "just work" ... and I'm still looking to see if that was the best idea. 

TLDR:bug:

Re: EMail Replies to the EA forum

Reply #7 –

I see the earlier emailed reply successfully posted after your tweaks.

Just for the record, I have had bounces occur from some email servers if I just send a graphic without any text, which suggests that "not properly recognizing" that Multipart/Related; means combine the parts, not just "pick one" to use, is not uncommon. Tis indeed a dark art.
Last Edit: August 04, 2022, 02:43:30 pm by Steeley
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #8 –

Well, I replied to this via email earlier - so far that one hasn't shown up.. (nor an even earlier reply to rjm's post.)

Thought I'd try it with multipart turned off, and discovered it was already deselected, so I turned it back on and will reply via email again and see if it works this time.

(looks like I might need a bigger quiver before this gets worked out..) .

-Steeley

Did you exchange, a walk-on part in the war for a lead role in a cage? [~ Pink Floyd: "Wish You Were Here"]

--
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #9 –

Quote from: Steeley – Well, I replied to this via email earlier - so far that one hasn't shown up.. (nor an even earlier reply to rjm's post.)

Thought I'd try it with multipart turned off, and discovered it was already deselected, so I turned it back on and will reply via email again and see if it works this time.

(looks like I might need a bigger quiver before this gets worked out..) .

-Steeley



Did you exchange, a walk-on part in the war for a lead role in a cage? [~ Pink Floyd: "Wish You Were Here"]

--

OK, that worked, yet another inbound to see if  I was responsible for the other two I referred to not posting..
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #10 –

OK, Spuds, I've probably junked up your email queue with test emails from an unregistered EA account address, and duplicate email replies to the same post, none of which should post, and they didn't, of course. (Too many things going on simultooneously on my end).
 
The reason for the madness was to try to do an email reply with the multipart turned off and confirm it would still post. Once I solved my rectal/cranial inversion issues I set an email to a post I had never replied to with multipart off and...

Alas, that doesn't appear to post either.  However, I'm not sure what is sent in that situation is RFC compliant.

Here's the pertinent parts...

In-reply-to: <bc6514a0d15122eb072ce12f5c21b918-m43964@elkarte.net>
References: <6151@elkarte.net>, <bc6514a0d15122eb072ce12f5c21b918-m43964@elkarte.net>
X-mailer: Pegasus Mail for Windows (4.80.1028)
Content-type: Multipart/Related; boundary="Message-Boundary-12625"
X-PMFLAGS: 570949760 0 1 P68TF1SD.CNM                      

--Message-Boundary-12625
Content-type: text/html; charset=ISO-8859-1
Content-transfer-encoding: Quoted-printable
Content-description: Mail message body

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
          "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html  xmlns=3D"http://www.w3.org/1999/xhtml" xml:lang=3D"en" lang=3D"en">=
<head>
<title></title>
<meta http-equiv=3D"content-type" content=3D"text/html;charset=3Dutf-8"/>
<meta http-equiv=3D"Content-Style-Type" content=3D"text/css"/>
</head>
<body>

{snip}

</body>
</html>

--Message-Boundary-12625--

If this concerns you I'll work with you further, otherwise I'll just turn multipart back on and we can call it good since that appears to work now..

// Deep inside every dilemma lies a solution that involves explosives //
 

Re: EMail Replies to the EA forum

Reply #11 –

I'll pull those emails in and see what is happening ....

There are (5) pending in the queue, (2) are easy (email not in DB and expired key) which translates to the key is valid but was not sent to you, and you already used that key, they are like one time passwords, but the admin can override on a fail.

The other (3) I need to pull in as they fail with no message body, which is the parser error ... fun!

Re: EMail Replies to the EA forum

Reply #12 –

Quote from: Spuds – I'll pull those emails in and see what is happening ....

There are (5) pending in the queue, (2) are easy (email not in DB and expired key) which translates to the key is valid but was not sent to you, and you already used that key, they are like one time passwords, but the admin can override on a fail.

The other (3) I need to pull in as they fail with no message body, which is the parser error ... fun!

At least two of those 5 are from a non-member addy (I have multiple "Identities" in Pegasus, and was switching back and forth multi-erroring), and at least two are email replies to previously email replied-to posts so those should fail anyway).

Only one (latest send date/time) should be a pure "empty" message body failure, from my EA account addy - and again, I'm not positive that's an RFC compliant format.
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #13 –

If it's any consolation, Spuds, if I turn off multipart in Pegasus, it doesn't post in my forum either..  (but it seems to recognize that I tried emailing a reply to the same message that failed earlier and thusly won't post a "properly formatted" reply attempt that follows).
// Deep inside every dilemma lies a solution that involves explosives //

Re: EMail Replies to the EA forum

Reply #14 –

Took a quick look and its again a parser failure, kind of.

The empty messages, as far as the parser was concerned, were shown as Multipart/Related and in those boundaries are a single HTML message, not Multipart/Alternative, just a single HTML message. 

TBH I'm not sure that is compliant, I thought all emails must contain a text/plain section, but we are talking about email here LOL

Now I had put in a catch for messages missing a plain/text section but containing a text/html section.  The issue is that does not run when that is in the Multipart/Related area only when in the Multipart/Alternative section.    Not sure there is a quick work around for that with the current structure of the parser,  may have to refactor that area.