I've changed this list to munge the From: field which allows the LUV server to DKIM sign the messages. This means that all mail going through this list will pass DKIM checks and therefore list members can set stricter anti-spam policies without losing LUV mail. I will do it to the luv-main list in the near future. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Sun, Nov 08, 2015 at 06:00:51PM +1100, Russell Coker via luv-talk wrote:
I've changed this list to munge the From: field which allows the LUV server to DKIM sign the messages. This means that all mail going through this list will pass DKIM checks and therefore list members can set stricter anti-spam policies without losing LUV mail.
I will do it to the luv-main list in the near future.
Please don't. This makes it impossible to reply privately. It's even worse than Reply-To: munging. craig -- craig sanders <cas@taz.net.au>
On Wed, Nov 11, 2015 at 09:11:37AM +1100, Craig Sanders via luv-talk wrote:
On Sun, Nov 08, 2015 at 06:00:51PM +1100, Russell Coker via luv-talk wrote:
I've changed this list to munge the From: field which allows the LUV [...] I will do it to the luv-main list in the near future.
Please don't.
This makes it impossible to reply privately.
It's even worse than Reply-To: munging.
Much worse, because it looks like you're not only munging the From: header, you're also munging the Reply-To: header (probably as a Q&D hack to "repair" the damage you've done by munging From:) how much spam, if any, have you ever seen that forged a luv-* list address in the From: header? You're breaking standards and useful functionality (private replies) in order to solve a non-existent problem. craig -- craig sanders <cas@taz.net.au>
On Wed, 11 Nov 2015 09:37:01 AM Craig Sanders via luv-talk wrote:
On Wed, Nov 11, 2015 at 09:11:37AM +1100, Craig Sanders via luv-talk wrote:
On Sun, Nov 08, 2015 at 06:00:51PM +1100, Russell Coker via luv-talk wrote:
I've changed this list to munge the From: field which allows the LUV [...] I will do it to the luv-main list in the near future.
Please don't.
This makes it impossible to reply privately.
It's even worse than Reply-To: munging.
Much worse, because it looks like you're not only munging the From: header, you're also munging the Reply-To: header (probably as a Q&D hack to "repair" the damage you've done by munging From:)
In Kmail "Reply" goes to the list, "Reply to all" goes to list and sender, "reply to authir" goes to the sender, and "reply to mailing list" goes to the list. In K9 "reply" goes to the sender and "reply to all" goes to the list as well. It seems to work in a reasonable way.
how much spam, if any, have you ever seen that forged a luv-* list address in the From: header? You're breaking standards and useful functionality (private replies) in order to solve a non-existent problem.
That is not the problem we are trying to solve. The issue is that more of the big sending domains are using DMARC entries (yahoo is one) and more of the big receiving domains are rejecting mail based on them (gmail is one). -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Wed, Nov 11, 2015 at 03:10:06PM +1100, Russell Coker wrote:
In Kmail "Reply" goes to the list, "Reply to all" goes to list and sender, "reply to authir" goes to the sender, and "reply to mailing list" goes to the list. In K9 "reply" goes to the sender and "reply to all" goes to the list as well. It seems to work in a reasonable way.
well, then, it's a good thing that absolutely everybody in the world uses either Kmail or K9. And no-one ever needs to specify their own preferred (or only viable/working) address in the Reply-To: header. Reply-To munging is broken, it destroys the sender's intent (and makes it *impossible* to contact them) when you override their Reply-To with yours. Mailman already uses the correct header Mail-Followup-To: (and Sender: for more primitive MUAs). Munging the From: header is similarly broken.
That is not the problem we are trying to solve. The issue is that more of the big sending domains are using DMARC entries (yahoo is one) and more of the big receiving domains are rejecting mail based on them (gmail is one).
there has to be another solution because the one you have chosen is broken. craig ps: surrendering to the arbitrary decrees and whims of giant corporations seems like a bad idea to me. -- craig sanders <cas@taz.net.au>
https://dmarc.org/wiki/FAQ "Mailing lists usually do not take authorship of the emails they relay. It means the From: header in the email will not contain the domain name of the mailing list, and if the mailing list add DKIM to all its emails, DKIM d= will not match. If the domain in the From: header is from an organization that publishes a DMARC record, the email is likely to not be delivered." I do not run a mailing list server at the moment so I did not investigate how to work around it.. Russell seems to know (and I guess I know how he does it;-) https://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/introd... "If a problem is not completely understood, it is probably best to provide no solution at all." This would have saved us from a few improvements over the last years.. DMARC seems to be one of them. Regards Peter On Wed, Nov 11, 2015 at 3:40 PM, Craig Sanders via luv-talk < luv-talk@luv.asn.au> wrote:
On Wed, Nov 11, 2015 at 03:10:06PM +1100, Russell Coker wrote:
In Kmail "Reply" goes to the list, "Reply to all" goes to list and sender, "reply to authir" goes to the sender, and "reply to mailing list" goes to the list. In K9 "reply" goes to the sender and "reply to all" goes to the list as well. It seems to work in a reasonable way.
well, then, it's a good thing that absolutely everybody in the world uses either Kmail or K9. And no-one ever needs to specify their own preferred (or only viable/working) address in the Reply-To: header.
Reply-To munging is broken, it destroys the sender's intent (and makes it *impossible* to contact them) when you override their Reply-To with yours. Mailman already uses the correct header Mail-Followup-To: (and Sender: for more primitive MUAs).
Munging the From: header is similarly broken.
That is not the problem we are trying to solve. The issue is that more of the big sending domains are using DMARC entries (yahoo is one) and more of the big receiving domains are rejecting mail based on them (gmail is one).
there has to be another solution because the one you have chosen is broken.
craig
ps: surrendering to the arbitrary decrees and whims of giant corporations seems like a bad idea to me.
-- craig sanders <cas@taz.net.au> _______________________________________________ luv-talk mailing list luv-talk@luv.asn.au http://lists.luv.asn.au/listinfo/luv-talk
Quoting Peter Ross via luv-talk (luv-talk@luv.asn.au):
"Mailing lists usually do not take authorship of the emails they relay. It means the From: header in the email will not contain the domain name of the mailing list, and if the mailing list add DKIM to all its emails, DKIM d= will not match. If the domain in the From: header is from an organization that publishes a DMARC record, the email is likely to not be delivered."
I do not run a mailing list server at the moment so I did not investigate how to work around it.. Russell seems to know (and I guess I know how he does it;-)
What you said. Craig, I am in sympathy with what you said, but, as far as I can see, Russell is _very_ arguably choosing the best of among various unpalatable options. You already saw my heads-up that Joe-jobs have just (IMO) taken a significant jump in intelligence, and I am predicting that successful delivery of forged-including-envelope mails to mailing lists, believably impersonating legitimate senders, is poised to start becoming a big problem. What stands in the way of that? Only two things: SPF and DMARC. (I notice that yahoo.com still uses DMARC's deprecated 2004 predecessor, DomainKeys.) All forms of mail-redirection have been caught in the crossfire, and are potential victims of Joe-job detection measures: /etc/alias redirections to off-system, ~/.forward files that redirect off-system, CGIs that send mail on behalf of users, mailing list manager, and many other casual methods of reflecting / redireting / impersonating other domains' mail that we became accustomed to, during the Internet's long period of pretending that we're all friends here. The SPF folks (Meng Wong of pobox.com, possibly others) proposed the Sender Rewriting Scheme (SRS) as a workaround for some of these redirectors -- but that scheme is quite awkward and there has been extremely little uptake, and mostly what's been happening is gradual deprecation of those redirection methods (the likes of /etc/alias and ~/.forward) at all. I'm really not surprised that mailing lists are finally at risk from DMARC implementation. Unlike the case with /etc/alias and ~/.forward, we don't want to just stop using them to send mail between systems. So mailing lists -- and Russ ss LUV listadmin -- are increasingly caught between a rock and a hard place. Personally, I think DMARC is overscoped. It _could_ have been limited to just validating the SMTP IP aka validating the 'From ' envelope sender -- but instead its designers went overboard and decided to extend the initial DomainKeys functionality and crypto-sign internal SMTP headers, too. This is one of several reasons I've leaned in the past towards SPF: It does only the essential, hence creates less collateral damage.[1] But both SPF and DMARC are increasingly relevant, and mailing lists that continue to flunk DMARC testing are going to have greater and greater delivery problems. I get why Russell's solution is unappetising. Believe me, I am with you on that. But I'm still waiting to hear anyone suggest a credible alternative. [1] SPF might have suffered a setback when Meng Wong announced in 2004 that he and Microsoft Corporation had agreed to merge SPF and Microsoft's vaguely similar 'Caller ID' scheme. This intended merger failed when, IIRC, it turned out that 'Caller ID' suffered from undisclosed patent problems. Subsequently, 'Caller ID' got deservedly ignored by the broader Internet community, and SPF as originally written has persisted but gets less press than the (IMO) hideously overengineered DMARC scheme.
I also meant to note that DMARC is a further overelaboration of the (IMO) already over-elaborate DKIM scheme, and the latter is actually what Russell has been talking about.
On Wed, Nov 11, 2015 at 04:17:49PM -0800, Rick Moen via luv-talk wrote:
Personally, I think DMARC is overscoped. It _could_ have been limited to just validating the SMTP IP aka validating the 'From ' envelope sender -- but instead its designers went overboard and decided to extend the initial DomainKeys functionality and crypto-sign internal SMTP headers, too. This is one of several reasons I've leaned in the past towards SPF: It does only the essential, hence creates less collateral damage.[1]
DMARC fundamentally broken if it uses the From: header instead of the envelope From address. With the sole exception of generating the envelope (including From_) and any missing required headers when a message is first sent (e.g. Message-Id, Date), from the POV of an MTA, headers are and should be nothing but irrelevant comments. And there is never **any** excuse for munging the Reply-To: header. Doing that is broken, and has been known to be broken for many years.
But both SPF and DMARC are increasingly relevant, and mailing lists that continue to flunk DMARC testing are going to have greater and greater delivery problems.
I guess i'm not telling you anything you don't already know, but SPF works because it doesn't care about headers. SPF checks compare the envelope From against the published SPF DNS record of the domain. if it comes from one of the servers allowed by the SPF record, then accept. otherwise reject (or accept and tag). That's all a mailing list needs. It re-sends the mail with its own envelope-from address (necessary for VERP as well as SPF), so the recipient server can check SPF to see if the sender is allowed to be sending from, e.g., luv.asn.au. The From: header inside the message is irrelevant, that's for end-user MUAs to handle. If the list admin is worried about subscriber forgery then the list server can check SPF on incoming messages. If list subscribers are worried about forgery of their addresses, then they (or their domain admins) can add SPF records to theri domains. SPF works, and it works better the more domains that use it. It was designed as a domain forgery prevention tool, and it works extremely well at doing just that...who knows better than the domain admins which servers are allowed to claim to be sending mail from that domain?
I get why Russell's solution is unappetising. Believe me, I am with you on that. But I'm still waiting to hear anyone suggest a credible alternative.
How about a domain-keys derivative that restricts itself to checking the envelope From address as it should? From: headers are comments, not addressing information. But that would probably be indistinguishable from SPF and therefore not give giant corporates like yahoo or google the control over all mail that they are working towards. craig ps: and this "Real Name via luv talk" appendage is ugly and wrong. I'll have to edit it out of every reply I send from now on - when I hit 'g' in mutt for a group-reply, I am *not* sending the message to "Rick Moen via luv-talk", I am sending the message to "luv-talk". If I want to reply to Rick Moen privately, I'll hit 'r' for private reply...except, I can't now because the list munges the From: header and my procmail has for many years partially undone the idiocy of Reply-To by renaming Reply-To headers to X-Old-Reply-To Speaking of mutt, this From: munging also breaks mutt's ability to detect list messages sent by me, and to highlight them by showing them in a different colour: color index cyan default '~P' # mail from myself -- craig sanders <cas@taz.net.au>
Quoting Craig Sanders via luv-talk (luv-talk@luv.asn.au):
DMARC fundamentally broken if it uses the From: header instead of the envelope From address.
What can I say? I dislike the scope of what DKIM / DMARC aspires to cover.
And there is never **any** excuse for munging the Reply-To: header. Doing that is broken, and has been known to be broken for many years.
{shrug} I've been singing in that choir long enough that I'm not sure even how long it's been since I joined it, just that my vocal cords are sore.
I guess i'm not telling you anything you don't already know, but SPF works because it doesn't care about headers.
What can I say? I like SPF exactly for its modest tailoring of scope. ;->
If the list admin is worried about subscriber forgery then the list server can check SPF on incoming messages.
If I had my way, everyone would do SPF and be at least a little leery of DKIM / DMARC / DomainKeys. But rejections based on DKIM / DMARC / DomainKeys are nonetheless likely to start being a real pragmatic problem, and unfortunately is likely to require pragmatic measures. I don't like those, either (so far), but must acknowledge the real-world problem.
How about a domain-keys derivative that restricts itself to checking the envelope From address as it should? From: headers are comments, not addressing information.
That'd be nice. We could call it Sort-of-like SPF. ;-> -- Cheers, "If you see a snake, just kill it. Rick Moen Don't appoint a committee on snakes." rick@linuxmafia.com -- H. Ross Perot McQ! (4x80)
On Wed, 11 Nov 2015, Rick Moen via luv-talk wrote:
If the list admin is worried about subscriber forgery then the list server can check SPF on incoming messages.
If I had my way, everyone would do SPF and be at least a little leery of DKIM / DMARC / DomainKeys. But rejections based on DKIM / DMARC / DomainKeys are nonetheless likely to start being a real pragmatic problem, and unfortunately is likely to require pragmatic measures.
Wouldn't the market solution to be to continue doing things from the technically correct point of view on infrastructure we maintain, and ignore companies that don't? yahoo senders, sending to a mailing list can't have their mail received by google recievers, and it's because yahoo are being silly regarding choice of standards? Easy. If the yahoo customers care about being listened to, then they'll change providers. -- Tim Connors
Quoting Tim Connors (tim.w.connors@gmail.com):
Wouldn't the market solution to be to continue doing things from the technically correct point of view on infrastructure we maintain, and ignore companies that don't?
yahoo senders, sending to a mailing list can't have their mail received by google recievers, and it's because yahoo are being silly regarding choice of standards? Easy. If the yahoo customers care about being listened to, then they'll change providers.
It's a cost-benefit calculation, I guess. Today Yahoo, tomorrow who else? Also, who else today, honestly? I'm honestly not sure. FWIW, I am _not_ (so far) doing the described sort of transformation on any of the Mailman mailing lists I administer for BALUG (San Francisco), SVLUG and CABAL (Silicon Valley), or others, and I've not yet heard complaints or seen any problems. Just a data point.
On Thu, Nov 12, 2015 at 04:51:22PM +1100, Tim Connors via luv-talk wrote:
Wouldn't the market solution to be to continue doing things from the technically correct point of view on infrastructure we maintain, and ignore companies that don't?
yahoo senders, sending to a mailing list can't have their mail received by google recievers, and it's because yahoo are being silly regarding choice of standards? Easy. If the yahoo customers care about being listened to, then they'll change providers.
and if yahoo cares about keeping their users, they'll change what they're doing. pandering to them gives them no reason to change. craig -- craig sanders <cas@taz.net.au>
On Thu, 2015-11-12 at 19:41 +1100, Craig Sanders wrote:
On Thu, Nov 12, 2015 at 04:51:22PM +1100, Tim Connors via luv-talk wrote:
Wouldn't the market solution to be to continue doing things from the technically correct point of view on infrastructure we maintain, and ignore companies that don't?
yahoo senders, sending to a mailing list can't have their mail received by google recievers, and it's because yahoo are being silly regarding choice of standards? Easy. If the yahoo customers care about being listened to, then they'll change providers.
and if yahoo cares about keeping their users, they'll change what they're doing.
Unfortunately, there are too many of the "great unwashed" who are uninformed, and just say that the others must be wrong. It is a social problem, that too much business merely want consumers, forgetting that it is their workers who are the customers, and most importantly, are citizens. I see too much similarities to the bread and circuses of the decline of the Roman Empire. Nick Hanauer had the right of it in predicting the pitchforks if the inequalities continue to grow. And they will with a population of uninformed and ignorant consumers who expect to be entertained and somewhat greedy. There are a number of components to solving the problem, technical and social. Recognising all of that is an essential to coping and finding a real solution rather than a mere stopgap that will also fail in time.
pandering to them gives them no reason to change.
craig
Regards, Mark Trickett
On Thu, 12 Nov 2015, Mark Trickett via luv-talk wrote:
On Thu, 2015-11-12 at 19:41 +1100, Craig Sanders wrote:
On Thu, Nov 12, 2015 at 04:51:22PM +1100, Tim Connors via luv-talk wrote:
Wouldn't the market solution to be to continue doing things from the technically correct point of view on infrastructure we maintain, and ignore companies that don't?
yahoo senders, sending to a mailing list can't have their mail received by google recievers, and it's because yahoo are being silly regarding choice of standards? Easy. If the yahoo customers care about being listened to, then they'll change providers.
and if yahoo cares about keeping their users, they'll change what they're doing.
Unfortunately, there are too many of the "great unwashed" who are uninformed, and just say that the others must be wrong. It is a social problem, that too much business merely want consumers, forgetting that it is their workers who are the customers, and most importantly, are citizens.
Not a problem for a linux mailing list. -- Tim Connors
On Thu, 12 Nov 2015 08:37:40 PM Mark Trickett via luv-talk wrote:
and if yahoo cares about keeping their users, they'll change what they're doing.
Unfortunately, there are too many of the "great unwashed" who are uninformed, and just say that the others must be wrong. It is a social problem, that too much business merely want consumers, forgetting that it is their workers who are the customers, and most importantly, are citizens.
Craig talks about Yahoo wanting to gain and keep users. LUV also wants to gain and keep users and making services like Yahoo just work with out lists is a part of doing that. I don't think that this is an ideal situation, but as Rick points out every option has downsides and this one seems to work well.
I see too much similarities to the bread and circuses of the decline of the Roman Empire. Nick Hanauer had the right of it in predicting the pitchforks if the inequalities continue to grow. And they will with a population of uninformed and ignorant consumers who expect to be entertained and somewhat greedy.
I don't think that an analogy to the deline of the Roman empire is relevant to mailing list policies. As for inequality causing the decline of empires, that's well known. Authors like William Gibson wrote about corporations with nation state powers. That makes for interesting stories but doesn't seem plausible. A key component of national armies is patriotism. Unless corporations can find people willing to die for them then they can't compete in that game. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Quoting Russell Coker via luv-talk (luv-talk@luv.asn.au):
I don't think that this is an ideal situation, but as Rick points out every option has downsides and this one seems to work well.
And I sincerely apologise in advance for arguing on both sides of the issue. ;-> But I did also seriously, on reflection, think it's worth pondering what actual examples of deliverability problems on account of DKIM or DMARC validity _actually_ exist. Maybe you have examples in hand. Over hear across the wide pond, I've not yet seen any, FWIW. Personally, I don't borrow trouble. OTOH (and there I go again, arguing both sides), I am slow to criticise fellow listadmins seeing things differently. -- Cheers, A host is a host, from coast to coast. Rick Moen And nobody talks to a host that's close, rick@linuxmafia.com Unless the host that isn't close is busy, hung, or dead. McQ! (4x80)
Hello Russell, On Thu, 2015-11-12 at 20:56 +1100, Russell Coker wrote:
On Thu, 12 Nov 2015 08:37:40 PM Mark Trickett via luv-talk wrote:
and if yahoo cares about keeping their users, they'll change what they're doing.
Unfortunately, there are too many of the "great unwashed" who are uninformed, and just say that the others must be wrong. It is a social problem, that too much business merely want consumers, forgetting that it is their workers who are the customers, and most importantly, are citizens.
Craig talks about Yahoo wanting to gain and keep users. LUV also wants to gain and keep users and making services like Yahoo just work with out lists is a part of doing that.
I don't think that this is an ideal situation, but as Rick points out every option has downsides and this one seems to work well.
I see too much similarities to the bread and circuses of the decline of the Roman Empire. Nick Hanauer had the right of it in predicting the pitchforks if the inequalities continue to grow. And they will with a population of uninformed and ignorant consumers who expect to be entertained and somewhat greedy.
I don't think that an analogy to the deline of the Roman empire is relevant to mailing list policies. As for inequality causing the decline of empires, that's well known. Authors like William Gibson wrote about corporations with nation state powers. That makes for interesting stories but doesn't seem plausible. A key component of national armies is patriotism. Unless corporations can find people willing to die for them then they can't compete in that game.
Not comparing the mailing list to the decline of the Roman Empire, more that the problems are much wider than the list, that our current "civilisation" is disintegrating, and the behaviours we are seeing are symptoms, and that to fix matters requires the technical fixes, and significant social changes. Ignoring the bigger picture is ignoring why any technical fix will fail on its own. As to the really large multinationals, they are co-opting, and corrupting, national governments. That will be detrimental all round. Your technical fix is a kludge, any fix will be a kludge with the current proliferation of incompatible "standards", I give you kudos for your efforts, and that they will help while making things more awkward. That is an inevitability. Regards, Mark Trickett
On Thu, Nov 12, 2015 at 08:56:57PM +1100, Russell Coker via luv-talk wrote:
Craig talks about Yahoo wanting to gain and keep users. LUV also wants to gain and keep users and making services like Yahoo just work with out lists is a part of doing that.
1. how many yahoo.com subscribers does LUV have? 2. is there any evidence that proves it is worth breaking standards in order to cater to yahoo subscribers?
I don't think that this is an ideal situation, but as Rick points out every option has downsides and this one seems to work well.
except that it doesn't work well, because it breaks both the From: header and the Reply-To: header. craig -- craig sanders <cas@taz.net.au>
On Fri, 13 Nov 2015, Craig Sanders via luv-talk wrote:
On Thu, Nov 12, 2015 at 08:56:57PM +1100, Russell Coker via luv-talk wrote:
Craig talks about Yahoo wanting to gain and keep users. LUV also wants to gain and keep users and making services like Yahoo just work with out lists is a part of doing that.
1. how many yahoo.com subscribers does LUV have?
2. is there any evidence that proves it is worth breaking standards in order to cater to yahoo subscribers?
I don't think that this is an ideal situation, but as Rick points out every option has downsides and this one seems to work well.
except that it doesn't work well, because it breaks both the From: header and the Reply-To: header.
Agreed with Craig on this one. And my LUV folder is now fugly with redundant department of redundancy style redundant information that everyone is from "via luv-talk". -- Tim Connors
On 2015-11-16 06:26, Tim Connors via luv-talk wrote:
[...] And my LUV folder is now fugly with redundant department of redundancy style redundant information that everyone is from "via luv-talk".
+1 Adding "via luv-list" is a mistake, regardless of the merits, or otherwise, of munging the headers. I understand it's a way of saying "look! I've screwed with the headers" but is not necessary. Anders
For me the addition adds clarity - easier to read and shows that something is happening - maybe on a journey - i.e. not necessarily final ... Mike On 16/11/15 18:02, Anders Holmström via luv-talk wrote:
On 2015-11-16 06:26, Tim Connors via luv-talk wrote:
[...] And my LUV folder is now fugly with redundant department of redundancy style redundant information that everyone is from "via luv-talk".
+1 Adding "via luv-list" is a mistake, regardless of the merits, or otherwise, of munging the headers. I understand it's a way of saying "look! I've screwed with the headers" but is not necessary.
Anders _______________________________________________ luv-talk mailing list luv-talk@luv.asn.au http://lists.luv.asn.au/listinfo/luv-talk
On Mon, 16 Nov 2015 06:02:09 PM Anders Holmström via luv-talk wrote:
On 2015-11-16 06:26, Tim Connors via luv-talk wrote:
[...] And my LUV folder is now fugly with redundant department of redundancy style redundant information that everyone is from "via luv-talk".
+1 Adding "via luv-list" is a mistake, regardless of the merits, or otherwise, of munging the headers. I understand it's a way of saying "look! I've screwed with the headers" but is not necessary.
From: "Anders Holmström via luv-talk" <luv-talk@luv.asn.au> From: "Anders Holmström" <luv-talk@luv.asn.au> So you're saying that of the above 2 options you prefer the second? Mailman doesn't appear to give that as an option, or if it does it's a site wide option not a per-list option. If I had been given a free choice between the 2 I would probably have chosen the second. But it might be that hacking the source is the only way of getting it. If there is a consense of opinion that the second option is the best way to do it then I will consider the option of hacking the source - but note that it's in a language that I don't use much and it may be beyond my skills in that area. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Tue, Nov 17, 2015 at 01:33:35AM +1100, Russell Coker via luv-talk wrote:
If there is a consense of opinion that the second option is the best way to do it then I will consider the option of hacking the source -
i think the general consensus is that nobody likes the From: or Reply-To: munging. craig -- craig sanders <cas@taz.net.au>
Quoting Craig Sanders via luv-talk (luv-talk@luv.asn.au):
i think the general consensus is that nobody likes the From: or Reply-To: munging.
FWIW, I do agree with Craig on this, Russell, though I applaud your benevolent intentions. -- Cheers, <blazemore> omg i love this song Rick Moen <blazemore> Now playing: Unknown Artist - Track 2 rick@linuxmafia.com @ 128 Kbps. (0:47/3:24) McQ! (4x80) <Javi> blazemore: Yeah, that's a bad-ass song.
Hello Russell, On Tue, 2015-11-17 at 01:33 +1100, Russell Coker wrote:
On Mon, 16 Nov 2015 06:02:09 PM Anders Holmström wrote:
On 2015-11-16 06:26, Tim Connors wrote:
[...] And my LUV folder is now fugly with redundant department of redundancy style redundant information that everyone is from "via luv-talk".
+1 Adding "via luv-list" is a mistake, regardless of the merits, or otherwise, of munging the headers. I understand it's a way of saying "look! I've screwed with the headers" but is not necessary.
From: "Anders Holmström via luv-talk" <luv-talk@luv.asn.au> From: "Anders Holmström" <luv-talk@luv.asn.au>
So you're saying that of the above 2 options you prefer the second? Mailman doesn't appear to give that as an option, or if it does it's a site wide option not a per-list option.
Thank you for your time and effort to sort out Mailman to the extent you have achieved.
If I had been given a free choice between the 2 I would probably have chosen the second. But it might be that hacking the source is the only way of getting it. If there is a consense of opinion that the second option is the best way to do it then I will consider the option of hacking the source - but note that it's in a language that I don't use much and it may be beyond my skills in that area.
It might well be that there is currently no satisfactory solution, short of working on the source code. I can comprehend why you have done what you have, and why it garners so much dissatisfaction. Were I capable of coding, I would cheerfully try to assist in coding a better solution, one that was standards compliant, and less prone to falling afoul of the standards that are having to be developed and implemented to cope with spam and other malicious emails. I would ask whether there are other ways of verifying that the incoming contributions are legitimate, not spoofed sender spam. I am not asking you to spend a lot of time and effort, but to all the contributors to this to try to make positive suggestions as to what might be possible, even if not currently implemented in the available software. My contribution is that it will require an element of social engineering. That is too big for LUV alone to tackle, but as global citizens, maybe it is one to which we can add a small push. Regards, Mark Trickett
Quoting Mark Trickett via luv-talk (luv-talk@luv.asn.au):
I would ask whether there are other ways of verifying that the incoming contributions are legitimate, not spoofed sender spam.
That is not the problem space. LUV's mailing list server is able to check DKIM/DMARC or SPF records to validate that the IP source matches published data in the incoming mail's claimed sending domain's DNS saying 'trust these IPs as authorised senders for this domain'. That works just fine. However, when the mail is relayed through Mailman and redirected out to subscribers' own SMTP hosts, those SMTP hosts will tend to fail the mail on the same tests that it passed on the initial transmission, because a new IP -- LUV's IP -- is now purporting to _also_ be an authorised sender IP for the sender's domain (even though LUV's IP _isn't_ listed in the sending domain's DKIM/DMARC or SPF DNS records). Russell's solution amounts to 'change the sender upon retransmission to the subscribers'. That solution runs headlong into those of us who consider having our correct 'From: ' information included in our SMTP headers to be a feature rather than a bug, e.g., it has even _worse_ effects than do the infamously wrong-headed idea of 'Reply-To munging' on the intended existence of reply-sender alongside reply-all response modes. Like Reply-To munging on steroids, the current solution makes reply-sender not work at all, and also conceals the sender's contact e-mail address entirely. Which is pretty serious collateral damage, which should IMO not be taken lightly. Making the retransmitted mail still pass DKIM/DMARC creates more damage and is even more of a problem than just making it pass SPF, because DKIM/DMARC tests the internal 'From: ' SMTP header while SPF does not. (Thus, Russell feels compelled to rewrite the 'From: ' header and substitute a -- pardon the expression -- forged sender of 'luv-talk@luv.asn.au' with GECOS field '[firstname] [lastname] via luv-talk'.)
I am not asking you to spend a lot of time and effort, but to all the contributors to this to try to make positive suggestions as to what might be possible, even if not currently implemented in the available software.
Unfortunately, if Russell wants traffic transiting through LUV's servers to always be able to continue to pass DKIM/DMARC even after retransmission, he'll need to rewrite the transmitted copies pretty much the way he's doing it. The trivial detail of the ' via luv-talk' suffix on the GECOS field can be fixed easily enough by just finding the *.py file that does that and editing it. However, that's cosmetic. E.g., it's slightly less bad if my SMTP headers get forged to say 'From: Rick Moen <luv-talk@luv.asn.au>' instead of 'From: Rick Moen via luv-talk <luv-talk@luv.asn.au>' -- but that GECOS suffix isn't the real problem numerous of us have raised. It's that it's _supposed_ to continue to say 'From: Rick Moen <rick@linuxmafia.com>', the way I sent it, without intermediate software interfering and purporting to change who sent the mail or from where. Ultimately, LUV either messes with user's mail contents in a fashion that conceals and falsifies the original 'From: ' information the sender specified, with additional damage of sabotaging reply-sender operations, or it doesn't. I appreciate what Russell is trying to do, and would not dream of impugning his good intentions, but would rather he not make LUV's server rewrite my (for one) mail in this way. The specific reason I question the necessity is that I'm not seeing all the other mailing lists being driven to this measure. Indeed, I'm not seeing that elsewhere at all, including on the numerous technical mailing lists *I* administer in the San Francisco Bay Area. Once again, I have the highest of respect for what Russell is trying to pull off. I just don't think the side-effects are justifiable. -- Cheers, <BombScare> i beat the internet Rick Moen <BombScare> the end guy is hard rick@linuxmafia.com -- seen on IRC McQ! (4x80)
On Tue, 17 Nov 2015 09:46:01 PM Rick Moen via luv-talk wrote:
However, when the mail is relayed through Mailman and redirected out to subscribers' own SMTP hosts, those SMTP hosts will tend to fail the mail on the same tests that it passed on the initial transmission, because a new IP -- LUV's IP -- is now purporting to _also_ be an authorised sender IP for the sender's domain (even though LUV's IP _isn't_ listed in the sending domain's DKIM/DMARC or SPF DNS records).
Russell's solution amounts to 'change the sender upon retransmission to the subscribers'.
List servers ARE changing the sender. Instead of receiving a message directly from the person you receive mail from the list server which has very different issues with anti-spam etc. This list server is also re-encoding the message entirely, for example my MUA doesn't base64 encode the message body but when my mail goes through the list server it is encoded in that way.
Ultimately, LUV either messes with user's mail contents in a fashion that conceals and falsifies the original 'From: ' information the sender specified, with additional damage of sabotaging reply-sender operations, or it doesn't.
I don't believe that it is in any way falsified. The recipient has no doubt about the name of the sender or the fact that the mail came via the list. It's also trivial to discover the email address of the original.
The specific reason I question the necessity is that I'm not seeing all the other mailing lists being driven to this measure. Indeed, I'm not seeing that elsewhere at all, including on the numerous technical mailing lists *I* administer in the San Francisco Bay Area.
The Linux Australia mailing lists have had continual ongoing issues in this regard. The last time I sent mail to the address advertised for contacting the LA council it was rejected because the LA server in question forwards mail to a different server that does SPF checks! The ongoing issue with LA servers is one of the factors that drove my work on the LUV server. Another issue is the fact that I want to sign my own mail. Through 2 different versions of Mailman I have not found another way of making it not break on DKIM signed mail that I send. Finally the world is just going towards DKIM. As much as some people have said that they want to just reject mail from Yahoo etc that's not something that LUV is going to do. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Quoting Russell Coker (russell@coker.com.au):
List servers ARE changing the sender. Instead of receiving a message directly from the person you receive mail from the list server which has very different issues with anti-spam etc.
{sigh} Um, I'll be charitable and assume you really didn't follow what I said. Here are the main SMTP + envelope headers from a post I sent yesterday to the main Silicon Valley Linux User Group discussion mailing list, as originally sent: From rick@linuxmafia.com Mon Nov 16 17: 2:17 2015 Date: Mon, 16 Nov 2015 17:22:17 -0800 From: Rick Moen <rick@linuxmafia.com> To: svlug@lists.svlug.org Subject: Re: [svlug] procmail (was: Some pretty serious parsing) Here, for comparison, is the retransmitted set of SMTP + envelope headers upon retransmission by SVLUG's Mailman instance and MTA: From svlug-bounces+rick=linuxmafia.com@lists.svlug.org Mon Nov 16 17: 2:51 2015 Date: Mon, 16 Nov 2015 17:22:17 -0800 From: Rick Moen <rick@linuxmafia.com> To: svlug@lists.svlug.org Subject: Re: [svlug] procmail (was: Some pretty serious parsing) Readers will please note that Mailman encodes the relaying information into the envelope 'From ' header but leaves all of my original SMTP headers alone. That is the way MLMs function all over the planet, and nobody in my experience is having DKIM or DMARC-related deliverability problems at this time. If it were otherwise, I would know, on account of being listadmin in a number of places where we'd be in trouble. I'm reasonably certain that the SVLUG, BALUG, and CABAL mailing list's, among many others I'm on, have significant numbers of Yahoo Mail users (even though Yahoo Mail has become really terrible), and we simply are not hearing of, or seeing deliverability problems. [...]
I don't believe that it is in any way falsified.
I am having a very difficult time believing you do not comprehend _exactly_ what Craig, I, and many others are saying. You would appear to be pretending to not understand, as surely we have all been abundantly clear and lucid. Now, I respect your right to make technical decisions you feel are right with LUV's mailing lists. I've described the damage that the current course of action is, in my view, doing. Please don't try to tell me I'm not seeing what I'm in fact seeing. Rather, if you wish to go ahead and do what you want, good luck to you. I'm going to leave it at that. [rest snipped]
Mark Trickett via luv-talk wrote:
Hello Russell,
On Tue, 2015-11-17 at 01:33 +1100, Russell Coker wrote:
On Mon, 16 Nov 2015 06:02:09 PM Anders Holmström wrote:
On 2015-11-16 06:26, Tim Connors wrote:
[...] And my LUV folder is now fugly with redundant department of redundancy style redundant information that everyone is from "via luv-talk".
+1 Adding "via luv-list" is a mistake, regardless of the merits, or otherwise, of munging the headers. I understand it's a way of saying "look! I've screwed with the headers" but is not necessary.
From: "Anders Holmström via luv-talk" <luv-talk@luv.asn.au> From: "Anders Holmström" <luv-talk@luv.asn.au>
So you're saying that of the above 2 options you prefer the second? Mailman doesn't appear to give that as an option, or if it does it's a site wide option not a per-list option.
It might well be that there is currently no satisfactory solution, short of working on the source code.
...can file a bug against upstream mailman, if there isn't one already. Then it might be fixed for you next time you dist-upgrade :-)
On Wed, Nov 18, 2015 at 11:22:32AM +1100, Trent W. Buck wrote:
Mark Trickett wrote:
It might well be that there is currently no satisfactory solution, short of working on the source code.
...can file a bug against upstream mailman, if there isn't one already.
Then it might be fixed for you next time you dist-upgrade :-)
Or there may already be a satisfactory solution. ========================================================================== TL;DR mailman 2.1.18 recommends using dmarc_moderation_action instead of from_is_list from 2.1.16 http://wiki.list.org/DEV/DMARC ========================================================================== I'd like to thank those who have already advocated (justifiably) against munging the "From:" field, allowing me to avoid the religious debate thus far. Although, since there's potentially a more widely acceptable solution, I must express that this munging can, and should be avoided. Accepting the consensus arrived at by RFC5323 §3.6.2: "In all cases, the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message." – [1] Where "SHOULD NOT" as defined by RFC2119 §4 means: "there may exist valid reasons in particular circumstances when the particular behaviour is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behaviour described with this label." – [2] I'm not convinced there's any "valid reasons" to munge the "From:" field in either of the following circumstances. 1) DKIM signing messages sent by the mailing list Unless there's a DMARC policy for a domain, there's no requirement for any parameter in the DKIM-Signature header field to match any other header field. In particular, the DKIM "d=" domain, and the domain in the "From:" field, aren't required to match in the absence of DMARC, (and ADSP, or DK) policy. Since this list's domain doesn't currently (or presumably, intend to) publish a DMARC policy record, I don't see DKIM signing of list mail as a valid reason to munge the "From:" field. 2) Lists receiving mail from domains with a restrictive policy The real issue here is that domains like Yahoo's publish a "p=reject" DMARC policy rule, and receiving domains (like Google's) are politely enforcing that policy by, you know, rejecting the mail that doesn't pass DMARC. Although rewriting the "From:" header on those posts is an option, (such as appending .INVALID to the domain, as per RFC2606) it's an *abysmal* choice, for reasons previously enumerated on this thread. Possible Solution: mailman 2.1.18 recommends using dmarc_moderation_action, instead of from_is_list from 2.1.16 [3] * Unset the from_is_list [4] "Munge From" which causes *all* of messages sent by the list to be munged. * Set dmarc_moderation_action [5] to one of; + Munge From - rewrite the From: and Reply-To: as in from_is_list + Wrap Message - wrap the message as in from_is_list Which should *only* affect messages received from domains with a restrictive DMARC policy (p=reject,p=quarantine) It seems as though this would be palatable to more people, and is probably a reason it was even implemented in 2.1.18 I'd personally prefer the latter, which preserves the original message (and header fields) in an RFC822 MIME sub-part (attachment), much like digests. FWIW: I'd paint the bike shed the following (different) colour: * Although wrapping the post to preserve the original header fields works, it's still a kludge, with those ugly from_is_list header fields, so I'd just reject the post(s) and send (a) sensible DMARC failure report(s) instead * Rejecting the post, as has already been stated by others on this thread, may provide the feedback to signers (via failure reports) to determine a better "market solution" * Disclaimer: I'm aware this solution has been deemed inappropriate for these lists; but in my own defence, during a past life I've endured the insanity of being a large site's postmaster. [1] https://tools.ietf.org/html/rfc5322#section-3.6.2 [2] https://tools.ietf.org/html/rfc2119#section-4 [3] http://wiki.list.org/DEV/DMARC [4] http://list.org/mailman-admin/general-personality.html [5] http://list.org/mailman-admin/sender-filters.html See also; DomainKeys Identified Mail (DKIM) Signatures https://tools.ietf.org/html/rfc6376 DKIM and Mailing Lists https://tools.ietf.org/html/rfc6377 DMARC https://tools.ietf.org/html/rfc7489
Quoting Joel W. Shea via luv-talk (luv-talk@luv.asn.au):
Possible Solution:
mailman 2.1.18 recommends using dmarc_moderation_action, instead of from_is_list from 2.1.16 [3]
* Unset the from_is_list [4] "Munge From" which causes *all* of messages sent by the list to be munged.
* Set dmarc_moderation_action [5] to one of; + Munge From - rewrite the From: and Reply-To: as in from_is_list + Wrap Message - wrap the message as in from_is_list
Which should *only* affect messages received from domains with a restrictive DMARC policy (p=reject,p=quarantine)
It seems as though this would be palatable to more people, and is probably a reason it was even implemented in 2.1.18
I'd personally prefer the latter, which preserves the original message (and header fields) in an RFC822 MIME sub-part (attachment), much like digests.
FWIW: I'd paint the bike shed the following (different) colour:
* Although wrapping the post to preserve the original header fields works, it's still a kludge, with those ugly from_is_list header fields, so I'd just reject the post(s) and send (a) sensible DMARC failure report(s) instead
* Rejecting the post, as has already been stated by others on this thread, may provide the feedback to signers (via failure reports) to determine a better "market solution"
Sounds eminently sane to me. If Yahoo mail's DMARC policy requires their domain's mail become screwed up, LUV should give it & confreres that, good and hard. -- Cheers, (morganj): 0 is false and 1 is true, correct? Rick Moen (alec_eso): 1, morganj rick@linuxmafia.com (morganj): bastard. McQ! (4x80) -- seen on IRC
"Joel W. Shea via luv-talk" <luv-talk@luv.asn.au> writes:
1) DKIM signing messages sent by the mailing list
Unless there's a DMARC policy for a domain, there's no requirement for any parameter in the DKIM-Signature header field to match any other header field. In particular, the DKIM "d=" domain, and the domain in the "From:" field, aren't required to match in the absence of DMARC, (and ADSP, or DK) policy.
Since this list's domain doesn't currently (or presumably, intend to) publish a DMARC policy record, I don't see DKIM signing of list mail as a valid reason to munge the "From:" field.
IIRC I think somebody said that Mailman breaks the DKIM on the original message. Which provokes the question - why not fix Mailman so it doesn't break the DKIM on the original message? IIRC this is because of the design of Mailman; it interprets the headers and writes them out again, so it can't write them back exactly the same as they were before. I don't have the references handy, I apologise in advance if I got the above wrong. Which seems to be saying we need to mangle the From: header due to poor design decisions in Mailman. Also see the "Annotations by mailing lists" section in https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail - It seems to suggest other options are available, e.g. using SPF to whitelist DKIM, or to use the Sender: header instead of the From: header. -- Brian May <brian@linuxpenguins.xyz> https://linuxpenguins.xyz/brian/
Quoting Brian May via luv-talk (luv-talk@luv.asn.au):
Which provokes the question - why not fix Mailman so it doesn't break the DKIM on the original message?
My understanding: Because the sending IP has changed. I might be missing something here, but the basic job of DKIM/DMARC or SPF is to provide a means of determining that the mail has arrived from an IP address authorised by the claimed sending domain's owner (what is in the internal SMTP 'From: ' header) as a legitimate sending point for outgoing mail.[1] The domain owner publishes this information in the domain's DNS. When I send a mail (bearing sending domain linuxmafia.com) to a LUV mailing list, it originates from my server's IP, 198.144.195.186. Any receiving MTA such as LUV's on the MX for luv.asn.au, can check my domain's published SPF record: $ dig -t txt linuxmafia.com +short "v=spf1 a mx -all" $ That says 'You should consider to be arriving from an authorised IP any mail from an IP matching my "A" or "MX" record.' $ dig -t mx linuxmafia.com +short 10 linuxmafia.com. $ dig -t a linuxmafia.com +short 198.144.195.186 $ Et voila. It matches. _Unfortunately_, upon retransmission by luv.asn.au, and re-mailed out to all mailing list subscribers, this time it arrives at the destination MTAs (for the subscribers) from luv.asn.au's IP address, 202.158.218.240. 202.158.218.240 is _not_ a match for sending domain linuxmafia.com's list of authorised sending IPs. So, if the claimed sending domain in the internal SMTP 'From: ' header still says linuxmafia.com, it can no longer pass an SPF check. I've used SPF for this example because my domain's DNS doesn't (yet) publish DKIM or DMARC records, and because I'm more familiar with SPF at this time.
I don't have the references handy, I apologise in advance if I got the above wrong.
Likewise. (it's been a long day here, I'm dead-tired, and also I'm short on time to study this. Sorry.) -- Cheers, (morganj): 0 is false and 1 is true, correct? Rick Moen (alec_eso): 1, morganj rick@linuxmafia.com (morganj): bastard. McQ! (4x80) -- seen on IRC
On Thu, 19 Nov 2015, Rick Moen via luv-talk wrote:
$ dig -t mx linuxmafia.com +short 10 linuxmafia.com. $ dig -t a linuxmafia.com +short 198.144.195.186 $
Et voila. It matches.
_Unfortunately_, upon retransmission by luv.asn.au, and re-mailed out to all mailing list subscribers, this time it arrives at the destination MTAs (for the subscribers) from luv.asn.au's IP address, 202.158.218.240.
202.158.218.240 is _not_ a match for sending domain linuxmafia.com's list of authorised sending IPs. So, if the claimed sending domain in the internal SMTP 'From: ' header still says linuxmafia.com, it can no longer pass an SPF check.
Isn't cascading Received: from .* (.*) by headers completely normal in mail? Isn't a mailing list just another MTA? Can't it just prepend another Received: line and forward it on? If that breaks DMARC/SFP, doesn't that then mean DMARC/SFP is fundamentally broken? -- Tim Connors
On Fri, Nov 20, 2015 at 01:53:10PM +1100, Tim Connors wrote:
Isn't cascading Received: from .* (.*) by headers completely normal in mail?
Yes, it's completely normal in "plain message forwarding"...
Isn't a mailing list just another MTA? Can't it just prepend another Received: line and forward it on?
Well, they sort of are, and they could just...
If that breaks DMARC/SFP, doesn't that then mean DMARC/SFP is fundamentally broken?
...which may (depending on the SFP policy) cause the mail to eventually bounce, thus a mailing list will rewrite the Return-Path (e.g. with a VERP address) Co-incidentally, your mail didn't pass an SFP check, and your message may have been rejected if Google didn't publish a SOFTFAIL policy. However, given the existing controversy, and from the tone of your post, I'm sure you already know all that...
Quoting Tim Connors (tim.w.connors@gmail.com):
Isn't cascading Received: from .* (.*) by headers completely normal in mail? Isn't a mailing list just another MTA?
You might have noticed that SPF RRs in DNS sometimes have include directives so as to say 'I expect to be at least sometimes forwarding my outboudn mail through _this_ domain's outbound mail servers, so you should include the IPs pointed to that domain's A record, or MX records etc." But, in any case, you have to be able to publish in your domain's SPF RR, in some way or other, all of the places deemed legitimate handlers of your domain's outbound mail, or there's no point. Example: GMail. $ dig -t txt gmail.com +short "v=spf1 redirect=_spf.google.com" $ dig -t txt _spf.google.com +short "v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all" $ I can't speak to how this is handled in DKIM / DMARC, but it's an implementation of the same core concept.
Can't it just prepend another Received: line and forward it on? If that breaks DMARC/SFP, doesn't that then mean DMARC/SFP is fundamentally broken?
I would think so. We need a centralized Internet where all people "live" on Google or Yahoo and Facebook and Twitter. All other traffic should be banned. They think you need SSL to have a website. So you need an authority to give you a certificate. You need a laptop with some WeirdBoot which prevents you to install software of your choice. You need an AppleID to run your phone. You.. don't worry, it's all good for you. You do not need to lock your door at home. You have nothing to hide, aren't you? Regards Peter
On Mon, Nov 23, 2015 at 11:49:39AM +1100, Peter Ross via luv-talk wrote:
[...]
You do not need to lock your door at home. You have nothing to hide, aren't you?
and don't forget your Internet-of-Things toaster, microwave, fridge, washing machine and Winston Smith brand telescreen. and, of course, your web-cam enabled toilet-roll holder. craig -- craig sanders <cas@taz.net.au>
On Thu, Nov 19, 2015 at 06:16:16PM -0800, Rick Moen via luv-talk wrote:
Quoting Brian May via luv-talk (luv-talk@luv.asn.au):
Which provokes the question - why not fix Mailman so it doesn't break the DKIM on the original message?
My understanding: Because the sending IP has changed.
I might be missing something here, but the basic job of DKIM/DMARC or SPF is to provide a means of determining that the mail has arrived from an IP address authorised by the claimed sending domain's owner (what is in the internal SMTP 'From: ' header) as a legitimate sending point for outgoing mail.[1] The domain owner publishes this information in the domain's DNS.
You *might* be conflating DKIM with DMARC (and in particular its relationship with SPF) As you have stated, SPF only protects the SMTP envelope (via policy); whereas DKIM protects the message itself (via cryptographic signature). DMARC expects at least one (or both) of the above to pass, *and* that the domain(s) align with those in the "From:" header field. Both SPF/DMARC policy, and DKIM public keys; are all published via DNS. [...]
Quoting Joel W. Shea via luv-talk (luv-talk@luv.asn.au):
You *might* be conflating DKIM with DMARC (and in particular its relationship with SPF)
I think you're right. Upon some review reading, my understanding is that MARC is an elaboration (expansion in scope) of DKIM (which in turn was an elaboration of DomainKeys), and DKIM implements digital signing & vetting of various SMTP headers and the body text. It doesn't attempt to validate the envelope, as does SPF. Yes, I do believe you've summarised the matter well.
On Sat, 21 Nov 2015 08:54:58 AM Rick Moen via luv-talk wrote:
Quoting Joel W. Shea via luv-talk (luv-talk@luv.asn.au):
You *might* be conflating DKIM with DMARC (and in particular its relationship with SPF)
I think you're right. Upon some review reading, my understanding is that MARC is an elaboration (expansion in scope) of DKIM (which in turn was an elaboration of DomainKeys), and DKIM implements digital signing & vetting of various SMTP headers and the body text. It doesn't attempt to validate the envelope, as does SPF.
DMARC tells the recipient what it should do when DKIM signatures don't match and/or when SPF records don't validate. It's additional to those things and sort of an expansion of ADSP. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Rick Moen via luv-talk <luv-talk@luv.asn.au> writes:
My understanding: Because the sending IP has changed.
I might be missing something here, but the basic job of DKIM/DMARC or SPF is to provide a means of determining that the mail has arrived from an IP address authorised by the claimed sending domain's owner (what is in the internal SMTP 'From: ' header) as a legitimate sending point for outgoing mail.[1] The domain owner publishes this information in the domain's DNS.
I don't think DKIM actually cares what the sender's address is. Unlike SPF. Just as long as there is a good signature, and it can look up the signature in DNS, I think that is all that matters. -- Brian May <brian@linuxpenguins.xyz> https://linuxpenguins.xyz/brian/
Quoting Brian May (brian@linuxpenguins.xyz):
I don't think DKIM actually cares what the sender's address is. Unlike SPF. Just as long as there is a good signature, and it can look up the signature in DNS, I think that is all that matters.
Could be. I am not that familiar with the spec.
On Fri, Nov 20, 2015 at 07:29:04AM +1100, Brian May wrote:
"Joel W. Shea via luv-talk" <luv-talk@luv.asn.au> writes:
1) DKIM signing messages sent by the mailing list
[...]
IIRC I think somebody said that Mailman breaks the DKIM on the original message.
I was referring to the mailing list making it's own signatures, but you're correct, in some circumstances mailmain will break signatures, for instance; by rewriting the From/Subject fields where they were signed by the original sender.
Which provokes the question - why not fix Mailman so it doesn't break the DKIM on the original message?
Mailman can already sensibly handle DKIM signatures.
IIRC this is because of the design of Mailman; it interprets the headers and writes them out again, so it can't write them back exactly the same as they were before.
I don't have the references handy, I apologise in advance if I got the above wrong.
I recall there being a long discussion thread on the mailman list, late last decade, regarding the handling of DKIM signature edge-cases, but I think things have improved since then
Which seems to be saying we need to mangle the From: header due to poor design decisions in Mailman.
Only if we want forwarded mail to pass DMARC policy, which is a seperate issue (the second one on my original post) [...]
On Fri, 20 Nov 2015 03:08:05 PM Joel W. Shea via luv-talk wrote:
IIRC I think somebody said that Mailman breaks the DKIM on the original message.
I was referring to the mailing list making it's own signatures, but you're correct, in some circumstances mailmain will break signatures, for instance; by rewriting the From/Subject fields where they were signed by the original sender.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802048 The previous version of Mailman that we used (from Debian/Wheezy) would reformat the DKIM header to use spaces instead of tabs which for some reason broke all the Linux DKIM software I tested with. Ideally we would have the DKIM checking handle this, I filed the above Debian bug about this. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802047 It would also be possible for Mailman to have special case code for DKIM headers. I filed the above bug report requesting this. But I don't expect that anything will happen in this regard as the new version of Mailman in Debian/Jessie now base64 encodes message bodies for no good reason and with no apparent way to disable it. It MIGHT be possible to make Mailman forward mail without breaking DKIM if the sender uses base64 encoding, but if the sender doesn't (as my favourite MUA doesn't) then that seems impossible.
Which provokes the question - why not fix Mailman so it doesn't break the DKIM on the original message?
Mailman can already sensibly handle DKIM signatures.
No it seems to be broken and becoming more broken. If you think this can be solved then please tell me how.
IIRC this is because of the design of Mailman; it interprets the headers and writes them out again, so it can't write them back exactly the same as they were before.
I don't have the references handy, I apologise in advance if I got the above wrong.
I recall there being a long discussion thread on the mailman list, late last decade, regarding the handling of DKIM signature edge-cases, but I think things have improved since then
Testing proves otherwise.
Which seems to be saying we need to mangle the From: header due to poor design decisions in Mailman.
Only if we want forwarded mail to pass DMARC policy, which is a seperate issue (the second one on my original post)
If we want to support Gmail recipients then we have to do that. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Fri, Nov 20, 2015 at 08:05:48PM +1100, Russell Coker wrote:
On Fri, 20 Nov 2015 03:08:05 PM Joel W. Shea wrote:
On Fri, Nov 20, 2015 at 07:29:04AM +1100, Brian May wrote:
IIRC I think somebody said that Mailman breaks the DKIM on the original message.
I was referring to the mailing list making its own signatures [...]
[...] The previous version of Mailman that we used (from Debian/Wheezy) would reformat the DKIM header to use spaces instead of tabs which for some reason broke all the Linux DKIM software I tested with.
Thanks, I'd forgotten about <201510172046.45136.russell@coker.com.au> http://lists.luv.asn.au/pipermail/luv-main/2015-October/008489.html
Ideally we would have the DKIM checking handle this [...]
libmail-dkim-perl behaves correctly in that scenario, since the header specified "c=simple/simple", where any change in that header will also break the signature, IIRC. Perhaps using "c=relaxed/relaxed" (as Google does) would resolve that issue? However, that would depend on the sender's signing policy (over which not all authors will have control), so the following...
[...] It would also be possible for Mailman to have special case code for DKIM headers [...]
...seems to be the ideal solution.
But I don't expect that anything will happen in this regard as the new version of Mailman in Debian/Jessie now base64 encodes message bodies for no good reason and with no apparent way to disable it.
Even if that weren't the case; it seems like the previous header mangling issue *might* be because of the Python standard library, and I'm unsure how easily it can be fixed.
It MIGHT be possible to make Mailman forward mail without breaking DKIM if the sender uses base64 encoding,
Assuming the header issue was also fixed...
but if the sender doesn't (as my favourite MUA doesn't) then that seems impossible.
...or there were also a way to disable it (or revert the change that introduced it)
Which provokes the question - why not fix Mailman so it doesn't break the DKIM on the original message?
I'll take a *guess*, and assume it's to avoid shaving a yak; the 2.1.x maintainers don't have the time (or can justify) working around DKIM edge cases by also having to work around the Python standard library.
Mailman can already sensibly handle DKIM signatures.
No it seems to be broken and becoming more broken. If you think this can be solved then please tell me how.
I agree, where "sensibly handle" means "preserve" (rather than "remove") In which case, I don't think there's a solution that doesn't require at least a DKIM aware mailing list manager (of which it appears Mailman 2.1.x is not)
IIRC this is because of the design of Mailman; it interprets the headers and writes them out again, so it can't write them back exactly the same as they were before.
In using the standard library to parse the message, that's exactly what appears to be happening.
I don't have the references handy, I apologise in advance if I got the above wrong.
No, seems you were right, on both counts.
[...]
[...]
Which seems to be saying we need to mangle the From: header due to poor design decisions in Mailman.
Only if we want forwarded mail to pass DMARC policy, which is a seperate issue (the second one on my original post)
If we want to support Gmail recipients then we have to do that.
Yes, unless DKIM signatures could be preserved; but again, only for those messages sent from a domain publishing a restrictive DMARC policy
On Thu, 19 Nov 2015 08:48:34 PM Joel W. Shea via luv-talk wrote: ==========================================================================
TL;DR mailman 2.1.18 recommends using dmarc_moderation_action instead of from_is_list from 2.1.16
Of the options for dmarc_moderation_action: 1) means mail will be rejected or flagged as spam by some recipients (everyone using Gmail for starters). 2) Is what we currently do. 3) is really ugly and will make things more difficult for readers. 4 and 5) we don't want to reject mail.
Accepting the consensus arrived at by RFC5323 §3.6.2: "In all cases, the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message." – [1]
Unless of course the message body and headers are being rewritten such that it's not the same message as was originally sent.
Where "SHOULD NOT" as defined by RFC2119 §4 means: "there may exist valid reasons in particular circumstances when the particular behaviour is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behaviour described with this label." – [2]
There are valid reasons and the implications are known.
I'm not convinced there's any "valid reasons" to munge the "From:" field in either of the following circumstances.
1) DKIM signing messages sent by the mailing list
Unless there's a DMARC policy for a domain, there's no requirement for any parameter in the DKIM-Signature header field to match any other header field. In particular, the DKIM "d=" domain, and the domain in the "From:" field, aren't required to match in the absence of DMARC, (and ADSP, or DK) policy.
Since this list's domain doesn't currently (or presumably, intend to) publish a DMARC policy record, I don't see DKIM signing of list mail as a valid reason to munge the "From:" field.
Whether the luv.asn.au domain has a ADSP or DMARC record is not relevant. What is relevant is domains like yahoo.com.
2) Lists receiving mail from domains with a restrictive policy
The real issue here is that domains like Yahoo's publish a "p=reject" DMARC policy rule, and receiving domains (like Google's) are politely enforcing that policy by, you know, rejecting the mail that doesn't pass DMARC.
Although rewriting the "From:" header on those posts is an option, (such as appending .INVALID to the domain, as per RFC2606) it's an *abysmal* choice, for reasons previously enumerated on this thread.
All the choices have downsides, this one seems like the least difficult.
Possible Solution:
mailman 2.1.18 recommends using dmarc_moderation_action, instead of from_is_list from 2.1.16 [3]
* Unset the from_is_list [4] "Munge From" which causes *all* of messages sent by the list to be munged.
* Set dmarc_moderation_action [5] to one of; + Munge From - rewrite the From: and Reply-To: as in from_is_list
Do you realise that you just wrote a long message railing against rewriting the From field and came to the conclusion that it's a good option?
* Rejecting the post, as has already been stated by others on this thread, may provide the feedback to signers (via failure reports) to determine a better "market solution"
In that case the "market solution" might be "don't deal with Linux people as they make things difficult". -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Russell, thank you for the response, and for the maintenance you are providing for LUV infrastructure. On Fri, Nov 20, 2015 at 01:27:18PM +1100, Russell Coker wrote:
On Thu, 19 Nov 2015 08:48:34 PM Joel W. Shea wrote:
[...]
Of the options for dmarc_moderation_action: [...] 2) Is what we currently do. 3) is really ugly and will make things more difficult for readers. [...]
If that's the case, either: from_is_list is also set, or dmarc_moderation_action is broken; as it should only be munging the "From:" field on messages arriving from domains with a restrictive DMARC policy. I concede the following point may be subjective; but I think wrapping is actually less-ugly, since it gives a clearer picture of what's happening, than simply munging the header; and agree it's more difficult for readers, in that it adds an extra keystroke for me to open the message in my MUA. (but I actually prefer that in the case of digests) [...]
the "From:" field SHOULD NOT contain any mailbox that does not belong to the author(s) of the message."
Unless of course the message body and headers are being rewritten such that it's not the same message as was originally sent.
I agree that's a fair interpretation, but (IMO) also implies that the forwarder of that message now claims authorship. [...]
I'm not convinced there's any "valid reasons" to munge the "From:" field in either of the following circumstances.
1) DKIM signing messages sent by the mailing list [...]
Whether the luv.asn.au domain has a ADSP or DMARC record is not relevant. What is relevant is domains like yahoo.com.
I should clarify this circumstance; it's only relevant where luv.asn.au is adding its own DKIM-Signature, it does not require the domain in the "From:" header field to align.
2) Lists receiving mail from domains with a restrictive policy
All the choices have downsides, this one seems like the least difficult.
I reluctantly agree, but *only* for messages sent from a domain with a DMARC policy of reject/quarantine. Hence dmarc_moderation_action being preferred *instead of* from_is_list.
Possible Solution:
Do you realise that you just wrote a long message railing against rewriting the From field and came to the conclusion that it's a good option?
Yes, I was adding my support to the consensus against rewriting that field. Again, I reluctantly agree it's probably the only solution that meets LUV requirements, but *only* where it's absolutely necessary, and not on every message forwarded by the mailing list(s).
* Rejecting the post, as has already been stated by others on this thread, may provide the feedback to signers (via failure reports) to determine a better "market solution"
In that case the "market solution" might be "don't deal with Linux people as they make things difficult". [...]
I'd contend that corporations implementing standards in their own interests, and ignoring legitimate real world usage; are making things difficult. Ideally, I'd also like to see the standard improved to allow receivers to defer to the additional signatures of "trusted" third-party relay/forwarders; although, determining trust is another problem.
On 12/11/2015 7:41 PM, Craig Sanders via luv-talk wrote:
and if yahoo cares about keeping their users, they'll change what they're doing.
Isn't Yahoo just a security nightmare in themselves? It seems that EVERYONE would do well to avoid yahoo as much as possible. A.
participants (12)
-
Anders Holmström -
Andrew McGlashan -
Brian May -
Craig Sanders -
Joel W. Shea -
Mark Trickett -
MikeH -
Peter Ross -
Rick Moen -
Russell Coker -
Tim Connors -
Trent W. Buck