On Wed, 13 Jan 2016 08:42:23 PM Erik Christiansen via luv-main wrote:
I got it wrong, was actually referring to From: address munging.
Nah, I wouldn't hold my breath. Hints on that one have been going to /dev/null since last year. It seems we're all supposed to undo the crap individually at our ends.
We had a problem of Gmail and other recipients falsely regarding mail from the LUV server as forged and assigning it to the spam folder. We are dealing with a couple of bugs in Mailman, what I consider to be a deficiency in OpenDKIM and some interactions between SPF and DMARC. If you would like to help with some of these technical issues then go for it. http://postmaster.facebook.com/reputation_and_authentication Facebook is now compelling sysadmins to use SPF or DKIM. This isn't going to go away. It's only a matter of time before Internode starts using DKIM to placate Facebook. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On 15/01/2016 12:14 AM, Russell Coker via luv-main wrote:
On Wed, 13 Jan 2016 08:42:23 PM Erik Christiansen via luv-main wrote:
I got it wrong, was actually referring to From: address munging.
Nah, I wouldn't hold my breath. Hints on that one have been going to /dev/null since last year. It seems we're all supposed to undo the crap individually at our ends.
We had a problem of Gmail and other recipients falsely regarding mail from the LUV server as forged and assigning it to the spam folder.
Nothing that can't be fixed with a filter (use the "Never send it to spam" option in the Gmail filter for LUV list mail).
We are dealing with a couple of bugs in Mailman, what I consider to be a deficiency in OpenDKIM and some interactions between SPF and DMARC.
If you would like to help with some of these technical issues then go for it.
http://postmaster.facebook.com/reputation_and_authentication
Facebook is now compelling sysadmins to use SPF or DKIM. This isn't going to go away. It's only a matter of time before Internode starts using DKIM to placate Facebook.
Looks like it's a case of having to follow suit., like it or not. -- 73 de Tony VK3JED/VK3IRL http://vkradio.com
On Fri, 15 Jan 2016 09:52:30 AM Tony Langdon via luv-main wrote:
We had a problem of Gmail and other recipients falsely regarding mail from the LUV server as forged and assigning it to the spam folder.
Nothing that can't be fixed with a filter (use the "Never send it to spam" option in the Gmail filter for LUV list mail).
That only works for senders who's domains don't have DMARC records. If someone sends mail from yahoo.com to the list then Gmail is expected to bounce the message as yahoo.com has a DMARC record requesting this. Also that requires that every Gmail user implement such a change. When we had Gmail users complaining about list mail going to spam it didn't seem that they were capable of fixing that.
Facebook is now compelling sysadmins to use SPF or DKIM. This isn't going to go away. It's only a matter of time before Internode starts using DKIM to placate Facebook.
Looks like it's a case of having to follow suit., like it or not.
Yes. I'd appreciate it if people would stop acting like I'm doing something I want to do here. I just want mail to go through reliably and I'm doing what is necessary to achieve that goal. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Russell Coker via luv-main <luv-main@luv.asn.au> wrote:
On Fri, 15 Jan 2016 09:52:30 AM Tony Langdon via luv-main wrote:
Facebook is now compelling sysadmins to use SPF or DKIM. This isn't going to go away. It's only a matter of time before Internode starts using DKIM to placate Facebook.
Looks like it's a case of having to follow suit., like it or not.
I predict that mailing lists, and mail behaviour in general, will simply need to adapt to SPF, DKIM and DMARC as the emerging means of establishing the authenticity of messages with a certain degree of assurance. What this means is that (provided the sending MTA only serves authorized users) there is reasonable assurance that the sender identified in the "From" header is the actual sender, and that certain header fields and the body have not been modified in transit.
Yes. I'd appreciate it if people would stop acting like I'm doing something I want to do here. I just want mail to go through reliably and I'm doing what is necessary to achieve that goal.
Widespread use of DMARC will result in changes to well established conventions. I don't personally object to having the list server rewrite the "From" field and add a "Reply-to" header that designates the original sender; but some people have needs which differ from mine, and for them it can be an inconvenience. If I were running a mailing list server, however, I would make it compatible with DMARC "reject" policies.
On Thu, Jan 14, 2016 at 08:56:52PM -0500, Jason White wrote:
Russell Coker via luv-main <luv-main@luv.asn.au> wrote:
Yes. I'd appreciate it if people would stop acting like I'm doing something I want to do here. I just want mail to go through reliably and I'm doing what is necessary to achieve that goal.
I understand that it's a complicated issue, and although it's not absolutely necessary to set from_as_list, that you are only doing that to make the header munging consistent across all messages, as dmarc moderation would only munge a subset. Although I'd personally prefer the latter, I suppose we must agree to disagree on that.
Widespread use of DMARC will result in changes to well established conventions. I don't personally object to having the list server rewrite the "From" field and add a "Reply-to" header that designates the original sender; but some people have needs which differ from mine, and for them it can be an inconvenience.
Yes, it also has the potential to reduce the distributed and decentralised nature of email; I.e. if you're not sending via a "reputable" service provider, your mail wont get routed. This is antithetical to an open Internet, Internet neutrality, and it could also be considered a means of censorship by some.
If I were running a mailing list server, however, I would make it compatible with DMARC "reject" policies.
Likewise. The more I think about it, the more I dislike the current DMARC standard, it'd be nice if it could also do "Sender:" field alignment in the case of SPF, where DKIM alignment has also been satisfied by at least one signature. This would probably work towards solving the legitimate cases of mail forwarding. I believe this has also been suggested by others in the past, but went nowhere; along with the concept of an "X-Original-Authentication-Result" header, although those could be easily forged, and we're back to the subjectivity of trust problem.
On Fri, 15 Jan 2016 04:01:28 AM Joel W. Shea via luv-main wrote:
Widespread use of DMARC will result in changes to well established conventions. I don't personally object to having the list server rewrite the "From" field and add a "Reply-to" header that designates the original sender; but some people have needs which differ from mine, and for them it can be an inconvenience.
Yes, it also has the potential to reduce the distributed and decentralised nature of email;
Not at all. The distributed and decentralised part of email is inherently not mailing lists. By definition lists are centralised!
I.e. if you're not sending via a "reputable" service provider, your mail wont get routed. This is antithetical to an open Internet, Internet neutrality, and it could also be considered a means of censorship by some.
No it just means that you have to use other services. If you and a bunch of like minded people wanted to have a list where no-one used DKIM/DMARC then there's nothing stopping you from doing it. As for censorship, if you are worried about that then you probably shouldn't be using Gmail at all.
I believe this has also been suggested by others in the past, but went nowhere; along with the concept of an "X-Original-Authentication-Result" header, although those could be easily forged, and we're back to the subjectivity of trust problem.
The real issue is that nothing we agree on matters much if Google and Yahoo don't agree.
On Fri, Jan 15, 2016 at 04:30:47AM +0000, Russell Coker wrote:
Not at all. The distributed and decentralised part of email is inherently not mailing lists. By definition lists are centralised!
By that logic, either are mailbox providers, as they are also centralised
No it just means that you have to use other services.
Yes, a "reputable" one, as defined by what ever criteria some few large corporations dictate.
[...]
Which would still fail if corporations suddenly started dictating that you must have a DMARC policy for your mail to be accepted.
As for censorship, if you are worried about that then you probably shouldn't be using Gmail at all.
Agreed, we should all run our own mail services. Or at least, select those with sane policies if we want to be uncensored.
The real issue is that nothing we agree on matters much if Google and Yahoo don't agree.
Also agreed, pretty much what I was inferring with the hyperbole about Internet nuetrality.
On Fri, 15 Jan 2016 03:56:38 PM Joel W. Shea wrote:
On Fri, Jan 15, 2016 at 04:30:47AM +0000, Russell Coker wrote:
Not at all. The distributed and decentralised part of email is inherently not mailing lists. By definition lists are centralised!
By that logic, either are mailbox providers, as they are also centralised
They are only centralised to the extent that you have to use them. For typical computer users it's very centralised. For typical members of this list running their own mail server is an option. Is there an email equivalent to torchat? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Fri, Jan 15, 2016 at 04:21:23PM +1100, Russell Coker wrote:
Is there an email equivalent to torchat?
I'm not not that parfait with torchat, but it's based on local hidden services, AFAICT. So I imagine any email service that only allows connections from tor (via hidden service), and only between other tor hidden services, would be the nearest analogue; for which there are many options.
On Fri, Jan 15, 2016 at 04:30:47AM +0000, Russell Coker wrote:
On Fri, 15 Jan 2016 04:01:28 AM Joel W. Shea via luv-main wrote:
Yes, it also has the potential to reduce the distributed and decentralised nature of email;
Not at all. The distributed and decentralised part of email is inherently not mailing lists. By definition lists are centralised!
yeah. and sucks to be us if we want to run our own lists and not google or yahoo or msn shite. being able to run mailing lists is an essential part of the open internet and *IS* de-centralised. At least until the corporates manage to kill off any alternatives to their spyware services via DKIM.
The real issue is that nothing we agree on matters much if Google and Yahoo don't agree.
To the contrary, nothing that google or yahoo demand matters if we just ignore them. Neither of them are necessary to the function of our list. If they want to do a dis-service to their users by rejecting mail from legitimate lists, that's a problem for them and their users, not a problem for us. if mail from the list sent to gmail etc bounces because of their misconfiguration, the correct response is to think "shit happens. this is nothing new, there's always been idiot mail admins at large corporations" and ignore it, not jump through their hoops that are designed to let them gain complete control over all email on the net. craig -- craig sanders <cas@taz.net.au>
On Fri, Jan 15, 2016 at 05:03:39PM +1100, Craig Sanders wrote:
being able to run mailing lists is an essential part of the open internet and *IS* de-centralised. At least until the corporates manage to kill off any alternatives to their spyware services via DKIM.
sorry. i've typed DKIM here and in other places, but meant DMARC. craig -- craig sanders <cas@taz.net.au>
Russell Coker via luv-main <luv-main@luv.asn.au> writes:
Not at all. The distributed and decentralised part of email is inherently not mailing lists. By definition lists are centralised!
nntp was suppose to be the distrbuted solution for mailing lists. Shame it didn't work out. Mailing lists have won, but the RFC822 headers weren't intended for mailing lists. -- Brian May <brian@linuxpenguins.xyz> https://linuxpenguins.xyz/brian/
On Fri, Jan 15, 2016 at 08:18:56PM +1100, Brian May via luv-main wrote:
nntp was suppose to be the distrbuted solution for mailing lists.
Excellent point!
Shame it didn't work out. Mailing lists have won, but the RFC822 headers weren't intended for mailing lists.
Indeed, and both were replaced by web forums in many instances. I guess there's no accounting for taste, eh?
Joel W. Shea via luv-main <luv-main@luv.asn.au> wrote:
On Thu, Jan 14, 2016 at 08:56:52PM -0500, Jason White wrote:
Widespread use of DMARC will result in changes to well established conventions. I don't personally object to having the list server rewrite the "From" field and add a "Reply-to" header that designates the original sender; but some people have needs which differ from mine, and for them it can be an inconvenience.
Yes, it also has the potential to reduce the distributed and decentralised nature of email;
I.e. if you're not sending via a "reputable" service provider, your mail wont get routed. This is antithetical to an open Internet, Internet neutrality, and it could also be considered a means of censorship by some.
That's an interesting observation. On the other side, it isn't hard to set up a "reputable" provider. I recently did so by configuring SPF, DKIM and DMARC on my mail server. I am nevertheless concerned by the centralization of mail services in the hands of large providers, and likewise the rise of large social networking sites which merge the features of mail, instant messaging, voice and video services, among others - all in a proprietary manner that lacks interoperability. Then there's the data mining which takes place to enable the social Web site operators to monetize the activities of their users.
On Sat, 16 Jan 2016 12:08:16 AM Jason White via luv-main wrote:
That's an interesting observation. On the other side, it isn't hard to set up a "reputable" provider. I recently did so by configuring SPF, DKIM and DMARC on my mail server.
If you send several messages to the Linux Australia list that aren't replies to other messages (IE they will have their subject munged) then you will trigger a mass unsubscribe of people who use services like Hotmail and Gmail. That's what happened when I did that in January 2015.
I am nevertheless concerned by the centralization of mail services in the hands of large providers, and likewise the rise of large social networking sites which merge the features of mail, instant messaging, voice and video services, among others - all in a proprietary manner that lacks interoperability. Then there's the data mining which takes place to enable the social Web site operators to monetize the activities of their users.
I think that the merging is the most serious threat we face. Gmail is a good mail service. Hangouts is a good IM system which does voice and video in a seamless manner. This combined with the fact that they are setup by default on almost every Android device, available from the Apple app store, and much of the functionality is available in any web browser makes it a compelling service. Setting up a mail server for yourself that does everything you desire from the Gmail feature set isn't trivial for people like us. IM is hard, ejabberd seems to be the best option, it's fragile and doesn't have the features of Hangouts and Jabber doesn't have a significant userbase of people who can talk to you. VOIP and video calls is harder still as it requires better net access than you can get easily and cheaply in Australia and some significant effort to set things up. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Thu, Jan 14, 2016 at 08:56:52PM -0500, Jason White wrote:
Russell Coker via luv-main <luv-main@luv.asn.au> wrote:
On Fri, 15 Jan 2016 09:52:30 AM Tony Langdon via luv-main wrote:
Facebook is now compelling sysadmins to use SPF or DKIM. This isn't going to go away. It's only a matter of time before Internode starts using DKIM to placate Facebook.
Looks like it's a case of having to follow suit., like it or not.
well, SPF is fine. no problem with that. DKIM is an ill-conceived abomination. It actually cares about the *headers* in a message rather than the **envelope*. To an MTA, headers are irrelevant, they're just comments....what matters is the envelope sender address and the envelope recipient address. And worse, DKIM cares about the From: header rather than the Sender: header. This is just broken in every possible way.
Yes. I'd appreciate it if people would stop acting like I'm doing something I want to do here. I just want mail to go through reliably and I'm doing what is necessary to achieve that goal.
unfortunately, you stil haven't impleemnted the minimum-damage option that only munges posts that are sent by users from domains that implement DKIM (like google or yahoo), and leaves other mail alone. it's not like anyone ever posts from those domains to our lists anyway. which is one of the more annoying things about this issue - the configuration messes things up for active participants on the list, and it doesn't even provide any benefit to the lurkers who never say or contribute anything. That's entirely the wrong thing to do. those who contribute may well stop bothering if they get annoyed enough, and non-contributors won't step forward to replace them...if they were inclined to, they'd already be posting. driving away those who write the posts (that both they and the lurkers read) is self-defeating.
Widespread use of DMARC will result in changes to well established conventions.
IMO it's an attempt by major corporate players to completely take over email so that no email is ever sent that they don't get a copy of to examine and index and use to build up profiles on individuals. and to sell to the NSA etc of course. Message forgery is a solved problem. SPF works. DKIM is a) overkill and b) unnecesary. If individual senders need more identity verification than SPF, there are numerous encryption and signing options available....with support built in to many MUAs.
I don't personally object to having the list server rewrite the "From" field and add a "Reply-to" header that designates the original sender; but some people have needs which differ from mine, and for them it can be an inconvenience.
NO! those who refuse to learn from history are doomed to repeat the same damned stupid mistakes. This issue was settled definitively in the 90s. Mailing lists should *never*, under any circumstances, mess with the Reply-To header. That belongs solely to the original sender. Lists have several alternatives they can use, including Mail-Followup-To: and List-Post: and Lists shouldn't mess with the From: header, either. No matter what corporate vermin demand. WGAF what facebook wants? how many emails from luv lists ever go to facebook? craig -- craig sanders <cas@taz.net.au>
On Fri, Jan 15, 2016 at 04:54:22PM +1100, Craig Sanders via luv-main wrote:
On Thu, Jan 14, 2016 at 08:56:52PM -0500, Jason White wrote:
Russell Coker via luv-main <luv-main@luv.asn.au> wrote:
[...] And worse, DKIM cares about the From: header rather than the Sender: header.
I'm assuming this was one of those places where you meant DMARC, because DKIM only cares about what you tell it to sign (or not, for that matter)
This is just broken in every possible way.
Yes. I'd appreciate it if people would stop acting like I'm doing something I want to do here. I just want mail to go through reliably and I'm doing what is necessary to achieve that goal.
[...] That's entirely the wrong thing to do. those who contribute may well stop bothering if they get annoyed enough, [...] driving away those who write the posts (that both they and the lurkers read) is self-defeating.
Conversely, it's actually caused me to contribute more than usual, merely to express said annoyance, I'm not sure I want to burn any more calories bothering in future, and have certainly considered unsubscribing.
Widespread use of DMARC will result in changes to well established conventions.
IMO it's an attempt by major corporate players to completely take over email so that no email is ever sent that they don't get a copy of to examine and index and use to build up profiles on individuals.
If we're being generous, it's probably just a pleasant side affect. However, I'm yet to see many
and to sell to the NSA etc of course.
Message forgery is a solved problem. SPF works. DKIM is a) overkill and b) unnecesary.
If individual senders need more identity verification than SPF, there are numerous encryption and signing options available....with support built in to many MUAs.
Agreed, PGP/GPG and S/MIME solved this long ago, with the added benefit of end-to-end encryption.
[...] those who refuse to learn from history are doomed to repeat the same damned stupid mistakes. This issue was settled definitively in the 90s.
Well said.
[...]
On Fri, 15 Jan 2016 06:54:22 PM Joel W. Shea via luv-main wrote:
Agreed, PGP/GPG and S/MIME solved this long ago, with the added benefit of end-to-end encryption.
Is it possible to put a PGP/GPG public key in the DNS and have an MUA use it? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Fri, Jan 15, 2016 at 07:45:30PM +1100, Russell Coker via luv-main wrote:
On Fri, 15 Jan 2016 06:54:22 PM Joel W. Shea via luv-main wrote:
Agreed, PGP/GPG and S/MIME solved this long ago, with the added benefit of end-to-end encryption.
Is it possible to put a PGP/GPG public key in the DNS and have an MUA use it?
Since you've asked; anything is possible[1], and since you're already aware of existing mechanisms (key servers) for distributing public PGP/GPG keys; I see what you're getting at, the verification of which is up to the end-user rather than a independent "trusted" third party, or by exhibiting the control of a domain via publishing in the DNS. [1] Although it's entirely possible, I'm not aware of any implementation and without thinking it through more thoroughly I'm unsure why one would want to.
On Fri, 15 Jan 2016 08:26:13 PM Joel W. Shea via luv-main wrote:
Is it possible to put a PGP/GPG public key in the DNS and have an MUA use it?
Since you've asked; anything is possible[1], and since you're already aware of existing mechanisms (key servers) for distributing public PGP/GPG keys; I see what you're getting at, the verification of which is up to the end-user rather than a independent "trusted" third party, or by exhibiting the control of a domain via publishing in the DNS.
[1] Although it's entirely possible, I'm not aware of any implementation and without thinking it through more thoroughly I'm unsure why one would want to.
With DKIM/DMARC the receiving MTA can do the checks and ensure that the mail is valid. If we wanted to use GPG/PGP in the same way then there needs to be an equivalent key management system. On Fri, 15 Jan 2016 08:35:48 PM Brian May via luv-main wrote:
Imagine if the big email providers, such as Google and Yahoo supported this sort of technology. They could push for the required client side encryption standards for browsers if they really wanted to. Suddenly encryption would be available to the masses.
I don't think this is going to happen however, those flying pigs would get in the way.
Or more seriously, I suspect Google don't want people using client side encryption, as this would prevent them scanning emails for advertising.
I don't think that Google and Yahoo have some nefarious plan. If there was a system for encrypting mail between MTAs (like using GPG with DKIM/DMARC type mechanisms for managing keys) then they could display targeted adverts and users get the simplicity of having things just work (GPG is too hard for most people). But GPG in the MUA is difficult for users and help desks. I don't use it as much as I might because the KDE implementation is annoyingly difficult. Using DKIM on my server is a much easier way of signing all my mail. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Russell Coker via luv-main <luv-main@luv.asn.au> writes:
[...] If there was a system for encrypting mail between MTAs (like using GPG with DKIM/DMARC type mechanisms for managing keys) then they could display targeted adverts and users get the simplicity of having things just work (GPG is too hard for most people).
That already exists - SMTP over SSL - while good, it doesn't solve any of the problems we are talking about here. -- Brian May <brian@linuxpenguins.xyz> https://linuxpenguins.xyz/brian/
On Fri, 15 Jan 2016 09:52:35 PM Brian May via luv-main wrote:
[...] If there was a system for encrypting mail between MTAs (like using GPG with DKIM/DMARC type mechanisms for managing keys) then they could display targeted adverts and users get the simplicity of having things just work (GPG is too hard for most people).
That already exists - SMTP over SSL - while good, it doesn't solve any of the problems we are talking about here.
Is there any way to publish the fact that I would prefer to receive mail via SSL? My understanding is that if a hostile party proxies the SMTP session such that the sender thinks that my MTA is incapable of SSL then the sender will happily send it unencrypted. Is there any way for a MUA to specify that a message should be bounced if it can't be sent between MTAs via SSL? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
Russell Coker <russell@coker.com.au> writes:
Is there any way to publish the fact that I would prefer to receive mail via SSL?
Not that I know of. I think you would have to configure the sending server to require it for the destination server. Which really isn't what you wanted.
My understanding is that if a hostile party proxies the SMTP session such that the sender thinks that my MTA is incapable of SSL then the sender will happily send it unencrypted.
Yes, I believe you could be correct here. The sending server has no secure way of knowing that the remote server supports SSL. Ideally this could be published in DNS, I am not aware of any standards to do this however. I haven't looked either. -- Brian May <brian@linuxpenguins.xyz> https://linuxpenguins.xyz/brian/
On Fri, Jan 15, 2016 at 09:14:09PM +1100, Russell Coker via luv-main wrote:
With DKIM/DMARC the receiving MTA can do the checks and ensure that the mail is valid. If we wanted to use GPG/PGP in the same way then there needs to be an equivalent key management system.
I think this would be akin to maintaining a keyring for mailing lists, which isn't actually a bad idea at all. As for the general case, it's in the too hard basket, as you mentioned below.
[...] But GPG in the MUA is difficult for users and help desks. [...]
Russell Coker via luv-main <luv-main@luv.asn.au> writes:
On Fri, 15 Jan 2016 06:54:22 PM Joel W. Shea via luv-main wrote:
Agreed, PGP/GPG and S/MIME solved this long ago, with the added benefit of end-to-end encryption.
Is it possible to put a PGP/GPG public key in the DNS and have an MUA use it?
Top google search for "GPG DNS" gives me this http://www.gushi.org/make-dns-cert/HOWTO.html Suspect nothing has been implemented in MUAs however. Also the date on that page is 2010... -- Brian May <brian@linuxpenguins.xyz> https://linuxpenguins.xyz/brian/
"Joel W. Shea via luv-main" <luv-main@luv.asn.au> writes:
Agreed, PGP/GPG and S/MIME solved this long ago, with the added benefit of end-to-end encryption.
Imagine if the big email providers, such as Google and Yahoo supported this sort of technology. They could push for the required client side encryption standards for browsers if they really wanted to. Suddenly encryption would be available to the masses. I don't think this is going to happen however, those flying pigs would get in the way. Or more seriously, I suspect Google don't want people using client side encryption, as this would prevent them scanning emails for advertising. -- Brian May <brian@linuxpenguins.xyz> https://linuxpenguins.xyz/brian/
On Fri, 15 Jan 2016 04:54:22 PM Craig Sanders via luv-main wrote:
DKIM is an ill-conceived abomination. It actually cares about the *headers* in a message rather than the **envelope*. To an MTA, headers are irrelevant, they're just comments....what matters is the envelope sender address and the envelope recipient address.
And worse, DKIM cares about the From: header rather than the Sender: header.
It cares about what the user sees. The purpose of DKIM is not to ensure that only Paypal can have an envelope sender saying paypal.com, it's purpose is to ensure that only Paypal can have a From: field with paypal.com.
This is just broken in every possible way.
If only people who developed SMTP in the early days had thought of spam and phishing.
Yes. I'd appreciate it if people would stop acting like I'm doing something I want to do here. I just want mail to go through reliably and I'm doing what is necessary to achieve that goal.
unfortunately, you stil haven't impleemnted the minimum-damage option that only munges posts that are sent by users from domains that implement DKIM (like google or yahoo), and leaves other mail alone.
I don't like the idea of having different handling methods for different messages. We have already had one user complain about this even though we aren't doing it!
it's not like anyone ever posts from those domains to our lists anyway.
We have Wen, Joel, Tony, Tim, Trent and others posting from Gmail. We have Lev, me, and others DKIM signing their own mail.
which is one of the more annoying things about this issue - the configuration messes things up for active participants on the list, and it doesn't even provide any benefit to the lurkers who never say or contribute anything.
Apart from the ones who receive mail viw Gmail, the ones who complained about my mail going to their spam folders which started me working on this.
That's entirely the wrong thing to do. those who contribute may well stop bothering if they get annoyed enough, and non-contributors won't step forward to replace them...if they were inclined to, they'd already be posting.
driving away those who write the posts (that both they and the lurkers read) is self-defeating.
But list traffic is significantly greater than usual at the moment.
Widespread use of DMARC will result in changes to well established conventions.
IMO it's an attempt by major corporate players to completely take over email so that no email is ever sent that they don't get a copy of to examine and index and use to build up profiles on individuals.
What makes you think that? When I send DKIM signed mail to you it's between my mail server, yours, and my DNS server.
Message forgery is a solved problem. SPF works. DKIM is a) overkill and b) unnecesary.
SPF doesn't stop forged headers unless you use DMARC.
I don't personally object to having the list server rewrite the "From" field and add a "Reply-to" header that designates the original sender; but some people have needs which differ from mine, and for them it can be an inconvenience.
NO!
those who refuse to learn from history are doomed to repeat the same damned stupid mistakes. This issue was settled definitively in the 90s.
Mailing lists should *never*, under any circumstances, mess with the Reply-To header. That belongs solely to the original sender.
Lists have several alternatives they can use, including Mail-Followup-To: and List-Post:
and Lists shouldn't mess with the From: header, either. No matter what corporate vermin demand. WGAF what facebook wants? how many emails from luv lists ever go to facebook?
The mail servers that people use to send mail to this list are also used by people who want to send mail to Facebook. Even if every single LUV member avoided ever using Facebook then we would still be affected by what they want. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Fri, Jan 15, 2016 at 07:01:30PM +1100, Russell Coker via luv-main wrote:
On Fri, 15 Jan 2016 04:54:22 PM Craig Sanders via luv-main wrote:
I don't like the idea of having different handling methods for different messages. We have already had one user complain about this even though we aren't doing it!
Technically, you are ... by munging the from field on all messages. To wit, the difference in handling becomes very obvious when only receiving the message directly, presumably because the recipient address is also in the "To:" or the "Cc:" field and the list then may (or may not) send a duplicate, exhibiting the different handling method.
[...] Apart from the ones who receive mail viw Gmail, the ones who complained about my mail going to their spam folders which started me working on this.
Since Mailman was breaking the signature by re-folding the headers, this could have been resolved in one of many ways; including signing with a relaxed header canonicalization, or by having the list server strip the DKIM signature entirely (having been verified by the MTA already). There may (or may not) have been discussion amongst the committee members about possible modifications to the list, but there doesn't appear to be any transparency regarding this, and there certainly wasn't any requests for consultation (commentary/suggestions) from ordinary list members *before* making a seemingly unilateral change.
On Fri, 15 Jan 2016 08:09:29 PM Joel W. Shea via luv-main wrote:
I don't like the idea of having different handling methods for different messages. We have already had one user complain about this even though we aren't doing it!
Technically, you are ... by munging the from field on all messages.
To wit, the difference in handling becomes very obvious when only receiving the message directly, presumably because the recipient address is also in the "To:" or the "Cc:" field and the list then may (or may not) send a duplicate, exhibiting the different handling method.
It is true that when someone receives 2 copies of the message (from the list and from a CC) they get 2 different versions. I'm not totally opposed to using the DMARC setting.
[...] Apart from the ones who receive mail viw Gmail, the ones who complained about my mail going to their spam folders which started me working on this.
Since Mailman was breaking the signature by re-folding the headers, this could have been resolved in one of many ways; including signing with a relaxed header canonicalization, or by having the list server strip the DKIM signature entirely (having been verified by the MTA already).
Mailman is also reencoding my messages as base64 for some reason. It's not doing it for everyone and other instances of Mailman don't so we can probably solve it. But this still won't solve the issues with DMARC.
There may (or may not) have been discussion amongst the committee members about possible modifications to the list, but there doesn't appear to be any transparency regarding this, and there certainly wasn't any requests for consultation (commentary/suggestions) from ordinary list members *before* making a seemingly unilateral change.
There wasn't a discussion because no-one on the committee felt the need for a discussion. I suggested that we have one but it seems that no-one had any problems with it. I enabled it on the committee list before luv-talk and everyone there seemed happy. For comparison the committee discussion about moving the LUV server (probably the most important issue the committee has discussed in the past year or so) was held on Google Hangouts and we only got 5/9 in attendance. The history of LUV shows that asking for commentary from members before making a change is something you only do if you don't want to make a change. It took us years to get public archives for the LUV lists with many discussions like this along the way. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Fri, Jan 15, 2016 at 08:34:35PM +1100, Russell Coker wrote:
Mailman is also reencoding my messages as base64 for some reason. It's not doing it for everyone and other instances of Mailman don't so we can probably solve it.
I vaguely recall something in the DKIM spec where a certain setting requires base64 encoding, but I don't remember what that was.
But this still won't solve the issues with DMARC.
Probably not :-(
On Fri, Jan 15, 2016 at 08:34:35PM +1100, Russell Coker wrote:
On Fri, 15 Jan 2016 08:09:29 PM Joel W. Shea via luv-main wrote:
There may (or may not) have been discussion amongst the committee [...]
There wasn't a discussion because no-one on the committee felt the need for a discussion. I suggested that we have one but it seems that no-one had any problems with it. I enabled it on the committee list before luv-talk and everyone there seemed happy.
I guess that works in-lieu of a discussion. Thanks for the explaination, I appreciate that.
For comparison the committee discussion about moving the LUV server (probably the most important issue the committee has discussed in the past year or so) was held on Google Hangouts and we only got 5/9 in attendance.
Are there any minutes taken/published for committee meetings? Are they even necessary? Other committee members feel free to chime in here.
The history of LUV shows that asking for commentary from members before making a change is something you only do if you don't want to make a change.
Ha ha, fair enough, I can sympathise with that :-)
It took us years to get public archives for the LUV lists with many discussions like this along the way.
Your memory on that issue is obviously much better than mine, thanks for the context.
On 15/01/16 20:51, Joel W. Shea via luv-main wrote:
Are there any minutes taken/published for committee meetings? Are they even necessary? Other committee members feel free to chime in here.
Yes, minutes are normally taken by the Secretary and published to the committee mailing list. Cheers, Andrew
On Fri, 15 Jan 2016 08:51:19 PM Joel W. Shea via luv-main wrote:
Are there any minutes taken/published for committee meetings?
It's a legal requirement of the act that governs associations in Victoria for there to be minutes of committee meetings. Members have a right to request to see such minutes but the committee has the right to refuse or excise parts for various listed reasons. All the best, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
On Fri, Jan 15, 2016 at 07:01:30PM +1100, Russell Coker wrote:
On Fri, 15 Jan 2016 04:54:22 PM Craig Sanders via luv-main wrote:
DKIM is an ill-conceived abomination. It actually cares about the *headers* in a message rather than the **envelope*. To an MTA, headers are irrelevant, they're just comments....what matters is the envelope sender address and the envelope recipient address.
And worse, DKIM cares about the From: header rather than the Sender: header.
It cares about what the user sees. The purpose of DKIM is not to ensure that only Paypal can have an envelope sender saying paypal.com, it's purpose is to ensure that only Paypal can have a From: field with paypal.com.
and that's the problem. after the message has been sent by the originating MUA or MTA (where To, From, CC, Bcc are used to construct the envelope sender & recipient), headers are merely comments. To treat them as anything different is just plain wrong. and restricting @paypal.com From: headers to just paypal-owned sender host is pointless anyway - phishers register every possible variant and look-alike of paypal.com and spam with that. often they don't even bother using a domain that looks or sounds even remotely like paypal.com - and it makes no difference. their typical victims are too stupid to notice or care - and it's not just a matter of ignorance, it's stupidity. phishing on the net has been around for at least a decade and a half now, it's not possible for an internet user over the age of about 5 to not know about it and realise they should be wary of unsolicitied emails asking them to click on a link or login to reactivate their account or check out this amazing multi-million dollar business deal or charity gift etc.
This is just broken in every possible way.
If only people who developed SMTP in the early days had thought of spam and phishing.
spam is horrible, something must be done. DMARC is something, so we must do DMARC. btw, like SPF, DKIM and DMARC are not anti-spam tools. they're anti-forgery tools. there's a huge difference. in the early days of SPF many people complained that it didn't do much to block spam (because spammers could forge domains without SPF records or just register and add SPF records to their own domains), and just plain refused to hear it when told that spam-blocking was not SPF's purpose. and if the developers of SMTP had thought of spam etc, and designed SMTP to have built-in authentication, it would have seriously damaged the open nature of the internet as we know it today. the fact that the net was built open and not locked down was a major contributor to its success.
unfortunately, you stil haven't impleemnted the minimum-damage option that only munges posts that are sent by users from domains that implement DKIM (like google or yahoo), and leaves other mail alone.
I don't like the idea of having different handling methods for different messages. We have already had one user complain about this even though we aren't doing it!
Tony's complaints have nothing to do with the list. when he's CC-ed on list mail, sometimes he gets the message from the list first, and sometimes he gets the directly CC-ed copy first. The list has nothing to do with that, and can not affect it in the slightest. it's not a problem you can solve or do anything about, so please don't use it as an excuse to DMARC-mangle every post.
it's not like anyone ever posts from those domains to our lists anyway.
We have Wen, Joel, Tony, Tim, Trent and others posting from Gmail. We have Lev, me, and others DKIM signing their own mail.
which is one of the more annoying things about this issue - the configuration messes things up for active participants on the list, and it doesn't even provide any benefit to the lurkers who never say or contribute anything.
Apart from the ones who receive mail viw Gmail, the ones who complained about my mail going to their spam folders which started me working on this.
if mailman is breaking DKIM-signed message then that needs to be fixed. mangling headers is a crappy workaround hack, not a fix.
But list traffic is significantly greater than usual at the moment.
a lot of that is this thread complaining about DMARC.
Message forgery is a solved problem. SPF works. DKIM is a) overkill and b) unnecesary.
SPF doesn't stop forged headers unless you use DMARC.
headers are irrelevant. you can't and shouldn't trust From: headers any more than you can trust any Received: header before the ones your own server adds.
The mail servers that people use to send mail to this list are also used by people who want to send mail to Facebook. Even if every single LUV member avoided ever using Facebook then we would still be affected by what they want.
as i said, solve the right (actual!) problem. if mailman's handling of DKIM-signed messages is broken then THAT is the problem that needs to be fixed. craig -- craig sanders <cas@taz.net.au>
On Sat, 16 Jan 2016 08:36:30 AM Craig Sanders via luv-main wrote:
It cares about what the user sees. The purpose of DKIM is not to ensure that only Paypal can have an envelope sender saying paypal.com, it's purpose is to ensure that only Paypal can have a From: field with paypal.com.
and that's the problem. after the message has been sent by the originating MUA or MTA (where To, From, CC, Bcc are used to construct the envelope sender & recipient), headers are merely comments. To treat them as anything different is just plain wrong.
Why are we having an argument about comments then? If they are just comments then it shouldn't be a big deal.
and restricting @paypal.com From: headers to just paypal-owned sender host is pointless anyway - phishers register every possible variant and look-alike of paypal.com and spam with that. often they don't even bother using a domain that looks or sounds even remotely like paypal.com - and it makes no difference. their typical victims are too stupid to notice or care - and it's not just a matter of ignorance, it's stupidity.
Stupidity is a problem. But what we want to do here is to make it possible for people who aren't particularly stupid to work this out without much effort.
and if the developers of SMTP had thought of spam etc, and designed SMTP to have built-in authentication, it would have seriously damaged the open nature of the internet as we know it today. the fact that the net was built open and not locked down was a major contributor to its success.
They could have designed encryption and signing features from the start and methods for recognising new senders.
Apart from the ones who receive mail viw Gmail, the ones who complained about my mail going to their spam folders which started me working on this.
if mailman is breaking DKIM-signed message then that needs to be fixed. mangling headers is a crappy workaround hack, not a fix.
Fine, Tell us how to fix it without mangling headers.
But list traffic is significantly greater than usual at the moment.
a lot of that is this thread complaining about DMARC.
I know that!
Message forgery is a solved problem. SPF works. DKIM is a) overkill and b) unnecesary.
SPF doesn't stop forged headers unless you use DMARC.
headers are irrelevant. you can't and shouldn't trust From: headers any more than you can trust any Received: header before the ones your own server adds.
Unless they are DKIM signed.
The mail servers that people use to send mail to this list are also used by people who want to send mail to Facebook. Even if every single LUV member avoided ever using Facebook then we would still be affected by what they want.
as i said, solve the right (actual!) problem. if mailman's handling of DKIM-signed messages is broken then THAT is the problem that needs to be fixed.
OK. Let's fix that. I don't have the time or skill to fix Mailman code, could you please do it for me? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Sat, Jan 16, 2016 at 12:31:47PM +1100, Russell Coker wrote:
On Sat, 16 Jan 2016 08:36:30 AM Craig Sanders via luv-main wrote: Why are we having an argument about comments then? If they are just comments then it shouldn't be a big deal.
because munging them screws up an MUAs ability to reply correctly. because overwriting the Reply-To: header can make it impossible to reply to the original sender. because what mailman is doing with DMARC is broken.
Stupidity is a problem. But what we want to do here is to make it possible for people who aren't particularly stupid to work this out without much effort.
they can. It's not hard: 1. Don't click on links in email. 2. Don't trust that the message actually came from whoever it claims to have come from unless it's signed and/or encrypted with a key you (your system) knows, and even then be wary (keys can't protect you when the sender is coerced to give up their private key) that's it. the rest is implementation details of those two things.
They could have designed encryption and signing features from the start and methods for recognising new senders.
those things are MUA functions (i.e. message payload), not MTA - i.e. irrelevant to the SMTP protocol.
Apart from the ones who receive mail viw Gmail, the ones who complained about my mail going to their spam folders which started me working on this.
if mailman is breaking DKIM-signed message then that needs to be fixed. mangling headers is a crappy workaround hack, not a fix.
Fine, Tell us how to fix it without mangling headers.
why me? i'm not the one who wants to change anything. you've changed things and caused breakage. a better idea would be for you to fix it instead of deciding that mangling headers is a reasonable thing to do. i.e. you want it, you fix it. also: a) i'm not a python programmer b) i disagree with the concept of DMARC, so i have no desire to implement it or fix a particularly broken implementation of it.
as i said, solve the right (actual!) problem. if mailman's handling of DKIM-signed messages is broken then THAT is the problem that needs to be fixed.
OK. Let's fix that. I don't have the time or skill to fix Mailman code, could you please do it for me?
translation: i've enabled a broken feature and rather than revert that change, i'll badger someone else to fix the problem for me. you enabled it, it's up to you to either fix it or disable it again. craig -- craig sanders <cas@taz.net.au>
On Sat, 16 Jan 2016 02:11:32 AM Craig Sanders via luv-main wrote:
On Sat, Jan 16, 2016 at 12:31:47PM +1100, Russell Coker wrote:
On Sat, 16 Jan 2016 08:36:30 AM Craig Sanders via luv-main wrote: Why are we having an argument about comments then? If they are just comments then it shouldn't be a big deal.
because munging them screws up an MUAs ability to reply correctly.
That means that they aren't comments.
because overwriting the Reply-To: header can make it impossible to reply to the original sender.
I'm up for negotiating on that matter.
because what mailman is doing with DMARC is broken.
It's doing the best it can in a difficult situation.
if mailman is breaking DKIM-signed message then that needs to be fixed. mangling headers is a crappy workaround hack, not a fix.
Fine, Tell us how to fix it without mangling headers.
why me? i'm not the one who wants to change anything. you've changed things and caused breakage.
a better idea would be for you to fix it instead of deciding that mangling headers is a reasonable thing to do.
I fixed it, mail is now being delivered reliably. If you want "comments" to be different in email then you can help figure out how to do that without compromising mail reliability.
i.e. you want it, you fix it.
Yes, you do that.
a) i'm not a python programmer
Same here.
b) i disagree with the concept of DMARC, so i have no desire to implement it or fix a particularly broken implementation of it.
I have things working quite well so I have little incentive to change Mailman code.
OK. Let's fix that. I don't have the time or skill to fix Mailman code, could you please do it for me?
translation: i've enabled a broken feature and rather than revert that change, i'll badger someone else to fix the problem for me. you enabled it, it's up to you to either fix it or disable it again.
Translation: You think that Yahoogroups, the Mailman developers and everyone else have got it wrong but can't be bothered showing us all how to do it in a way that you like.
On Sat, Jan 16, 2016 at 04:05:47AM +0000, Russell Coker wrote:
On Sat, 16 Jan 2016 02:11:32 AM Craig Sanders via luv-main wrote:
On Sat, Jan 16, 2016 at 12:31:47PM +1100, Russell Coker wrote:
On Sat, 16 Jan 2016 08:36:30 AM Craig Sanders via luv-main wrote: Why are we having an argument about comments then? If they are just comments then it shouldn't be a big deal.
because munging them screws up an MUAs ability to reply correctly.
That means that they aren't comments.
as far as an MTA is concerned, they're comments. rejecting mail based on the From: header is fundamentally broken. Lists mangling the From: header to prevent broken MTAs from rejecting the mail is a) pandering to brokenness and b) broken in itself. MUAs make use of various message headers (but even then it's still a mistake to think of them as addressing info - MTAs are responsible for delivery, so what they define as address info is what is relevant in that context). MTAs don't and shouldn't.
because overwriting the Reply-To: header can make it impossible to reply to the original sender.
I'm up for negotiating on that matter.
what's to negotiate? you're either going to fix the damage you caused or you're not. i'm not going to beg you to do what. you're the list admin, if you insist on breaking the list, there's nothing i can do about it.
because what mailman is doing with DMARC is broken.
It's doing the best it can in a difficult situation.
even if you can manage to think of that as being only partly broken, partly broken is still broken. it's doing the wrong thing with DMARC. that means it's a "feature" that shouldn't be used.
I fixed it, mail is now being delivered reliably. If you want "comments" to be different in email then you can help figure out how to do that without compromising mail reliability.
you haven't fixed anything. you've solved one minor problem that affected only a handful of people (who inflicted it on themselves by using DKIM) at the cost of screwing up the From: and Reply-To: headers on all messages forwarded by the list, which affects all subscribers.
I have things working quite well so I have little incentive to change Mailman code.
no, you don't have things working well. you have broken the From: and Reply-To: headers.
Translation: You think that Yahoogroups, the Mailman developers and everyone else have got it wrong
the fact that yahoogroups does it is undeniable proof that it's wrong. they never get anything right, they've been screwing up mail since the 90s. the mailman devs have obviously got it wrong. even they admit that their implementation is buggy and broken.
but can't be bothered showing us all how to do it in a way that you like.
you're assuming that there is such a way. IMO DMARC is broken-by-design so it's impossible to do it in any good or even reasonable way. in other words: the way i'd like is to not do it at all. i've said that repeatedly. craig -- craig sanders <cas@taz.net.au>
On Sat, 16 Jan 2016 05:00:58 PM Craig Sanders via luv-main wrote:
because overwriting the Reply-To: header can make it impossible to reply to the original sender.
I'm up for negotiating on that matter.
what's to negotiate? you're either going to fix the damage you caused or you're not. i'm not going to beg you to do what. you're the list admin, if you insist on breaking the list, there's nothing i can do about it.
If you believed that there was nothing you could do then you wouldn't have spent months arguing.
because what mailman is doing with DMARC is broken.
It's doing the best it can in a difficult situation.
even if you can manage to think of that as being only partly broken, partly broken is still broken. it's doing the wrong thing with DMARC. that means it's a "feature" that shouldn't be used.
OK, please go tell Yahoo etc that they are doing it wrong. Let's leave the list server in it's current configuration until you convince Yahoo and Gmail of the error of their ways.
I fixed it, mail is now being delivered reliably. If you want "comments" to be different in email then you can help figure out how to do that without compromising mail reliability.
you haven't fixed anything. you've solved one minor problem that affected only a handful of people (who inflicted it on themselves by using DKIM) at the cost of screwing up the From: and Reply-To: headers on all messages forwarded by the list, which affects all subscribers.
Actually the Gmail users didn't do anything, they just signed up for a mail service knowing nothing about DKIM or the Gmail actions that would happen when they received DKIM signed mail via a list.
Translation: You think that Yahoogroups, the Mailman developers and everyone else have got it wrong
the fact that yahoogroups does it is undeniable proof that it's wrong. they never get anything right, they've been screwing up mail since the 90s.
If they were so wrong then Gmail wouldn't implement DKIM/DMARC checks on mail it receives.
the mailman devs have obviously got it wrong. even they admit that their implementation is buggy and broken.
What is buggy is the fact that they can't preserve headers and body encoding.
but can't be bothered showing us all how to do it in a way that you like.
you're assuming that there is such a way. IMO DMARC is broken-by-design so it's impossible to do it in any good or even reasonable way.
in other words: the way i'd like is to not do it at all. i've said that repeatedly.
It's OK to wish that Yahoo, Gmail, Hotmail, Facebook, and other big companies would do things differently. But your wishes aren't going to change anything. Let's stick to discussing the reality of how to deal with mail to/from such services. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Sat, Jan 16, 2016 at 05:08:23PM +1100, Russell Coker wrote:
OK, please go tell Yahoo etc that they are doing it wrong. Let's leave the list server in it's current configuration until you convince Yahoo and Gmail of the error of their ways.
I would personally join the DMARC working group, if I thought I could make any difference; but judging from the RFC, I can infer that my only proposed suggestions have already been considered and placed out of scope.
On Sat, Jan 16, 2016 at 05:08:23PM +1100, Russell Coker wrote:
On Sat, 16 Jan 2016 05:00:58 PM Craig Sanders via luv-main wrote: If you believed that there was nothing you could do then you wouldn't have spent months arguing.
i haven't. i've mostly ignored it for most of the last few months since you broke the list and only got sucked into it in the last few days because the thread's been going on for about a week. and my procmail rule partially unbreaks the list (except for the munged Reply-To: header which 1. is unfixable and 2. isn't used much by subscribers here AFAIK)
OK, please go tell Yahoo etc that they are doing it wrong. Let's leave the list server in it's current configuration until you convince Yahoo and Gmail of the error of their ways.
if you intend to do nothing, then just say so - don't make up some bullshit impossible goal that "might" get you to unbreak the list.
Actually the Gmail users didn't do anything, they just signed up for a mail service knowing nothing about DKIM or the Gmail actions that would happen when they received DKIM signed mail via a list.
it's really difficult to have much sympathy for the technical problems experienced by people who chose to use a spyware service run by a giant corporation. They CHOSE to have no control over their mail, they get to just suck it up and accept whatever problems that causes. They don't get to assume that their abdication of responsibility for their own email automatically entitle them to cause problems for others.
If they were so wrong then Gmail wouldn't implement DKIM/DMARC checks on mail it receives.
google has their own reasons for destroying independently run mailing lists.
the mailman devs have obviously got it wrong. even they admit that their implementation is buggy and broken.
What is buggy is the fact that they can't preserve headers and body encoding.
there's also the fact that mailman strips attachments and makes other changes to the message body which is going to invalidate any signing of the original msg. Look, the major problem with DMARC is that it uses the From: header. If it used the Sender: header instead (or used it IF it exists in the headers), then mailman could do the right thing by stripping attachments, changing the encoding, or whatever and declare that the list was the Sender: and DKIM sign it.
you're assuming that there is such a way. IMO DMARC is broken-by-design so it's impossible to do it in any good or even reasonable way.
in other words: the way i'd like is to not do it at all. i've said that repeatedly.
It's OK to wish that Yahoo, Gmail, Hotmail, Facebook, and other big companies would do things differently. But your wishes aren't going to change anything. Let's stick to discussing the reality of how to deal with mail to/from such services.
i didn't wish they did things differently, at least that's not the argument i'm making here. my attitude is that their unreasonable demands about changing the nature of email and mailing lists should be ignored, not surrendered/pandered to. craig -- craig sanders <cas@taz.net.au>
On Sat, 16 Jan 2016 09:01:11 PM Craig Sanders via luv-main wrote:
OK, please go tell Yahoo etc that they are doing it wrong. Let's leave the list server in it's current configuration until you convince Yahoo and Gmail of the error of their ways.
if you intend to do nothing, then just say so - don't make up some bullshit impossible goal that "might" get you to unbreak the list.
I have been very clear from the start that I intend to have the lists work reliably for delivering mail from all users (including Yahoo) and too all users (including Yahoo and Gmail). I have no plans to change that. I am entirely serious that if DMARC was to go away I would remove the DMARC specific setting.
Actually the Gmail users didn't do anything, they just signed up for a mail service knowing nothing about DKIM or the Gmail actions that would happen when they received DKIM signed mail via a list.
it's really difficult to have much sympathy for the technical problems experienced by people who chose to use a spyware service run by a giant corporation. They CHOSE to have no control over their mail, they get to just suck it up and accept whatever problems that causes. They don't get to assume that their abdication of responsibility for their own email automatically entitle them to cause problems for others.
So this is really a debate about who's the most elite under the guise of discussing list headers?
the mailman devs have obviously got it wrong. even they admit that their implementation is buggy and broken.
What is buggy is the fact that they can't preserve headers and body encoding.
there's also the fact that mailman strips attachments and makes other changes to the message body which is going to invalidate any signing of the original msg.
Yes. But I'm not aware of other list software that does what we want and solves these problems.
Look, the major problem with DMARC is that it uses the From: header.
If it used the Sender: header instead (or used it IF it exists in the headers), then mailman could do the right thing by stripping attachments, changing the encoding, or whatever and declare that the list was the Sender: and DKIM sign it.
Well that wasn't what DMARC was designed to do.
It's OK to wish that Yahoo, Gmail, Hotmail, Facebook, and other big companies would do things differently. But your wishes aren't going to change anything. Let's stick to discussing the reality of how to deal with mail to/from such services.
i didn't wish they did things differently, at least that's not the argument i'm making here. my attitude is that their unreasonable demands about changing the nature of email and mailing lists should be ignored, not surrendered/pandered to.
We have to either make the list work with mail to/from such services or exclude them from the lists. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Sat, Jan 16, 2016 at 09:32:14PM +1100, Russell Coker wrote:
Actually the Gmail users didn't do anything, they just signed up for a mail service knowing nothing about DKIM or the Gmail actions that would happen when they received DKIM signed mail via a list.
it's really difficult to have much sympathy for the technical problems experienced by people who chose to use a spyware service run by a giant corporation. They CHOSE to have no control over their mail, they get to just suck it up and accept whatever problems that causes. They don't get to assume that their abdication of responsibility for their own email automatically entitle them to cause problems for others.
So this is really a debate about who's the most elite under the guise of discussing list headers?
WTF has "elite" got to do with it? if you choose to use a crappy service, you get a crappy service. same as if you choose to eat dogfood out of a can or, worse, McDonalds, you get to eat crappy pseudo-food. choosing crap means you get crap. that's just one of the unalterable facts about how the universe works. you can't choose crap things and magically get good things.
Look, the major problem with DMARC is that it uses the From: header.
If it used the Sender: header instead (or used it IF it exists in the headers), then mailman could do the right thing by stripping attachments, changing the encoding, or whatever and declare that the list was the Sender: and DKIM sign it.
Well that wasn't what DMARC was designed to do.
DMARC's design is broken.
We have to either make the list work with mail to/from such services or exclude them from the lists.
so? how is that different to any other site or individual who deliberately (or incompetently) breaks the expected standards? are google and facebook and yahoo etc allowed to redefine standards just by breaking them and demanding that everyone else follows their new way? who else is allowed to do that? do you have to be a mega-corporation with billions of dollars or is that priviledge available to us peasants too? craig -- craig sanders <cas@taz.net.au> BOFH excuse #320: You've been infected by the Telescoping Hubble virus.
On 16/01/16 21:58, Craig Sanders via luv-main wrote:
how is that different to any other site or individual who deliberately (or incompetently) breaks the expected standards? are google and facebook and yahoo etc allowed to redefine standards just by breaking them and demanding that everyone else follows their new way? who else is allowed to do that? do you have to be a mega-corporation with billions of dollars or is that priviledge available to us peasants too?
Unfortunately due to the network effect you have to live with the defacto standards of whatever is actually implemented in practice, however badly it breaks the previously agreed upon standards. See also: NAT, firewalls, web browsers, video drivers, UEFI, etc. etc. It's a world of suck, and we can complain about the golden rule (those with the gold make the rules) and point out how much better the world would be if that wasn't the case, but ultimately we do users a disservice if we prevent things from working just because we object to the unethical behaviour of the other players. I'm all for loudly shaming organisations who break things, but I don't think it helps users to refuse to interoperate with them. Should we insist that LUV members boycott Gmail, Yahoo, Facebook etc. unless and until they fix these email issues? I don't think many of our members would thank us. Cheers, Andrew
On 16.01.16 22:24, Andrew Pam via luv-main wrote:
I'm all for loudly shaming organisations who break things, but I don't think it helps users to refuse to interoperate with them. Should we insist that LUV members boycott Gmail, Yahoo, Facebook etc. unless and until they fix these email issues? I don't think many of our members would thank us.
If we do not, then we define our list as a subservice of the capturing hegemon. Erik
On 16.01.16 21:01, Craig Sanders via luv-main wrote:
On Sat, Jan 16, 2016 at 05:08:23PM +1100, Russell Coker wrote:
Actually the Gmail users didn't do anything, they just signed up for a mail service knowing nothing about DKIM or the Gmail actions that would happen when they received DKIM signed mail via a list.
it's really difficult to have much sympathy for the technical problems experienced by people who chose to use a spyware service run by a giant corporation. They CHOSE to have no control over their mail, they get to just suck it up and accept whatever problems that causes. They don't get to assume that their abdication of responsibility for their own email automatically entitle them to cause problems for others.
Corporations cannot be allowed to deny the freedom of internet traffic. If their DSHIT needs a header, then let them tack one on. It is both in principle and in practice unacceptable to allow them to hijack existing headers, particularly as it is to the practical detriment of list users. If a minority of list members have made the mistake of using gmail, then I _do_ have sympathy - sufficient to cordially recommend them moving to a service which does not set out to break independent mailing lists. Out list is _not_ primarily a subservice of gmail ... well, until recently, it wasn't.
If they were so wrong then Gmail wouldn't implement DKIM/DMARC checks on mail it receives.
google has their own reasons for destroying independently run mailing lists.
+1 ...
It's OK to wish that Yahoo, Gmail, Hotmail, Facebook, and other big companies would do things differently. But your wishes aren't going to change anything. Let's stick to discussing the reality of how to deal with mail to/from such services.
i didn't wish they did things differently, at least that's not the argument i'm making here. my attitude is that their unreasonable demands about changing the nature of email and mailing lists should be ignored, not surrendered/pandered to.
Foolishly surrendering to one leads to surrendering to the others in turn, as they work to copy gmail's hijack strategy. It is much better to restore the status quo ante, and let gmail suffer loss of users as a result of their sabotage of mailing lists. Erik
On Sat, Jan 16, 2016 at 12:31:47PM +1100, Russell Coker wrote:
[...] headers are merely comments. To treat them as anything different is just plain wrong.
Why are we having an argument about comments then? If they are just comments then it shouldn't be a big deal.
Unfortunately the RFC5322.From field is one of the few comments that MUST be present, which most MUA depend on to make the (claimed) author visible to the user. It's disappointing that MUA weren't expected to highlight authenticated authors instead (which they can already do with PGP and S/MIME), and like we do in the browser world with SSL/TLS authenticated servers. So rather than depend on MUA, the DMARC working group, along with the RFC authors from Yahoo and Google, for better or worse decided to ensure MTA took ownership of that responsibility instead. To make matters worse, they will not entertain any alternatives, as the RFC, and working group, clearly makes alternative fields (incl. "Sender") out of scope
On 15.01.16 19:01, Russell Coker via luv-main wrote:
It cares about what the user sees. The purpose of DKIM is not to ensure that only Paypal can have an envelope sender saying paypal.com, it's purpose is to ensure that only Paypal can have a From: field with paypal.com.
If that is the goal, do we not evilly abuse it by using it to ensure that list members _cannot_ have a genuine From: field? The reason that mutt's ~(~P) pattern no longer identifies my posts is that the header munge munges them as authored by the list server. Faking the From: field is what spammers do, innit? Erik
On Sat, 16 Jan 2016 08:00:49 PM Erik Christiansen via luv-main wrote:
On 15.01.16 19:01, Russell Coker via luv-main wrote:
It cares about what the user sees. The purpose of DKIM is not to ensure that only Paypal can have an envelope sender saying paypal.com, it's purpose is to ensure that only Paypal can have a From: field with paypal.com.
If that is the goal, do we not evilly abuse it by using it to ensure that list members _cannot_ have a genuine From: field? The reason that mutt's ~(~P) pattern no longer identifies my posts is that the header munge munges them as authored by the list server. Faking the From: field is what spammers do, innit?
The user visible From field is to tell the user where the message came from. In this case it came from a mailing list. The list server verifies the message it receives, processes it, and sends a new message out. One could argue that Mailman should create a new Message-ID when doing so, but it is convenient to have the same Message-ID through the process. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On 16.01.16 20:37, Russell Coker wrote:
On Sat, 16 Jan 2016 08:00:49 PM Erik Christiansen via luv-main wrote:
If that is the goal, do we not evilly abuse it by using it to ensure that list members _cannot_ have a genuine From: field? The reason that mutt's ~(~P) pattern no longer identifies my posts is that the header munge munges them as authored by the list server. Faking the From: field is what spammers do, innit?
The user visible From field is to tell the user where the message came from. In this case it came from a mailing list. The list server verifies the message it receives, processes it, and sends a new message out. One could argue that Mailman should create a new Message-ID when doing so, but it is convenient to have the same Message-ID through the process.
One can argue any kind of nonsense. The need to rationalise an unpopular and broken munge is understandable motivation. I know of no other list which inflicts similar pain on list members - and certainly do not experience such disservice on other lists. It is irrational to pretend that my posts come from the list server - an internet host which is merely one of several servers through which our posts flow. In properly administered servers, the flow is more transparent. If you screwed with the Message-ID too, I think it would be time to unsubscribe. I suggest that you have lost the plot. You have still not informed us why you insist on denying list members a genuine From: field, merely to grant the same to others. (My understanding is that it is because you have not found a workable solution to some other minor issue, so you've created a bigger problem.) Erik
On Sat, 16 Jan 2016 09:20:03 PM Erik Christiansen via luv-main wrote:
You have still not informed us why you insist on denying list members a genuine From: field, merely to grant the same to others. (My understanding is that it is because you have not found a workable solution to some other minor issue, so you've created a bigger problem.)
You are obviously not reading any messages that I write. I am not going to explain myself again. It's time for this discussion to end. Please take all further replies to luv-talk. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On 16.01.16 21:42, Russell Coker wrote:
On Sat, 16 Jan 2016 09:20:03 PM Erik Christiansen via luv-main wrote:
You have still not informed us why you insist on denying list members a genuine From: field, merely to grant the same to others. (My understanding is that it is because you have not found a workable solution to some other minor issue, so you've created a bigger problem.)
You are obviously not reading any messages that I write. I am not going to explain myself again.
It's time for this discussion to end.
Please take all further replies to luv-talk.
No. We will discuss the adequacy of the performance of the list server on this list. Thank you for your participation. Erik
Russell Coker via luv-main <luv-main@luv.asn.au> writes:
On Sat, 16 Jan 2016 09:20:03 PM Erik Christiansen via luv-main wrote: You are obviously not reading any messages that I write. I am not going to explain myself again.
It's time for this discussion to end.
I tend to agree - this discussion seems to be going around in circles. I think Russell Coker has clearly explained his position, and others have clearly stated that they are unhappy with this. Continuing this discussion is unlikely to presuade anybody to change their mind or lead to any other type of positive outcome. ...unless anybody can provide significant arguments that *have* *not* already beed made. I think all relevant issues have been raised however. Possibly the most productive thing would be if LUV had some sort of wiki, and people could write create *proposals* to *fix* the problem (not just a complaint that the current sitation is wrong), with arguments for and against. Oh wait, maybe we do have a wiki. wiki.luv.asn.au. Seems to be very slow however, and possibly broken (pages come up empty - not sure if this is expected or not). -- Brian May <brian@linuxpenguins.xyz> https://linuxpenguins.xyz/brian/
On 17.01.16 11:32, Brian May via luv-main wrote:
Possibly the most productive thing would be if LUV had some sort of wiki, and people could write create *proposals* to *fix* the problem (not just a complaint that the current sitation is wrong), with arguments for and against.
A complete solution, i.e. one not introducing new problems, would have been to munge headers for gmail recipients only. I.e. solve the problem where it exists - don't pollute the list aimlessly. That would also serve to defend an independent community against internet hegemons - a non-trivial achievement in itself, with long-term benefits. I understand that Russel is content with half a solution, and that is why examination of the residual problems "must cease forthwith - so there!" Well, if "nowhere near" is to be "good enough", then so be it. Just please don't pretend it is any more then half a solution. I'll just unmunge the headers on receipt, I think, as Craig is doing, IIRC. Two horrible hacks can make a whole solution, I guess. Erik (Going bush for a week, so you can probably tie a knot in this thread after all. ;)
Brian May via luv-main <luv-main@luv.asn.au> wrote:
Russell Coker via luv-main <luv-main@luv.asn.au> writes:
On Sat, 16 Jan 2016 09:20:03 PM Erik Christiansen via luv-main wrote: You are obviously not reading any messages that I write. I am not going to explain myself again.
It's time for this discussion to end.
I tend to agree - this discussion seems to be going around in circles.
I think Russell Coker has clearly explained his position, and others have clearly stated that they are unhappy with this. Continuing this discussion is unlikely to presuade anybody to change their mind or lead to any other type of positive outcome.
I agree. I'll simply state that I have added a DMARC policy to my own domain, whilst configuring SPF and DKIM. I want mailing lists to respect this and therefore not to break the SPF, DKIM signature or DMARC alignment. If they do, they have to rewrite the "From" header. Thus the problem isn't just with gmail.com, yahoo.com and other large providers - it's with anyone who uses DMARC, SPF and DKIM to help others distinguish between legitimate and illegitimate messages purportedly originating from their mail server. I expect the number of these domains to grow; all of these specifications are in RFCs.
On Sun, 17 Jan 2016 11:23:29 AM Jason White via luv-main wrote:
I agree. I'll simply state that I have added a DMARC policy to my own domain, whilst configuring SPF and DKIM. I want mailing lists to respect this and therefore not to break the SPF, DKIM signature or DMARC alignment. If they do, they have to rewrite the "From" header.
That use of From: is against RFC5322 and its antecedents, a mailing list should only change/add the Sender: field. If DMARC breaks because of this then its DMARC that needs to be fixed. # The "From:" field specifies the author(s) of the message, that is, the # mailbox(es) of the person(s) or system(s) responsible for the writing # of the message. # The "Sender:" field specifies the mailbox of the agent responsible # for the actual transmission of the message. # For example, if a secretary were to send a message for another person, # the mailbox of the secretary would appear in the "Sender:" field and the # mailbox of the actual author would appear in the "From:" field. This usage of the headers goes back to RFC822 from 1982, codified in its example headers. # A.2.2. Secretary-sent # # George Jones logs in as Jones on his host. His secre- # tary, who logs in as Secy sends mail for him. Replies to the # mail should go to George. # # From: George Jones <Jones@Group> # Sender: Secy@Other-Group All the best, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
On Fri, Jan 15, 2016 at 09:52:30AM +1100, Tony Langdon via luv-main wrote:
On 15/01/2016 12:14 AM, Russell Coker via luv-main wrote:
We had a problem of Gmail and other recipients falsely regarding mail from the LUV server as forged and assigning it to the spam folder.
Nothing that can't be fixed with a filter (use the "Never send it to spam" option in the Gmail filter for LUV list mail).
Unless it's being rejected by Gmail entirely, which is the case with messages sent from some Yahoo domain(s), that can't easily be fixed without breaking one standard or another.
We are dealing with a couple of bugs in Mailman, what I consider to be a deficiency in OpenDKIM and some interactions between SPF and DMARC.
If you would like to help with some of these technical issues then go for it.
I'm very tempted to investigate the Mailman header folding bug/feature, although I'd understand the maintainers not wanting to deal with edge cases like that, considering there's already effort on a newer version. I'd be even more inclined to fix this, if it weren't for the fact that SPF would fail DMARC domain alignment anyway... Amending the DMARC standard, and/or accommodating "trusted forwarders" might be the only solution, but then you have yet another problem, the subjectivity of trust.
http://postmaster.facebook.com/reputation_and_authentication
Facebook is now compelling sysadmins to use SPF or DKIM. This isn't going to go away. It's only a matter of time before Internode starts using DKIM to placate Facebook.
Looks like it's a case of having to follow suit., like it or not.
I think SPF or DKIM are far more bearable than DMARC (especially the inflexible domain alignment); if only that standard allowed either SPF or DKIM alignment, rather than both (AFAIK), but implementations may differ anyway. I'll eventually test this myself. The problem DMARC was designed to solve is that MUA will typically display the "From:" header as the sender, and it's easier to implement some crackpot standard to enforce alignment of that with authentication mechanisms, than expect every MUA to change behaviour by warning if a sender's authentication has not been verified.
participants (9)
-
Andrew Pam -
Brian May -
Chris Samuel -
Craig Sanders -
Erik Christiansen -
Jason White -
Joel W. Shea -
Russell Coker -
Tony Langdon