This is not a valid excuse. To me it sounds in fact like a mere rationalization. The marginal benefit of a dollar is no less in this case for the first dollar than the last dollar. So if you actually believe what you are saying, you would give a substantial portion, eg 40-50%, of your salary to such causes.
The fact that precisely no-one has stepped up with evidence that they do so is telling. As I said before, it is easy to be generous with other people's money.
Verbal pronouncements of one's moral virtue are cheap which is why they are so common.
Tim