allmailfrom setting not having the desire effect

[paulgit]

I want all the emails that Nullmailer sends to come from email address new@xxxx.xx so I set this address in my allmailfrom file. The SMTP relay host I am using expects email from this address, but when I send a test message then there is a bounce in the log file and then it's moved to the failed folder, look at the file the first line implies the re-write has worked but the actual from address in the email is still the old one:

new@xxxx.xx
paul@xxxx.xx

Received: (nullmailer pid 944 invoked by uid 0);
  Wed, 08 Apr 2020 13:33:05 -0000
To: paul@xxxx.xx
Subject: Test Email
From: old@xxxx.xx
Date: Wed, 08 Apr 2020 14:33:05 +0100
Message-Id: <1586352785.655229.943.nullmailer@xxxx.xx.xxxx.xx>

This is occurring on Nullmailer 2.2 as distributed with Debian 10.

Is there perhaps another setting I need?

[berezovskyi]

I am having a similar problem. I realised that envelope sender change as discussed in #13 and #55 will not do much good if the provider strictly checks username==sender==from, as Migadu does:

554 5.7.1 From header does not match envelope sender.

I am still not sure what the sendmail-bin wrapped mentioned by @jahir in #55 means, but seems like it could be a real solution.

[mehov]

Unfortunately, allmailfrom seems to add return path only, and I'm still getting:

smtp: Failed: 550 5.7.0 Sender or From header address rejected: not owned by authorized user

Setting the global

NULLMAILER_HOST=example.com
NULLMAILER_USER=sender

in /etc/environment helps with direct sendmail "recipient@example.com" calls, but has no effect on the cron mail.

A wrapper script has technically worked, but my SMTP still rejected with Message rejected under suspicion of SPAM, so not an option either.

Found this dma mention which seems like it could be an alternative and will have to look into it.

[henning-schild]

It is an issue that deserves fixing directly here, not with a wrapper. Having control over "From" and "Sender" is often needed. Not sure it should be done via allmailfrom or another mechanism to stay compatible to older versions of nullmailer.

But honestly allmailfrom not actually touch the From: is all but intuitive ;)

[lkraav]

But honestly allmailfrom not actually touch the From: is all but intuitive ;)

Yeah, allmailsender would perhaps be more accurate.

[adippl]

allmailfrom seems to affect From: line in logs, but doesn't affect actual mail.
I guess It could be a bug.

I've tried to check how how allmailfrom works and modify it, but I couldn't figure it out.
I've created small hack for sendmail which works like -f flag but loads From: from a file.
It could be more reliable and easier solution than enviroment variables.

It works with nullmailer's sendmail and mail from mailutils.
I had issue with cronie (cron). Cron mails somehow retain their original From: field.
I still had to change MAILFROM variable in cron config file to set proper From:

Filepath is hardcoded because I'm not familiar with automake/autoconf generated variables.
It's c style code because that's what I'm more familiar with.

diff --git a/src/sendmail.cc b/src/sendmail.cc
index 534ad39..3a81845 100644
--- a/src/sendmail.cc
+++ b/src/sendmail.cc
@@ -29,6 +29,7 @@
 #include "forkexec.h"
 #include "setenv.h"
 #include "cli++/cli++.h"
+#include "stdio.h"
 
 const char* cli_program = "sendmail";
 const char* cli_help_prefix = "Nullmailer sendmail emulator\n";
@@ -100,6 +101,32 @@ bool setenvelope(char* str)
   return true;
 }
 
+bool force_From()
+{
+	char* str = NULL;
+	FILE* f = NULL;
+	char fpath[]{"/etc/nullmailer/forced_from"};
+	size_t strSize = -1;
+	ssize_t gl_ret = 0;
+	if(access(fpath, F_OK)==0){
+		f = fopen( fpath, "r");
+		gl_ret = getline(&str, &strSize, f);
+		if( gl_ret == -1 ) {
+			free(str);
+			fclose(f);
+			return false;
+		}
+		fclose(f);
+		// cut off the last character if it's newline
+		if ( str[gl_ret-1] == '\n' )
+			str[gl_ret-1] = '\0';
+		setenvelope(str);
+		free(str);
+		return true;
+	}
+	return false;
+}
+
 int do_exec(const char* program, const char* args[])
 {
   mystring path = program_path(program);
@@ -136,6 +163,8 @@ int cli_main(int argc, char* argv[])
         extra_args[extra_argc++] = o_from;
       }
     }
+    if(!o_from)
+      force_From();
 
     for(int i = 0; i < argc; i++)
       extra_args[extra_argc++] = argv[i];

[jarthod]

I just encountered this problem too when trying to use nullmailer with a custom From. Is there any other way (than modifying the source code) to achieve this? do we know if it's a bug or not? or shall I just use another service?

[adippl]

I think I have figured it out.
Some programs like cronie cron daemon generate mail files with their own From:
Right now allmailfrom cannot overwrite From: line which already exist in mail file piped into sendmail.
It sometimes sets only Return-Path: because this header field was not already set in mail file generated by crond.

I have created a patch which replaces From: if allmailfrom is configured.
This is example of a cron mail with correctly working allmailfrom
user and hostname are replaced, but original name of the sender is unchanged.

X-Cron-Env: <HOME=/>
X-Cron-Env: <LOGNAME=root>
X-Cron-Env: <USER=root>
Date: Thu, 26 May 2022 15:45:01 +0200
Message-Id: <1653572701.537208.8045.nullmailer@REMOVED>
From: CronDaemon <allmailfrom@fake.email>
X-TUID: Pb8F2z6y9nw3

I have created a pull request with my fix. #84

[gnugnug]

Unfortunately setting NULLMAILER_USER and NULLMAILER_HOST doesn't help when the program that creates the email already sets a sender address. mail -r foo@bar.tld <recipient> will still create an email with the header "From: foo@bar.tld" even with NULLMAILER_USER and NULLMAILER_HOST set.

nullmailer-inject has the flag "f", which: "Ignore and remove any From header lines and always insert a generated one"
Nuking the original From header (which we don't want) would theoretically fix the problem, unfortunately nullmailer-inject runs before the return path address is replaced by the value from /etc/nullmailer/allmailfrom.

As a workaround we can use a wrapper script as suggested in this thread and in https://www.atwillys.de/content/linux/force-nullmailer-to-use-a-fixed-from-address/:

1. Move /usr/sbin/sendmail to /usr/sbin/sendmail-bin
2. Create a wrapper script at /usr/sbin/sendmail and make it executable:

#!/bin/bash
/usr/sbin/sendmail-bin "$@" -f "$(cat /etc/nullmailer/allmailfrom)" </dev/stdin

This will append the address from /etc/nullmailer/allmailfrom as a sender address to every sendmail call. After this the "f" flag will kick in, remove the original from header and all occurences of the original from address will be gone. Now every script on your system can create emails with whatever from addresses it wants, as those addresses will all be overwritten. You only have to make sure to always have NULLMAILER_FLAGS=f set. Add NULLMAILER_FLAGS=f to /etc/environment (for PAM aware logins) and export NULLMAILER_FLAGS=f to /etc/profile.d/nullmailer.sh (for Bash).

[kyrofa]

Well, darn, just hit this on Debian 12 (Bookworm). I quickly fell in love with the simplicity of nullmailer, but my SMTP server (postmark) is pretty strict about the email address in the From line. I need it to be a specific one.

[LaXiS96]

Same here, Zoho bounces email with 553 Relaying disallowed as MAILER-DAEMON@myhost.mydomain when sending a test email with echo test | mail user@domain.tld, ignoring the value of allmailfrom.
I was previously using msmtp as a sendmail-compatible MTA but it doesn't queue failed messages, so it isn't a good match for e.g. cron mail. Looks like nullmailer can't help me either...

[walhallaRV]

Same here. this issue was opened in 2020 and there is no fix for this ignored allmailfrom? Anyone another MTA with retry queu and the option to "rewrite" senders / from like postfix canonical or atleast everytime one fixed sender-address?

[walhallaRV]

Solved: try MSMTP (https://marlam.de/msmtp/). Working like a charme and yes, sender re-writing works with just one line in the cfg-file. Atleast in Debian it is in the rep (apt-get install msmtp msmtp-mta).

Good manual (with cfg - examples) and a bunch of options (https://marlam.de/msmtp/msmtp.html)

Happy now, everything works!

[UffeJakobsen]

@walhallaRV

Solved: try MSMTP (https://marlam.de/msmtp/). Working like a charme and yes, sender re-writing works with just one line in the cfg-file. Atleast in Debian it is in the rep (apt-get install msmtp msmtp-mta).

Good manual (with cfg - examples) and a bunch of options (https://marlam.de/msmtp/msmtp.html)

A word of warning - while I agree that msmtp has many nice features - and a good documentation - it is not equivalent with nullmailer in the most common use case (at least from my point of view).

msmtp does not have a decent queue implementation - if the target system cannot be reached in the moment the mail is sent - the mail is discarded :-/

nullmailer on the other hand does have a robust queue implementation with periodical retries etc..

This makes nullmailer a much better choice for multi server/virtual-machine installations where you do not want the burden of a full MTA on every system - but just want any message relayed away from the systems and into a centralized mail system.

msmtp has (what looks like a somewhat flaky implementation of) a queueing system made up from several shell scripts.
At least on archlinux the msmtp package does not have any systemd service/daemons setup included - meaning that a centralized daemon and queuing setup for all users on the system will have to be done manually... including periodical retries etc...

More on the topic here: https://wiki.archlinux.org/title/Msmtp#Using_msmtp_offline

[walhallaRV]

@walhallaRV

Solved: try MSMTP (https://marlam.de/msmtp/). Working like a charme and yes, sender re-writing works with just one line in the cfg-file. Atleast in Debian it is in the rep (apt-get install msmtp msmtp-mta).
Good manual (with cfg - examples) and a bunch of options (https://marlam.de/msmtp/msmtp.html)

A word of warning - while I agree that msmtp has many nice features - and a good documentation - it is not equivalent with nullmailer in the most common use case (at least from my point of view).

msmtp does not have a decent queue implementation - if the target system cannot be reached in the moment the mail is sent - the mail is discarded :-/

nullmailer on the other hand does have a robust queue implementation with periodical retries etc..

This makes nullmailer a much better choice for multi server/virtual-machine installations where you do not want the burden of a full MTA on every system - but just want any message relayed away from the systems and into a centralized mail system.

msmtp has (what looks like a somewhat flaky implementation of) a queueing system made up from several shell scripts. At least on archlinux the msmtp package does not have any systemd service/daemons setup included - meaning that a centralized daemon and queuing setup for all users on the system will have to be done manually... including periodical retries etc...

More on the topic here: https://wiki.archlinux.org/title/Msmtp#Using_msmtp_offline

Hi,
@ nullmailer: which sense makes any queue if it tries to sent email with a "stupid" from because this allmailfrom does not work and email is rejected??? i prefer to loose one email because the smarthost is unreachable (99.9% uptime) than loosing all mails because they are rejected. No doubts, nukkmailer maybe a good solution. But options that dont work, searching hours to find the error "you made yourself" and at the end realizing that many other have the same problem (unsolved for 4 years). And any "quick & dirty" solution where you dont know if it works everytime / situation? And after the next update ...? No, thanks.

@ msmtp: you are not right! You just have to install the msmtp-mta package and you have a daemon running, listening on on localhost, port 25 (or whatever IP/port you can set in the cfg) -> https://marlam.de/msmtp/msmtp.html#Example-using-msmtpd-as-a-system-service.
Queue: will try what happens if the smarthost is not reachable. But I can find it in the logfile and maybe I have to resend one mail manually. Can live with it. Better than none email because rewrite doesnt work.

After fighting 2 hours with nullmailer (unsuccessful) it took me 5 minutes to setup msmtp (working). You can make a different cfg for every user (systemwide/root and/or user cfgs), can use different smarthosts for different emails, have aliases, can rewrite nearly everything. But best of all: 10 lines in the cfg and is running "out of the box".

Everybody is free to use whatever he wants / prefers or might be your preference. I just wanted to mention here because several guys have the same problem and it seems that nobody cares about it (mail from). After nullmailer I decided to install postfix. Just local via smarthost is done in 5 minutes too. But didnt want to have the whole postfix just to relay emails. Then I found msmtp, gave a last try before postfix and bingo ;)

[UffeJakobsen]

@ msmtp: you are not right! You just have to install the msmtp-mta package and you have a daemon running, listening on on localhost, port 25 (or whatever IP/port you can set in the cfg) -> https://marlam.de/msmtp/msmtp.html#Example-using-msmtpd-as-a-system-service. Queue: will try what happens if the smarthost is not reachable. But I can find it in the logfile and maybe I have to resend one mail manually. Can live with it. Better than none email because rewrite doesnt work.

msmtp-mta package will only give you a simple (relay) daemon listening on localhost sending to smarthost - but still no disk-based queue with periodic retries/retransmits.

msmtp-mta package (on archlinux) does not have any systemd service/daemon setup included in the package.

I do not currently have the problem with the 'allmailfrom'. My smart host is dedicated - so I could just change the rules on the smart host side - if I had the problem. But still it would be nice if the 'allmailfrom' was fixed in nullmailer.

With a quite large number of systems (and even more vm's) running - I prefer receiving all emails (eg warnings from disk smart monitoring tools etc etc) - and I do prefer to get them deferred - not loosing them - in the event that the smart host for some reason is down/unavailable.
For the above use case nullmailer is doing exactly what it needs to do - msmtp will not do the job without lots of integration work.
But then again msmtp may do the job to a satisfactory level for others with other use cases.
My only reason for writing these comments was to warn others that msmtp is not a plugin replacement for nullmailer - at least not on all features - thus preventing people from too quickly jumping to wrong conclusions without realizing the differences between nullmailer and msmtp. It was not my intention to side track this issue ticket with a long off-topic discussion.

[walhallaRV]

Maybe is an arch - problem? Debian does ->

msmtpd.service - msmtp daemon
Loaded: loaded (/lib/systemd/system/msmtpd.service; enabled; preset: enabled)
Active: active (running) since Tue 2024-07-01 16:17:34
Docs: man:msmtpd(1)
Main PID: 36416 (msmtpd)
Tasks: 1 (limit: 9311)
Memory: 640.0K
CPU: 26ms
CGroup: /system.slice/msmtpd.service
└─36416 /usr/bin/msmtpd --interface=127.0.0.1 --port=25

Yes, I want or need status / alarm / ... emails from VM too. But once again: maybe better lose one email than no email.

And for users who use gmail, hotmail, web.de, etc. as smarthost: how many times this machines are offline?

Regarding the queue I didnt have time to check what happens when smarthost is unreachable. But I remember - when I read (overflew ;) the manual the first time, I read something about "DSN configurations / options". But I didnt read at this moment because was not interesting ;) Think that I read also that there is ETRN availlable? Will doublecheck later if I have more time to play around with it.

I didnt want to be OT here too. Therefore I just mentioned that there is an alternative availlable. With working rewrite out of the box for whom it is important and doesnt like "quick & dirty hacks". Who ever has other preferences, has to decide himself which "dead is better" ;)

[LaXiS96]

@walhallaRV the fact that msmtp can run as a daemon does not imply that it can queue failed messages and retry sending them periodically, which is one of the major features of nullmailer.

allmailfrom should simply do what it says on the tin: override the From header with the specified address in all mail. That should fix my issue with Zoho, which expects the From header to match the login or a registered alias.

I am currently using msmtp with its queue scripts, it has been working okay so far (for automated system mail like cron).

[walhallaRV]

I just answered to UffeJakobsens coment that msmtp "does not have any systemd service/daemon setup included" - but it has. This is no queuing, sure, I know!

If nullmail would have worked like expected (and pronounced) - I would have used. I use msmtp too mainly for sending cron / script reports / results. And it works, I get all emails with correct from.