emails are imported more than once

[mmartin-bbw]

Describe the bug
I run mail2most as a systemd service. In the config file i use the following filter section:

  [Profile.Filter]
		Folders = ["Inbox"]
		Unseen = true
		From = [""]
		To  = ["mattermost@mycompany.com"]
		Subject = [""]
		TimeRange = "24h"
		IgnoreMailErrorNotifications = true

After an E-Mail gets imported it is still marked as unread. I don't know whether the idea is that the email gets marked as unread or whether mail2most is supposed to keep track of what it already imported in some other way. However, the effect is that the same email gets imported with every time mail2most runs ( every TimesInterval ).

To Reproduce
Steps to reproduce the behavior:

1. configure mail2most with an imap email account and a mattermost account
2. send email to said account
3. watch target channel in mattermost
4. See duplicate of same email getting posted to target channel every time mail2most looks for new emails

Expected behavior

• Email gets sent to configured email account
• gets imported once
• further runs of mail2most ignore same email because it was "seen" before

Release version
v1.11.9

Architecture
linux amd64

[c-seeger]

Hey @mmartin-bbw. What email provider are you using in this case? In my testing the seen flag provided by the IMAP protocol works very inconsistent among different email provider implementations. For some it worked for some it was just ignored.

In general there should be no duplicates since mail2most stores a unique identifier of the mail in its data.json file. Can you check if this file was created properly?

[mmartin-bbw]

My email provider is hostpoint.ch. I do not know what they are using as an MTA, I just know they run everything on FreeBSD.
A data.json file was created, but it only contains

[
  null
]

Here is the systemd service definition that I am using:

# mail2most
[Unit]
Description=mail2most

[Service]
Type=simple
WorkingDirectory=/opt/mail2most
ExecStart=/opt/mail2most/mail2most -c conf/mail2most.conf
Restart=always
RestartSec=60

[Install]
WantedBy=multi-user.target

[c-seeger]

Hm interesting. That seems to be the issue. Do you see anything in the logs hinting mail2most not able to write the data.json?

[mmartin-bbw]

log excerpt via journalctl:
Mar 08 15:22:09 mattermost mail2most[1098456]: time="2024-03-08T15:22:09+01:00" level=info msg="unseen mails" ids="[2 3 4 5]"
I noticed that when I delete the data.json file that it gets re-written when mail2most finds and imports an email that fits the filter critera, but it contains again just the empty array.

[c-seeger]

Are the messages send to mattermost? Can you provide the full log output (don't forget to remove any security relevant information)?

[mmartin-bbw]

I can give you some more lines from the output of "journalctl -u mail2most.service --since -5min -f" right around the time when i sent an email to the mail2most email account. I

Mar 12 11:52:16 mattermost mail2most[1098456]: time="2024-03-12T11:52:16+01:00" level=info msg="processing mails" folder=Inbox
Mar 12 11:52:26 mattermost mail2most[1098456]: time="2024-03-12T11:52:26+01:00" level=info msg="processing mails" folder=Inbox
Mar 12 11:52:36 mattermost mail2most[1098456]: time="2024-03-12T11:52:36+01:00" level=info msg="processing mails" folder=Inbox
Mar 12 11:52:46 mattermost mail2most[1098456]: time="2024-03-12T11:52:46+01:00" level=info msg="processing mails" folder=Inbox
Mar 12 11:52:46 mattermost mail2most[1098456]: time="2024-03-12T11:52:46+01:00" level=info msg="unseen mails" ids="[1]"
Mar 12 11:52:46 mattermost mail2most[1098456]: time="2024-03-12T11:52:46+01:00" level=info msg="found mail" subject="test xx" uid=23
Mar 12 11:52:47 mattermost mail2most[1098456]: time="2024-03-12T11:52:47+01:00" level=error msg="Mattermost Error" Error=": Ungültiger oder fehlender email Parameter in Anfrage-URL., "
Mar 12 11:52:57 mattermost mail2most[1098456]: time="2024-03-12T11:52:57+01:00" level=info msg="processing mails" folder=Inbox
Mar 12 11:52:57 mattermost mail2most[1098456]: time="2024-03-12T11:52:57+01:00" level=info msg="unseen mails" ids="[1]"
Mar 12 11:53:07 mattermost mail2most[1098456]: time="2024-03-12T11:53:07+01:00" level=info msg="processing mails" folder=Inbox
Mar 12 11:53:07 mattermost mail2most[1098456]: time="2024-03-12T11:53:07+01:00" level=info msg="unseen mails" ids="[1]"
Mar 12 11:53:19 mattermost mail2most[1098456]: time="2024-03-12T11:53:19+01:00" level=info msg="processing mails" folder=Inbox
Mar 12 11:53:19 mattermost mail2most[1098456]: time="2024-03-12T11:53:19+01:00" level=info msg="unseen mails" ids="[1]"

As you can see, there is even an error, can you tell what the error means?

[95-M]

i have the same issue it drains the vps bandwith

[pilz]

tl;dr: configure mattermost user email instead of username worked for me.

I see the same behavior.

time="2024-05-26T15:50:12+02:00" level=info msg="found mail" subject="Redmine to mattermost support channel test" uid=984539465
time="2024-05-26T15:50:12+02:00" level=debug msg="user not found in system" error=": Unable to find the user., failed to find User: resource: User id: email=xxx@xxx.xxx"
time="2024-05-26T15:50:12+02:00" level=debug msg="mattermost post" bytes=556 channel=o4jdaq598py4xdcifymtfosimy subject="\n>_[Mattermost] Your password has been updated_\n\n"
time="2024-05-26T15:50:12+02:00" level=error msg="Mattermost Error" Error=": Invalid or missing email parameter in request URL., "
time="2024-05-26T15:50:12+02:00" level=debug msg="mattermost post" bytes=412 channel=o4jdaq598py4xdcifymtfosimy subject="\n>_Redmine to mattermost support channel test_\n\n"
time="2024-05-26T15:50:12+02:00" level=error msg="Mattermost Error" Error=": Invalid or missing email parameter in request URL., "

The problem might be "Invalid or missing email parameter in request URL"
I set the mattermost log level to debug to see if this is an error coming from mattermost, but there is no trace of this.

The messages are read from the mailserver, the posting to the mattermost channel happens successful. The only think not happening is the writing of the data.json, and therefore the emails are not remembered as read.

I played around a bit and even though my mattermost server is configured to allow usernames or emails as login, apparently the api does not. I had to change the username in the mail2most.conf to the mattermost email address. Then the data.json is properly written.