appendFull is broken with MIME messages

[mpscholten]

When writing a MIME message, the use of BS.lines breaks the MIME encoding.

OpenAI o1 just found this bug:

The likely culprit is how the appendFull function handles newlines. Notice it does:

mailLines = BS.lines mailData
...
mapM_ (bsPutCrLf $ stream conn) mailLines
bsPutCrLf (stream conn) BS.empty

That means:
1. It first calls BS.lines mailData, which splits on \n (LF), leaving any \r characters intact at the end of each line.
2. Then for each line, it appends CRLF again via bsPutCrLf.

So if your mailData (the fully rendered RFC 5322 message) already has CRLF line endings, splitting on \n leaves each line ending with \r. Then mapM_ (bsPutCrLf ...) will write <original line>\r\r\n to the server, effectively doubling up the \r character. This corrupts the raw message in transit. Gmail sees something like:

Subject: Something\r\r
MIME-Version: 1.0\r\r
... etc.

and can’t parse it properly into headers + body.

How to fix

You want to pass the exact bytes of your RFC 5322 message to the IMAP server. A safer approach is something like:

appendFull :: IMAPConnection -> MailboxName -> ByteString
           -> Maybe [Flag] -> Maybe CalendarTime -> IO ()
appendFull conn mbox mailData flags' time = do
    let len = BS.length mailData  -- exact raw length
        tstr = maybe "" ((" "++) . datetimeToStringIMAP) time
        fstr = maybe "" ((" ("++) . (++")") . unwords . map show) flags'
        cmd  = concat
            [ "APPEND ", mbox
            , fstr, tstr
            , " {", show len, "}"
            ]

    (buf, num) <- sendCommand' conn cmd
    when (BS.null buf || (BS.head buf /= '+')) $
        fail "illegal server response"

    -- Write the mail data in one shot, unmodified
    BS.hPut (stream conn) mailData
    -- Then a final CRLF
    bsPutCrLf (stream conn) BS.empty

    buf2 <- getResponse $ stream conn
    let (resp, mboxUp, ()) = eval pNone (show6 num) buf2
    case resp of
        OK _ _   -> mboxUpdate conn mboxUp
        NO _ msg -> fail ("NO: " ++ msg)
        BAD _ msg -> fail ("BAD: " ++ msg)
        PREAUTH _ msg -> fail ("PREAUTH: " ++ msg)

This way, you’re not calling BS.lines. The server gets exactly the raw email you constructed, with the correct CRLF lines, the correct blank line between headers and body, etc.

Workaround

let sanitized = mailData
             -- remove any `\r` we already have:
             |> BS.filter (/= 0x0D)
             -- now `BS.lines` will split on `\n` only
    mailLines = BS.lines sanitized

...

mapM_ (bsPutCrLf $ stream conn) mailLines
bsPutCrLf (stream conn) BS.empty