SPF permanent errors

[macc0]

Describe the bug

Some incoming email is rejected because of failed SPF checks. Same email is received fine by a hotmail.com address.

What do you think is wrong?

Steps to reproduce

Any emails received from hegre.com or metart.com. Like signing up to their newsletters.

Log files

2022-05-03T11:21:20.610Z smtp: incoming message	{"msg_id":"5b63fbdb","sender":"bounce+5c9361.a2654d-XXXX=XXXX.xyz@metart.com","src_host":"a228-193.mailgun.net","src_ip":"143.55.228.193:57053"}
2022-05-03T11:21:20.614Z smtp: RCPT ok	{"msg_id":"5b63fbdb","rcpt":"XXXX@XXXX.xyz"}
2022-05-03T11:21:20.874Z smtp: DATA error	{"check":"check.spf","msg_id":"5b63fbdb","reason":"too many MX records","smtp_code":550,"smtp_enchcode":"5.7.23","smtp_msg":"SPF authentication failed with a permanent error"}

2022-05-03T12:27:28.387Z smtp: incoming message	{"msg_id":"ba875450","sender":"no-reply@hegresupport.com","src_host":"03.hegre.com","src_ip":"184.94.152.250:38220"}
2022-05-03T12:27:28.453Z smtp: RCPT ok	{"msg_id":"ba875450","rcpt":"XXXX@XXXX.xyz"}
2022-05-03T12:27:28.630Z smtp: DATA error	{"check":"check.spf","msg_id":"ba875450","reason":"multiple matching DNS records","smtp_code":550,"smtp_enchcode":"5.7.23","smtp_msg":"SPF authentication failed with a permanent error"}

Configuration file

## Maddy Mail Server - default configuration file (2021-08-16)
# Suitable for small-scale deployments. Uses its own format for local users DB,
# should be managed via maddyctl utility.
#
# See tutorials at https://maddy.email for guidance on typical
# configuration changes.
#
# See manual pages (also available at https://maddy.email) for reference
# documentation.

# ----------------------------------------------------------------------------
# Base variables

$(hostname) = {env:MADDY_HOSTNAME}
$(primary_domain) = {env:MADDY_DOMAIN}
$(local_domains) = $(primary_domain)

log /data/maddy.log

tls file /data/tls_cert.pem /data/tls_key.pem

# ----------------------------------------------------------------------------
# Local storage & authentication

# pass_table provides local hashed passwords storage for authentication of
# users. It can be configured to use any "table" module, in default
# configuration a table in SQLite DB is used.
# Table can be replaced to use e.g. a file for passwords. Or pass_table module
# can be replaced altogether to use some external source of credentials (e.g.
# PAM, /etc/shadow file).
#
# If table module supports it (sql_table does) - credentials can be managed
# using 'maddyctl creds' command.

auth.pass_table local_authdb {
    table sql_table {
        driver sqlite3
        dsn credentials.db
        table_name passwords
    }
}

# imapsql module stores all indexes and metadata necessary for IMAP using a
# relational database. It is used by IMAP endpoint for mailbox access and
# also by SMTP & Submission endpoints for delivery of local messages.
#
# IMAP accounts, mailboxes and all message metadata can be inspected using
# imap-* subcommands of maddyctl utility.

storage.imapsql local_mailboxes {
    driver sqlite3
    dsn imapsql.db
}

# ----------------------------------------------------------------------------
# SMTP endpoints + message routing

hostname $(hostname)

table.chain local_rewrites {
    optional_step regexp "(.+)\+(.+)@(.+)" "$1@$3"
    optional_step static {
        entry postmaster postmaster@$(primary_domain)
    }
    optional_step file /etc/maddy/aliases
}

msgpipeline local_routing {
    # Insert handling for special-purpose local domains here.
    # e.g.
    # destination lists.example.org {
    #     deliver_to lmtp tcp://127.0.0.1:8024
    # }

    destination postmaster $(local_domains) {
        modify {
            replace_rcpt &local_rewrites
        }

        deliver_to &local_mailboxes
    }

    default_destination {
        reject 550 5.1.1 "User doesn't exist"
    }
}

smtp tcp://0.0.0.0:25 {
    limits {
        # Up to 20 msgs/sec across max. 10 SMTP connections.
        all rate 20 1s
        all concurrency 10
    }

    dmarc yes
    check {
        require_mx_record
        dkim
        spf
    }

    source $(local_domains) {
        reject 501 5.1.8 "Use Submission for outgoing SMTP"
    }
    default_source {
        destination postmaster $(local_domains) {
            deliver_to &local_routing
        }
        default_destination {
            reject 550 5.1.1 "User doesn't exist"
        }
    }
}

submission tls://0.0.0.0:465 tcp://0.0.0.0:587 {
    limits {
        # Up to 50 msgs/sec across any amount of SMTP connections.
        all rate 50 1s
    }

    auth &local_authdb

    source $(local_domains) {
        check {
            authorize_sender {
                prepare_email &local_rewrites
                user_to_email identity
            }
        }

        destination postmaster $(local_domains) {
            deliver_to &local_routing
        }
        default_destination {
            modify {
                dkim $(primary_domain) $(local_domains) default
            }
            deliver_to &remote_queue
        }
    }
    default_source {
        reject 501 5.1.8 "Non-local sender domain"
    }
}

target.remote outbound_delivery {
    limits {
        # Up to 20 msgs/sec across max. 10 SMTP connections
        # for each recipient domain.
        destination rate 20 1s
        destination concurrency 10
    }
    mx_auth {
        dane
        mtasts {
            cache fs
            fs_dir mtasts_cache/
        }
        local_policy {
            min_tls_level encrypted
            min_mx_level none
        }
    }
}

target.queue remote_queue {
    target &outbound_delivery

    autogenerated_msg_domain $(primary_domain)
    bounce {
        destination postmaster $(local_domains) {
            deliver_to &local_routing
        }
        default_destination {
            reject 550 5.0.0 "Refusing to send DSNs to non-local addresses"
        }
    }
}

# ----------------------------------------------------------------------------
# IMAP endpoints

imap tls://0.0.0.0:993 tcp://0.0.0.0:143 {
    auth &local_authdb
    storage &local_mailboxes
}

Environment information

• maddy version: docker container foxcpp/maddy:latest

[foxcpp]

maddy has a rather strict SPF handling by default. These domains indeed have SPF policies that are not valid per RFC 7208.
I guess we should review the defaults and see if it is better to ignore such errors (as hotmail probably does). In meanwhile, you can set it manually:

    check {
        require_mx_record
        dkim
        spf {
            permerr_action ignore # <---
        }
    }

[macc0]

Thank you. That's what I did eventually.
I commented out the spf line which probably amounts to the same thing.

When investigating the issue, I came upon this blog post:
https://www.mailgun.com/blog/deliverability/email-isnt-authentic-enough-misfit-series/

Some of these issues seem quite common.

I guess we should review the defaults and see if it is better to ignore such errors (as hotmail probably does).

Right. Or maybe allow whitelisting certain domains in the config file.

[foxcpp]

I commented out the spf line which probably amounts to the same thing.

This disables SPF checking completely which is probably not something you want to do (as this also effectively disables DMARC).