Postfix and Authenticated SMTP

Status: we've enabled authentication, but can't connect to SMTP on port 25 with say an e-mail client to test the real way, and have a new error for testing on the server: 538 5.7.0 Encryption required for requested authentication mechanism

The main resource:
http://workaround.org/articles/ispmail-etch/#step-9-authenticated-smtp

backup resource:
http://postfix.state-of-mind.de/patrick.koetter/smtpauth/requirements.html

ben-agaric: I followed the Workaround.org ISP-style Postfix for Debian Etch some time ago, generally things have worked great, and now I'm going back to add SMTP authentication (step 9) - http://workaround.org/articles/ispmail-etch/#step-9-authenticated-smtp
[4:14pm] ben-agaric: I think it's not working for at least two distinct reasons but I don't know how to make progress on either
[4:16pm] ben-agaric: dumber and maybe more obvious first: I can't get a reaction when I try to send mail using smtp.agaricdesign.com (our domain name) or mail.agaricdesign.com on port 25
[4:23pm] ben-agaric: how do i tell postfix to listen on port 25?

[4:24pm] mwalling: you have no MX record for agaricdesign.com
[4:25pm] mwalling: and MX records cannot refer to CNAME's
[4:25pm] ben-agaric: really? receiving mail is working fine
[4:25pm] mwalling: dig agaricdesign.com MX
[4:26pm] mwalling: you'll see no records
[4:26pm] mwalling: i just opened a smtp session with the server answering on 66.135.37.243 as well
[4:27pm] mwalling: 220 mail.democraticmedia.ca ESMTP Postfix (Debian/GNU)

ben-agaric: i have set up an mx record and am trying to understand how they are supposed to work, but as wrong as it may have been before i don't think it was the problem for sending mail through port 25
[4:43pm] ben-agaric: any next stes for testing or just wait for that to resolve first?

adaptr: ben-agaric: what is an "mx subdomain" ?
[4:58pm] adaptr: ben-agaric: it's quite simple: if you receive mail addressed to a domain owned by postfix, you have an MX record
[4:58pm] adaptr: if not, you would not, never ever, receive mail
[4:59pm] mwalling: !tell ben-agaric sasl

knoba: mwalling wants me to tell you: "sasl" : SASL is 'Simple Authentication and Security Layer', necessary for SMTP AUTH, and provided to Postfix by addin software. Cyrus SASL and/or Dovecot IMAP/POP3 can provide SASL. See http://www.postfix.org/SASL_README.html for details.

[5:00pm] ben-agaric: i doubt it was doing anything, but our dns had a mx.agaricdesign.com pointing to agaricdesign.com
[5:00pm] ben-agaric: and we have definitely been receiving e-mail!
[5:00pm] adaptr: no! why ? he was happily lost, and now you are steering him
[5:00pm] adaptr: funspoiler
[5:00pm] webPragmatist: adaptr: how can i check if this is true "# NOTE: remember to add the clamav user to the amavis group, and to properly set clamd to init supplementary groups"
[5:01pm] ben-agaric: thanks, i've been through that readme-- no, actually a similar but less complete looking readme!
[5:01pm] adaptr: webPragmatist: did you do it ?
[5:01pm] ben-agaric: nope, the same readme

ben@server:~$ sudo postconf -n
Password:
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
config_directory = /etc/postfix
inet_interfaces = all
inet_protocols = all
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 734003000
message_size_limit = 209715200
mydestination = server.pwgd.org, localhost.pwgd.org, localhost
myhostname = mail.democraticmedia.ca
mynetworks = 192.168.50.0/24
myorigin = /etc/mailname
recipient_delimiter = +
relayhost =
smtp_generic_maps = hash:/etc/postfix/smtp_generic_maps
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
smtpd_sasl_path = private/auth
smtpd_sasl_type = dovecot
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/ssl/certs/postfix.pem
smtpd_tls_key_file = /etc/ssl/private/postfix.pem
smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache
smtpd_use_tls = yes
unknown_local_recipient_reject_code = 450
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf
virtual_gid_maps = static:5000
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_transport = dovecot
virtual_uid_maps = static:5000

ben-agaric: the line i'm quite sure i got wrong; it was a desperate guess:
[5:09pm] ben-agaric: smtp_sasl_password_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

mad: does postfix support smtp authentication?
[7:19pm] vice-versa: yes
[7:19pm] vice-versa: !sasl
[7:19pm] knoba: vice-versa: "sasl" : SASL is 'Simple Authentication and Security Layer', necessary for SMTP AUTH, and provided to Postfix by addin software. Cyrus SASL and/or Dovecot IMAP/POP3 can provide SASL. See http://www.postfix.org/SASL_README.html for details.

Mar 28 13:00:34 server postfix/smtp[8780]: fatal: specify a password table via the `smtp_sasl_password_maps' configuration parameter
Mar 28 13:00:35 server postfix/master[8635]: warning: process /usr/lib/postfix/smtp pid 8780 exit status 1
Mar 28 13:00:35 server postfix/master[8635]: warning: /usr/lib/postfix/smtp: bad command startup -- throttling

Mar 28 13:25:59 server postfix/smtp[9036]: fatal: specify a password table via the `smtp_sasl_password_maps' configuration parameter
Mar 28 13:26:00 server postfix/master[8635]: warning: process /usr/lib/postfix/smtp pid 9036 exit status 1
Mar 28 13:26:00 server postfix/master[8635]: warning: /usr/lib/postfix/smtp: bad command startup -- throttling

ben@server:~$ sudo /etc/init.d/postfix restartPassword:

Stopping Postfix Mail Transport Agent: postfix.
Starting Postfix Mail Transport Agent: postfix.
ben@server:~$ telnet localhost smtpTrying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mail.democraticmedia.ca ESMTP Postfix (Debian/GNU)
ehlo agaricdesign.com
250-mail.democraticmedia.ca
250-PIPELINING
250-SIZE 209715200
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

Again, from the server:

telnet agaricdesign.com 25

Trying 66.135.37.243...
Connected to agaricdesign.com.
Escape character is '^]'.
220 mail.democraticmedia.ca ESMTP Postfix (Debian/GNU)
ehlo agaricdesign.com
250-mail.democraticmedia.ca
250-PIPELINING
250-SIZE 209715200
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
AUTH PLAIN YmVuQGFnYXJpY2Rlc2lnbi5jb20AYmVuQGFnYXJpY2Rlc2lnbi5jb20AemVsZGExMw==
503 5.5.1 Error: authentication not enabled

503 5.5.1 Error: authentication not enabled

ben@server:~$ sudo postconf -n

Password:
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
config_directory = /etc/postfix
inet_interfaces = all
inet_protocols = all
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 734003000
message_size_limit = 209715200
mydestination = server.pwgd.org, localhost.pwgd.org, localhost
myhostname = mail.democraticmedia.ca
mynetworks = 192.168.50.0/24
myorigin = /etc/mailname
recipient_delimiter = +
relayhost =
smtp_generic_maps = hash:/etc/postfix/smtp_generic_maps
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
smtpd_sasl_path = private/auth
smtpd_sasl_type = dovecot
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/ssl/certs/postfix.pem
smtpd_tls_key_file = /etc/ssl/private/postfix.pem
smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache
smtpd_use_tls = yes
unknown_local_recipient_reject_code = 450
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf
virtual_gid_maps = static:5000
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_transport = dovecot
virtual_uid_maps = static:5000

http://www.iptools.com/dnstools.php?tool=dns&user_data=agaricdesign.com&type=MX

; <<>> DiG 9.2.4 <<>> -t MX agaricdesign.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62205
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 3

;; QUESTION SECTION:
;agaricdesign.com. IN MX

;; ANSWER SECTION:
agaricdesign.com. 86400 IN MX 10 agaricdesign.com.

;; AUTHORITY SECTION:
agaricdesign.com. 86400 IN NS ns1.geodns.net.
agaricdesign.com. 86400 IN NS ns2.geodns.net.

;; ADDITIONAL SECTION:
agaricdesign.com. 86400 IN A 66.135.37.243
ns2.geodns.net. 150742 IN A 72.51.32.75
ns1.geodns.net. 150742 IN A 69.28.203.75

;; Query time: 41 msec
;; SERVER: 70.84.160.11#53(70.84.160.11)
;; WHEN: Fri Mar 28 16:36:55 2008
;; MSG SIZE rcvd: 144

Same as:

Ebony-II ben$ dig agaricdesign.com mx

The @ symbol replaced with & below to discourage spam.
mwalling's test:

Mar 29 08:58:29 server postfix/smtpd[15667]: 266C2944848: client=you.dontlike.us[67.18.208.100]
Mar 29 08:58:54 server postfix/cleanup[15657]: 266C2944848: message-id=<20080329135829.266C2944848&mail.democraticmedia.ca>
Mar 29 08:58:54 server postfix/qmgr[14692]: 266C2944848: from=<postmaster&you.dontlike.us>, size=381, nrcpt=1 (queue active)
Mar 29 08:58:54 server postfix/pipe[15681]: 266C2944848: to=<postmaster&agaricdesign.com>, relay=dovecot, delay=42, delays=42/0.01/0/0.06, dsn=2.0.0, status=sent (delivered via dovecot service)
Mar 29 08:58:54 server postfix/qmgr[14692]: 266C2944848: removed
Mar 29 08:59:07 server postfix/smtpd[15667]: disconnect from you.dontlike.us[67.18.208.100]
Mar 29 08:59:41 server postfix/smtpd[15653]: lost connection after CONNECT from smtp2.daviswv.net[70.101.252.226]

my normal mail send: same, right? So mwalling wasn't testing port 25 / authenticated SMTP

Mar 29 09:03:26 server postfix/smtpd[15703]: connect from smtp01.lnh.mail.rcn.net[207.172.4.11]
Mar 29 09:03:26 server postfix/smtpd[15703]: B2AC4944848: client=smtp01.lnh.mail.rcn.net[207.172.4.11]
Mar 29 09:03:26 server postfix/cleanup[15707]: B2AC4944848: message-id=<47EE4C27.2&mlncn.com>
Mar 29 09:03:26 server postfix/qmgr[14692]: B2AC4944848: from=<benjamin&mlncn.com>, size=754, nrcpt=1 (queue active)
Mar 29 09:03:26 server postfix/pipe[15708]: B2AC4944848: to=<postmaster&agaricdesign.com>, relay=dovecot, delay=0.2, delays=0.18/0.01/0/0.02, dsn=2.0.0, status=sent (delivered via dovecot service)
Mar 29 09:03:26 server postfix/qmgr[14692]: B2AC4944848: removed
Mar 29 09:03:31 server postfix/smtpd[15703]: disconnect from smtp01.lnh.mail.rcn.net[207.172.4.11]

[09:55am] ben-agaric: Postfix is listening on SMTP, but I can't engage port 25 from the outside: http://rafb.net/p/ec0PFp35.html
[09:55am] war9407: ben-agaric: firewall.
[09:56am] ben-agaric: Thanks war9407. Darn it, I set up this server. Why am I doing things I don't tell myself about?
[09:56am] cpm: ben-agaric, what says netstat -nuat | grep '\:25' ?
[09:56am] mwalling: ben-agaric: i've already telnet'ed into your server
[09:57am] FlyingSquirrel32: also make sure you have inet_interfaces = all in main.cf [check- this was fine]
[09:57am] mwalling: i did it yesterday, remember?
[09:57am] ben-agaric: for port 25? but how? I can get no activity in the mail log
[09:57am] cpm: btw, it works for me.
[09:58am] cpm: ben-agaric, you need to talk to yourself.
[09:58am] cpm: ;-)
[09:58am] ben-agaric: sudo netstat -nuat | grep '\:25' gave (i think the important line) tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN
[09:59am] mwalling: ben-agaric: look in your maillog now
[09:59am] ben-agaric:
ben@server:~$ sudo netstat -nuat | grep '\:25'
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN    
tcp        0      0 66.135.37.243:25        67.18.208.100:47572     ESTABLISHED
tcp        0      0 66.135.37.243:25        70.101.252.226:50803    ESTABLISHED
tcp        0      0 66.135.37.243:33779     72.52.144.201:25        TIME_WAIT 
tcp6       0      0 :::25                   :::*                    LISTEN
<-- there's everything
[09:59am] ben-agaric: i think i need an interpreter to talk to myself
[09:59am] mwalling: you should have a message from postmaster@you.dontlike.us -> postmaster@example.com
[10:00am] mwalling: are you doing NAT or something?

[10:00am] ben-agaric: Yeah, I have that. But I'm trying to send, not receive. To send with authenticated SMTP (and so through port 25)

[10:02am] ben-agaric: here's the log from mwalling test: [see above, with comparison]

[10:56am] ben-agaric: mwalling: can you explain a little more about what I am apparently missing with virtually every step of the concept of Authenticated SMTP. I have no proxies or NAT set up that I know of, and have access to change anything-- frighteningly, I have rather more power than knowledge at my disposal

Other: sending e-mail and staying on Yahoo's good side with domain keys:
http://www.enterux.com/en/resources/yahoo-domainkeys-howto-debian
http://dkim.org/info/dkim-faq.html#implementation
http://dkim.org/deploy/index.htm

ben@server:~$ sudo /sbin/iptables -L

Chain INPUT (policy ACCEPT)
target     prot opt source               destination        

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination        

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination        

Useful:
http://www.postfix-book.com/debugging.html

Resolution