lwc:linux:sympa [wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Install =====
==== apt install sympa 6.2.70 on ubuntu 24.04 ====
  * Start with [[https://www.sympa.community/manual/install.html|the official installation docs using apt]]
  * Then when that fails :-o

=== mysql ===
  * with install ''dbconfig-common'' to set up DB using mysql just doesn't work.
  * it didn't work for mailman and it didn't work here
  * it Says "password doesn't match requirements" (which isn't true).
  * you have to set up the DB yourself
<code>
sudo mysql -p -e "CREATE DATABASE IF NOT EXISTS sympa CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; CREATE USER IF NOT EXISTS 'sympa'@'localhost' IDENTIFIED BY '<RANDOM PASSWORD HERE>'; GRANT ALL PRIVILEGES ON sympa.* TO 'sympa'@'localhost'; FLUSH PRIVILEGES;"

sudo bash -c 'cat >> /etc/sympa/sympa/sympa.conf <<EOF
###\\\\ Database parameters ////###
db_type mysql
db_host localhost
db_user sympa
db_passwd <RANDOM PASSWORD HERE>
db_name sympa
EOF'
</code>

=== nginx ===
//apparently standard setup is with Apache? But this works//
<code>
server {
    listen 80;
    server_name <SUBDOMAIN NAME HERE>;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl; 
    server_name <SUBDOMAIN NAME HERE>;

    ssl_certificate /etc/letsencrypt/live/<SOMETHINGSOMETHING>/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/<SOMETHINGSOMETHING>//privkey.pem;

    root /usr/lib/cgi-bin/sympa;
    
    access_log /var/log/nginx/sympa-access.log;
    error_log /var/log/nginx/sympa-error.log;

    location /static-sympa/ {
        alias /usr/share/sympa/static_content/;
        access_log off;
    }

    location /css-sympa/ {
        alias /var/lib/sympa/css/;
        access_log off;
    }

    location /pictures-sympa/ {
        alias /var/lib/sympa/pictures/;
        access_log off;
    }

    location / {
        include fastcgi_params;
        fastcgi_pass unix:/run/sympa/wwsympa.socket;
        fastcgi_param SCRIPT_FILENAME /usr/lib/cgi-bin/sympa/wwsympa.fcgi;
        fastcgi_param SCRIPT_NAME "";
        fastcgi_param PATH_INFO $uri;
    }
}
</code>

=== Postfix ===

== main.cf ==
  * Postfix tries to run some sympa stuff as an unprivileged user. Add this line to have it run as the sympa user instead:
<code>
sympa     unix  -       n       n      -        -       pipe
  flags=R user=sympa argv=/usr/lib/sympa/bin/queue ${recipient}
</code>
  * add this to the ''mydestination'' line: ''<YOUR SUBDOMAIN>''
  * add ''hash:/etc/postfix/transport'' to the ''transport_maps'' line
  * add ''hash:/etc/sympa/aliases.sympa.postfix'' to ''alias_maps''
  * add ''alias_database = hash:/etc/sympa/aliases.sympa.postfix''
  * suggestion (for sending larger emails): ''message_size_limit = 41943040''
== master.cf ==
<code>
sympa     unix  -       n       n      -        -       pipe
  flags=R user=sympa argv=/usr/lib/sympa/bin/queue ${recipient}
</code>

=== /etc/sympa/sympa/sympa.conf ===
<code>
wwsympa_url <YOUR_SUBDOMAIN>

# 1. Add email address text when list is created
alias_manager /usr/lib/sympa/bin/alias_manager.pl

# 2. compile email address text and add to db
aliases_program /usr/sbin/postalias

# 3. file Location for text and db
sendmail_aliases /etc/sympa/aliases.sympa.postfix

# If you have trouble change from default log_level 1
log_level 2

# The sympa manager defaults to sympa so emails to subscribe/unsubscribe etc. will come from sympa@DOMAIN. You can change this. For example, to have them come from manager instead add the below line:
email manager

</code>
==== Notes ====
  * config is at ''/etc/sympa/sympa/sympa.conf''
  * when reading official documentation ''$SYSCONFDIR'' is ''/etc/sympa''
  * when reading official documentation ''$LIBEXECDIR'' is ''/usr/share/sympa/bin''

==== Troubleshooting ====
  * sympa logs go to ''syslog''. View in realtime: ''sudo tail -f /var/log/syslog | grep --line-buffered -Ei 'sympa|wwsympa''
  * postfix logs go to ''/var/log/mail.log''

===== Configuration =====

==== Amazon SES relay ====
  * if forwarding emails to Amazon SES for sending, Amazon gets upset because of a duplicate header: ''Precedence''
    * This is because Sympa adds a Precedence: bulk (or list) header automatically to all list traffic to help spam filters identify it as a mailing list.
    * Apparently the outgoing sender might have also added a Precedence header or maybe sympa is just confused :-o
    * The result is that there is no Precedence header at all but that header has been obsoleted anyway so nothing lost
  * to fix, add to ''/etc/sympa/sympa/sympa.conf'': ''remove_outgoing_headers Precedence''
    * also available as a setting in the UI in ''sending/receiving setup'' -> ''Header fields to be removed before message distribution (remove_outgoing_headers)(default)''

==== Customizing Templates ====
Sympa uses a "Cascading" template system. It looks for a template (like welcome.tt2) in this specific order:
  - List Level: ''/var/lib/sympa/list_data/[list_name]/mail_tt2/'' (Highest Priority)
  - Robot/Domain Level: ''/etc/sympa/[domain]/mail_tt2/''
  - Site Level: ''/etc/sympa/mail_tt2/''
  - Default: ''/usr/share/sympa/default/mail_tt2/'' (Lowest Priority)
=== Cascaded Templates for messages sent to the list ===
  * have 2 footers, one plaintext (''message.footer'') and one html (''message.footer.mime'')

  - List Level: ''/etc/sympa/list_configs/[list_name]/message.footer.mime''
  - Robot/Domain Level: ''/etc/sympa/lists.lawrencemakers.org/message.footer.mime''
  - Site/Global Level: /''etc/sympa/message.footer.mime''
  - Default: ''/usr/share/sympa/default/message.footer.mime''

==== Unsubscribe Header ====
  * There's the "standard" list-unsubscribe header as defined in RFC 2369 and a newer "one-click" header as defined in RFC 8058
  * Sympa v6.2.70 as packaged with ubuntu 24.04 generates the standard header automatically by default
  * The newer "one-click" header is not supported until v6.2.72

=== The Standard Header (RFC 2369) ===
  * If the header is working properly there will be a line of hidden metadata at the top of an email that looks something like
<code>
List-Unsubscribe: <mailto:sympa@<DOMAIN>?subject=SIGNOFF%20<LIST_NAME>>, <https://<DOMAIN>/signoff/<LIST_NAME>/user@email.com>
</code>
  * If this header is present, Gmail, Yahoo, and "friends" will automatically display their native "Unsubscribe" button next to the sender's name.

==== List Config ====
  * config file located at: ''/var/lib/sympa/list_data/<LISTNAME>/config''
  * To get the emails sent from the list instead of the sender in the UI:
    * ''Admin'' -> ''Edit List Config'' -> ''DKIM/DMARC/ARC''
      * ''DMARC Protection'' -> ''All''
      * ''New From name format'' -> ''"Name" (via list)''
  * Change the from address from the sender to the list:
    * ''Admin'' -> ''Edit List Config'' -> ''Sending/Receiving Setup''
    * ''"Reply address"'' -> change to ''All''
  * Prepend the list name to the subjects when sending out:
    * ''Admin'' -> ''Edit List Config'' -> ''Sending/Receiving Setup''
    * ''Subject tagging (custom_subject)'' -> ''[% list.name %]''
  * Get rid of the "via list" in the from:
    * ''Admin'' -> ''Edit List Config'' -> ''DKIM/DMARC/ARC''
    * ''New From name format (phrase)'' -> set to ''Name''
  * consider increasing max message size:
    * ''Admin'' -> ''Edit List Config'' -> ''Sending/Receiving Setup''
    *  ''Maximum message size (max_size)(default)'' -> ''20971520''
  * allow subscribers to view who is subscribed:
    * ''Who can review subscribers (review)'' -> ''Restricted to subscribers (private)''
  * create a list template to change the defaults for a new list:
<code>
# make a home for the new default override
sudo mkdir -p /etc/sympa/create_list_templates
# copy from the original default
sudo cp -r /usr/share/sympa/default/create_list_templates/discussion_list /etc/sympa/create_list_templates/
# edit to your heart's content:
sudo emacs -nw /etc/sympa/create_list_templates/discussion_list/config.tt2
</code>

==== Prettying it up ====
=== Logo and Favicon ===
  * the logo does not change the size of the blue line at the top of the site so you will get a logo that straddles blue and white
  * with the nginx setup above the logo and favicon images should live at ''/usr/share/sympa/static_content''
  * to get sympa to see them edit/add these lines to ''/etc/sympa/sympa/sympa.conf'':
<code>
logo_html_definition <a href="https://<DOMAIN>"><img src="/static-sympa/<FILENAME>" alt="my great logo" style="max-height: 120px; width: auto; border: 0;"></a>
# adjust max-height as appropriate
favicon_url /static-sympa/<BASEFILENAME>.ico
</code>

==== Null Sender ====
=== Defining the problem ===
  * Sympa uses null sender (<>) when sending automated administrative messages like "post rejected: you aren't a member of the list"
  * Amazon SES rejects null sender on SMTP relay (all my testing confirmed this and their AI FAQ thingie says this as well)
  * Sending direct (skipping Amazon SES relay) fails too at least for gmail, who blocks null sender from my server address and accepts other addresses from the same server IP
  * I tried many many attempts to get Postfix to replace the null sender with noreply@ and after hours of this gave up
=== Workaround ===
//yes I know this is a terrible workaround but couldn't figure out anything better//
  * Edit Sympa's ''/usr/share/sympa/lib/Sympa/Mailer.pm'' to use ''no-reply@'' instead of null. In the file:
<code>
#change
($return_path eq '<>' ? '' : $return_path), '--', @rcpt;
#to
($return_path eq '<>' ? 'no-reply@SYMPA_SUBDOMAIN' : $return_path), '--', @rcpt;
</code>
  * //for lawrencemakers' case it's '''no-reply@lists.lawrencemakers.org''//
  * write a simple script to make sure you are notified if ''Mailer.pm'' ever gets updated:
<code>
#!/bin/bash

FILE="/usr/share/sympa/lib/Sympa/Mailer.pm"
TARGET_MD5="<PUT MD5 HERE>"


# Check if the file exists
if [ ! -f "$FILE" ]; then
    /usr/bin/echo "Error: File '$FILE' not found."
    /usr/bin/logger -t checkMailer "Error: File '$FILE' not found."
    /usr/bin/swaks --from FROM_ADDR --to TO_ADDR --h-Subject "checkMailer: FILE NOT FOUND" --body "Error: File '$FILE' not found." --server localhost

    exit 1
fi

# Get the md5 of the file
MD5=$(md5sum /usr/share/sympa/lib/Sympa/Mailer.pm | awk {'print $1'})

# Compare the md5 to the target
if [ "$MD5" == "$TARGET_MD5" ]; then
    /usr/bin/echo "INFO: md5 for $FILE matches target."
    /usr/bin/logger -t checkMailer "INFO: md5 for $FILE matches target."
    /usr/bin/swaks --from FROM_ADDR --to TO_ADDR --h-Subject "checkMailer: md5 matches" --body "Good news: the md5 of $FILE has not changed" --server localhost
else
    /usr/bin/echo "WARNING: md5 for $FILE does NOT match target."
    /usr/bin/logger -t checkMailer "WARNING: md5 for $FILE does NOT match target."
    /usr/bin/swaks --from FROM_ADDR --to TO_ADDR --h-Subject "checkMailer: md5 does NOT match" --body "Bad news: the md5 of $FILE changed. Update null sender now." --server localhost
fi

</code>
  * ''chmod 755'' and set up a daily cron for the script
  * set Sympa to dump any bounced emails to noreply@:
    * in ''/etc/sympa/aliases.sympa.postfix'' add ''no-reply:              "|/bin/true"''
    * ''sudo postalias /etc/sympa/aliases.sympa.postfix''
    * ''sudo postfix reload''
=== Oh wait but we aren't done: Admin email is passing Google DKIM but failing Outlook DKIM ===
Outlook is pretty unhappy with ''Content-Type: multipart/report''. To fix this:
  * change ''/etc/sympa/mail_tt2/delivery_status_notification.tt2'' from ''Content-Type: multipart/report'' to ''Content-Type: text/plain''
  * //To make it look pretty, you will definitely want make some other changes to the template as well//
==== Misc ====
  * It's a good idea to set the domain up with [[https://postmaster.google.com/|Google Postmaster Tools]] for some analytics sending to gmail addresses if you ever need it
  * There's a list template which sets the defaults for new lists when they are created. It's possible to override with a new template. I did not look further into this.
  * completely removing a list needs to happen from the CLI: ''sudo /usr/lib/sympa/bin/sympa.pl --purge_list=LISTNAME@SUBDOMAIN_OF_WEB_UI''
  * From [[https://www.sympa.community/manual/customize/basics-templates.html|the documentation]]: "By default, text body of mail template (except attached part described in above) is wrapped. X-Sympa-NoWrap pseudo-header field prevents line wrapping."