____________________________________________

NEWSGROUP MODERATION TOOL DEVELOPERS GUIDE

Salahuddin <salahuddin@member.fsf.org>
____________________________________________

<2024-01-21 Sun>

Table of Contents
_________________

1. local DNS server
2. Setup USENET server
... 1. enable IHAVE
... 2. enable incoming
... 3. Server logs
3. Setup USENET client
... 1. IHAVE command not enabled
... 2. Install rnews
... 3. Test rnews
4. Setup local Mail Server
... 1. Setup procmail
... 2. Message formatting
... 3. add custom headers
... 4. add Approved header(s)
... 5. Test Post
5. Simulate newsgroup post
6. Develop incoming mail handler

1 local DNS server
==================

Setup a local DNS server (bind9 or other), and add DNS entries of your
local virtual servers.

usenet-server.home.lab - USENET server
usenet-client.home.lab - client will post news to USENET server via
rnews

note: Please make sure your virtual servers are using your DNS server.

2 Setup USENET server
=====================

usenet-server.home.lab

setup inn2 from source or in Debian based distribution.
,----
| # apt-get install inn2
`----

edit /etc/news/inn.conf
,----
| pathhost: usenet-server.home.lab
| domain: usenet-server.home.lab
`----

create groups
,----
| $ /usr/sbin/ctlinnd newgroup demo.comments
`----

edit /var/lib/news/newsgroups
,----
| demo.comments Demo comments
`----

edit /etc/news/readers.conf
,----
| auth "demo" {
| hosts: "*"
| default: "<demo>"
| }
|
| access "demo" {
| users: "<demo>"
| newsgroups: "demo.comments"
| access: RPA
| }
`----

2.1 enable IHAVE
~~~~~~~~~~~~~~~~

Enable IHAVE to send news from rnews from other client.

edit /etc/news/control.ctl change drop -> doit
,----
| ihave:*:*:doit
`----

2.2 enable incoming
~~~~~~~~~~~~~~~~~~~

edit /etc/news/incoming.conf
,----
| peer other {
| hostname: "HOST_OR_IP_OF_YOUR_CLIENT_WITH_RNEWS"
| }
`----

Replace HOST_OR_IP_OF_YOUR_CLIENT_WITH_RNEWS with your
usenet-client.home.lab name or IP.
It will allow usenet-client.home.lab to post news to
usenet-server.home.lab via rnews.

restart
,----
| # systemctl stop inn2
| # systemctl start inn2
`----

2.3 Server logs
~~~~~~~~~~~~~~~

,----
| # journalctl -u inn2.service
`----

3 Setup USENET client
=====================

usenet-client.home.lab

You may test your USENET server using a USENET client or telnet.

,----
| telnet usenet-server.home.lab 119
| POST
| //It will return you a Message-ID.
| // add Message-ID in your manual post
| Path: hi!not-for-mail
| From: Home <demo@demo>
| Newsgroups: demo.comments
| Date: Fri, 5 Jan 2024 21:29:04 -0000 (UTC)
| Subject: test
| Organization: A poorly-installed InterNetNews site
| Message-ID: <unb5kg$17e$1@usernet-server.dynamic.lab>
| MIME-Version: 1.0
| Content-Type: text/plain; charset=UTF-8
|
| test
|
| QUIT
`----
Now, your message should appear in demo.comments news group.

3.1 IHAVE command not enabled
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

example: error IHAVE command not enabled
----------------------------------------------------------------------
,----
| telnet usenet-server.home.lab 119
| Trying 192.168.0.196...
| Connected to usenet-server.home.lab.
| Escape character is '^]'.
| 200 usenet-server.home.lab InterNetNews NNRP server INN 2.7.1 ready
(posting ok)
| IHAVE <unb5kg$18e$1@usenet-client.home.lab>
| 480 IHAVE command disabled by administrator
`----

note: rnews uses IHAVE command.
Please make sure your server has IHAVE command enabled.

3.2 Install rnews
~~~~~~~~~~~~~~~~~

Install rnews from source or in Debian based distribution.
,----
| # apt-get install inn2-inews
`----

3.3 Test rnews
~~~~~~~~~~~~~~

create test_message.txt
,----
| Path: hi!not-for-mail
| From: home <demo@demo>
| Newsgroups: demo.comments
| Date: Fri, 5 Jan 2024 21:29:04 -0000 (UTC)
| Subject: test2
| Message-ID: <unb5kg$19e$1@usenet-client.dynamic.lab>
|
| test2
`----

post message
,----
| /usr/bin/rnews -h usene-server.home.lab -P 119 -S
usenet-server.home.lab < test_message.txt
`----

verify the message post status using your usenet reader.

4 Setup local Mail Server
=========================

usenet-client.home.lab

Setup local Mail Server for moderation (Sendmail, Exim, others).

warning: *by default mail server disable email exchange for local IP
series.*
If you have local IP series 192.168.0.x, please make sure you allow
this series for mail exchange.

Example: in Exim /etc/exim4/exim4.conf.template
,----
| dnslookup:
| debug_print = "R: dnslookup for $local_part@$domain"
| driver = dnslookup
| domains = ! +local_domains
| transport = remote_smtp
| same_domain_copy_routing = yes
| # ignore private rfc1918 and APIPA addresses
| ignore_target_hosts = 0.0.0.0 : 127.0.0.0/8 : 192.168.0.0/16 :\
| 172.16.0.0/12 : 10.0.0.0/8 : 169.254.0.0/16 :\
| 255.255.255.255
| no_more
| .endif
`----
remove 192.168.0.0/16 to allow local mail exchange.

4.1 Setup procmail
~~~~~~~~~~~~~~~~~~

,----
| add $HOME/.procmailrc
| ------------------------
| DEFAULT=$HOME/Maildir/
| MAILDIR=$HOME/Maildir/
| LOCKFILE=$HOME/.lockmail
| LOGFILE=$MAILDIR/procmail.log
| LOGABSTRACT=yes
| VERBOSE=yes
|
| :0c
| | ../YOUR_INCOMING_MAIL_HANDLER
`----

"YOUR_INCOMING_MAIL_HANDLER" will read mail via STDIN and save for
further processing. You need to notify the sender an acknowledgment of
the submission (automatically or manually).

4.2 Message formatting
~~~~~~~~~~~~~~~~~~~~~~

If your comment or any header line is too long, it should be
reformatted to next line with space(s).
Example:
,----
| HEADER: your long
| comment for moderation
`----

This will fail while posting in newsgroup.

You need to either make the comment shorter or add space(s) as prefix
after header line
,----
| HEADER: your long
| comment for moderation
`----

4.3 add custom headers
~~~~~~~~~~~~~~~~~~~~~~

add your custom headers with the original submission for approval
,----
| X-SCRM-Policy: https://example.com/
| X-SCRM-Info-1: Send submissions to demo@example.com
| X-SCRM-Info-2: Send technical complaints to demo-admin@example.com
| X-SCRM-Info-3: Send complaints about policy to demo-board@example.com
| X-Comment: moderators do not necessarily agree or disagree with
this article.
| X-Robomod: YOUR_MODERATION, demo@example.com
| X-Moderation-1: YOUR moderation tool
| X-Moderation-2: See https://example.com/
`----

4.4 add Approved header(s)
~~~~~~~~~~~~~~~~~~~~~~~~~~

Remove unnecessary header(s) and add approved header before
submission.
,----
| formail -f -I Path: \
| -I X-Moderate-For: \
| -I Return-Path: \
| -I X-Mailer: \
| -I "Date:" \
| -I "X-400-Received:" \
| -I Received: \
| -I "From " \
| -a "Approved: YOUR_APPROVAL_EMAIL" \
| -I Lines: \
| -I Cc: \
| -I Status:
|
| > post_to_send.txt
`----

4.5 Test Post
~~~~~~~~~~~~~

Now you can post to your local newsgroup server via rnews.
,----
| /usr/bin/rnews -h usene-server.home.lab -P 119 -S
usenet-server.home.lab < post_to_send.txt
`----

After successful posting, move the submitted mail submission in a
different directory to avoid re-posting. You may automate the process
by implementing scripts and web-interface.

If you plan to use it in a production/external USENET server, you need
to add your login password in /etc/news/passwd.nntp
,----
| EXTERNAL_USENET_SERVER_HOST:YOUR_LOGIN:YOUR_PASSWORD
`----
Use port with SSL/TLS (563) instead of insecure port 119.

5 Simulate newsgroup post
=========================

You may use usenet-client.home.lab for testing.
Send test message with usenet header (using -a).

You need to set -r --return-address=ADDRESS explicitly. Otherwise, it
will send mail as SENDER_USER@localhost and exim will not deliver
submission response to this account.

Destination "USER" is the account with procmail configured for
moderation.
,----
| $ echo "Test message." | mail -s "Test" -r SENDER_USER@EMAILHOST
'USENETMOD <USER@YOUR_DOMAIN>' -a 'Newsgroups: YOUR_NEWSGROUP'
`----

6 Develop incoming mail handler
===============================

Develop YOUR_INCOMING_MAIL_HANDLER to moderate newsgroup posts.