NFMD: A news feed mailer daemon

NFMD is a program which grabs news feeds from a specified file and sends daily updates via e-mail.

It parses RSS/Atom feeds natively, and for other sources you can pass a service that converts data to RSS i.e. RSS hub instances.

Links:

• nfmd-0.3.deb for Debian 12 Bookworm
• sources

§ Usage

• Get your favourite feeds in a config text file:

18:05 email@foogle.com
[header]
<style>#video { display: none; }</style>
[schemes]
https://t.me https://rsshub.app/telegram/channel
[urls]
# This page provides RSS natively
https://myrrc.dev/rss.xml # This is an inline comment
# This page doesn't so we'll query rsshub.app
https://t.me/red_spades

• Install deb package or compile from source and copy binary to a server
• Run nfmd -c *config* set

§ Why

I follow around 100 Telegram channels and also other sources like YouTube, Reddit, as well as many small websites. At some point I decided I want everything in a single place. Although most people I know use Telegram for that purpose, messenger style doesn’t seem reasonable due to continuous distraction. Even if you mute notifications and archive chats, you still have items to click through. This is solved in Telegram mobile with swipe up gesture, taking you to next channel or dialog, however, on desktop there is no such functionality. Another idea is to separate messaging and news reading as for me these are two different activities.

Therefore another solution was searched. The easy way was to convert everything to RSS. There are many clever apps like ratt but the need to write manual rules was a bit frustrating, so I discovered RSS hub. A simple and elegant one - pass a link, get RSS. If you don’t want to rely on public instances, host your own.

Next bike-shedding step was to find an RSS reader. Most of my computer time except for the browser is spent in the terminal. Terminal readers like photon may be awesome, and I tried using them, but there still are downsides:

• You need to have the reader up and running in order to fetch new data.
• Rss readers don’t usually have a good way to aggregate data per day.
• In rare cases when you need to read from mobile, you need another app.

The final decision was to use e-mail. It’s simple, it handles unread items, you can perform fetching and filtering on the server in case your computer is turned off.

§ How

The initial plan included writing a daemon which would wake up at specified time, fetch sources from a TOML file in parallel, filter them to get last ones only, convert to HTML and send an email to me. I chose Rust, picked up some libraries and made a prototype. It was… big, around 20MB total. Playing with musl builds, removing libunwind, no luck. Such a simple program definitely shouldn’t be that big.

So I reconsidered the functionality:

• Removed all authentication. I run a Postfix/Dovecot e-mail server so program can just send to localhost.
• Removed complex wakeup rules. I realised I really wanted news only one time a day
• Removed tokio and parallelism. One consideration was the complexity - even if serial execution is way slower¹, I need the program to be executed one time a day only. Second - multiple concurrent requests in a short period can lead to being blocked by source servers. Third - my server has only one core, so many threads wouldn’t bring much benefit.
• Removed TOML in favor of plain text. With a list of URLs, any markup is excessive.
• Replaced manual demonization with cron. In most systems cron or anacron is available, and you can use it for 1) reliable wakeups, and 2) restarts if program crashed. Also, a long-running program is something that may cause pain if there’s a memory leak or some other bug.

After refactoring, the following dependencies were left:

• data fetching via curl bindings.
• email sending via lettre
• data parsing via rss and atom

Then I read curl can send SMTP requests, however, the Rust crate didn’t have this, so I decided to switch to C².

§ Bells and whistles

The next prototype was way faster and smaller, around 14kb, but still didn’t feel ideally comfortable. I added missing functionality:

• Comments after URLs for tagging various sources
• Prefix replacement so that you could add https://t.me/tokacomics and not https://rsshub.rss.tips/telegram/channel/tokacomics.
• E-mail styling via inline CSS: customizable, my styles specifically were width formatting mostly:

a { text-decoration: none; }
video { display:none; }
body,img { width: 600px; }
body {
    padding-left:25%;
    padding-right:25%;
}

§ Packaging

Debian packages as they’re 1) easy to build via pbuilder 2) easy to install on target server 3) (relatively) easy to write.