Deploy RSS with K8S as a full self-hosted solution - RssHub + Tiny Tiny RSS

preface

What is RSS?

RSS is a format for describing and synchronizing website content and is the most widely used XML application. RSS provides a technological platform for the rapid dissemination of information, making everyone a potential provider of information. After publishing an RSS file, the information contained in this RSS feed can be directly called up by other sites, and since this data is in standard XML format, it can also be used in other terminals and services, which is a format for describing and synchronizing website content.

RSS is widely used in online news channels, blogs, and wikis. Using RSS feeds provides faster access to information, and websites provide RSS output, which helps users get the latest updates to the content of the site. Web users can read RSS-enabled website content on the client side without opening the site content page with the help of RSS-enabled aggregation tool software.

Why do I still need RSS?

Two purposes:

1. RSS and RSS readers as efficient feed aggregators eliminate the time spent browsing various websites every day to search for information, discover high-quality content, and improve reading efficiency.
2. Through RSS feeds, get low-frequency but important information, such as important software updates, forecasts and warnings.

Introduction to Tiny Tiny RSS

Tiny Tiny RSS is a free and open source RSS aggregation reader based on PHP. Requires self-hosting and deployment as a web-based RSS reader.

Introduction to RssHub

RSSHub homepage

RSSHub is an open source, easy-to-use, and easily extensible RSS generator that generates RSS feeds for any quirky content. RSSHub is rapidly growing with the power of the open source community and is now compatible with thousands of items of content on hundreds of websites

Main Purpose:

• Convert non-RSS formats to RSS for subscription; As Slogan puts it, “🍰 Everything can RSS.”

• Discover more interesting feeds.

For example, my subscription through RssHub is:

• Bilibili
• InfoQ Hot Topic
• Local treasure spotlight information
• Douban - a high-scoring movie that is being released
• Notice of power outage in the urban area
• N-card driver update
• …

It is used as a repository with an aggregate lot of content RSS feeds.

📚️ Reference:

For example, I want to subscribe to the timeline of a user on Twitter called DIYgod

according to Twitter user time line by document, routed as /twitter/user/:idhandful :id with the user name, the path is /twitter/user/DIYgod, plus the domain name https://rsshub.app, a feed generates:https://rsshub.app/twitter/user/DIYgod(opens new window)

Then we can put https://rsshub.app/twitter/user/DIYgod Add to any RSS reader (of course, it can also be Tiny Tiny RSS).

where domain name https://rsshub.app Can be replaced with you Self-deploying domain name

In addition, if you need to subscribe to some specific content, such as:

• bilibili content that users (myself) care about
• Weibo Personal timeline

RssHub needs to be deployed and configured separately.

Deployment schema

Overview

Tiny Tiny RSS has a public HTTPS domain name (e.g.:https://ttrss.e-whisper.com), I log in directly to the domain for RSS reading;

Tiny Tiny RSS feeds can come from:

1. Sites that support RSS, such as:OpenShift blog The corresponding RSS address is:https://cloud.redhat.com/blog/rss.xml
2. The RssHub I deployed myself, the public network HTTPS domain name is:https://rss.e-whisper.com

Tiny Tiny RSS

1. Tiny Tiny RSS is deployed in a K8S cluster rss ns inside;
2. Based on Awesome-TTRSS/docker-compose.yml at main · HenryQW/Awesome-TTRSS (github.com), the components that need to be deployed are:
   1. tiny tiny RSS, which needs to have a PVC store for icons
   2. The database for tiny Tiny RSS, PostgreSQL 13, requires a PVC store for database data.
3. The components are all single-node deployments, regardless of high availability;
4. Tiny Tiny RSS publishes domain names through Ingress + SVC;

RssHub

1. RssHub is deployed in a K8S cluster rss ns inside;
2. Based on RSSHub/docker-compose.yml at master · DIYgod/RSSHub (github.com) , the components that need to be deployed are:
   1. rsshub
   2. browserless chrome
   3. redis, which requires a PVC store for cached data.
3. The components are all single-node deployments, regardless of high availability;
4. RssHub publishes domain names through Ingress + SVC;

Prerequisites

1. You need to have your own domain name, such as:e-whisper.com, the specific domain name is:
   1. ttrss.e-whisper.com
   2. rss.e-whisper.com
2. And the domain name has been filed in China, and ports 80 and 443 can be used normally;
3. The domain is hosted on DNSPod or similar DNS providers and can easily modify the DNS Record;
4. You need to have a certificate corresponding to the domain name, this time you need to have:ttrss.e-whisper.com and rss.e-whisper.com , which can be a single-domain certificate or a wild-domain name certificate.
5. The K8S cluster has been set up
6. K8S clusters have Ingress Controllers
7. K8S clusters have StorageClass or can provide PV storage. (This article provides K8S clusters by default.) local-path storageclass)
8. This time, the two domain names were configured through K8S Traefik’s IngressRoute, configuring Ingress and certificates;
9. Installed:K8S Utility No. 5 -kompose - Dongfeng Weiming Technology Blog (e-whisper.com), the tool is used to convert docker-compose.yml Convert to K8S yaml quickly

📚️ Reference:

You can request a free certificate on Letsencrypt for dnspod via cert-manager:

• Use cert-manager to issue a free certificate for dnspod’s domain name Written by Roc

implement

Tiny Tiny RSS build

that docker-compose.yml Over here:

Awesome-TTRSS/docker-compose.yml at main · HenryQW/Awesome-TTRSS (github.com),

1. Modifications docker-compose

There are 2 places that need to be modified:

1. Environment variable:
   1. SELF_URL_PATH=https://ttrss.e-whisper.com/ (your own domain name)
   2. DB_PASS=changeit (PostgreSQL database password)
2. use kompose Conversion, before conversion, needs to docker-compose.yml Supplement the relevant information to ensure the success of the K8S service conversion, specifically by adding it to each Docker Compose service ports Field.docker-compose.yml The changes can be found here:

version: "3"
services:
  service.rss:
    environment:
      - SELF_URL_PATH=https://ttrss.e-whisper.com/ # please change to your own domain
      - DB_PASS=changeit # use the same password defined in `database.postgres`  
...
  service.mercury:
    ports:
      - 3000:3000
...
  service.opencc: # set OpenCC API endpoint to `service.opencc:3000` on TTRSS plugin setting page
    ports:
      - 3000:3000
...
  database.postgres:
    environment:
      - POSTGRES_PASSWORD=changeit
    ports:
      - 5432:5432      

For easy viewing, the relevant yaml files are placed here:east4ming/rsshub-ttrss-k8s-deploy (gitee.com)

Modified docker-compose.yml Check it out here:ttrss/docker-compose.yml · east4ming/rsshub-ttrss-k8s-deploy - CodeCloud - Open Source China (gitee.com)

2. Use kompose conversion

The command is as follows:

at docker-compose.yml Execute in the directory:

kompose convert -o ./k8s/ --pvc-request-size 2Gi

📝 Note:

--pvc-request-size 2Gi Adjust as needed.

After conversion, the directory structure is as follows: (NetWorkPolicy files will also be generated after conversion, and I personally delete the relevant files and labels if they think it is not necessary; In addition, the generated file contains fields that contain . , just in case, all replaced with -):

└── ttrss
    ├── docker-compose.yml
    └── k8s
        ├── database-postgres-claim0-persistentvolumeclaim.yaml
        ├── database-postgres-deployment.yaml
        ├── database.postgres-service.yaml
        ├── feed-icons-persistentvolumeclaim.yaml
        ├── service-mercury-deployment.yaml
        ├── service-opencc-deployment.yaml
        ├── service-rss-deployment.yaml
        ├── service.mercury-service.yaml
        ├── service.opencc-service.yaml
        └── service.rss-service.yaml

In addition, you need to manually create an ingress for external exposure services (in this case, Traefik CRD - IngressRoute).

1. Tiny Tiny Rss
   1. Deployment: service-rss-deployment.yaml (🐾 Here you also need to add an ENV: DB_HOST to access the DB via DB SVC)
   2. Service: service.rss-service.yaml (Used to connect with Ingress and provide external services)
   3. IngressRoute: ingress-rss-service.yaml
2. DB - PostgreSQL
   1. Deployment: database-postgres-deployment.yaml
   2. SVC: database.postgres-service.yaml (Tiny Tiny RSS connects to the DB through this SVC)
   3. PVC: database-postgres-claim0-persistentvolumeclaim.yaml(Apply for persistent storage)
3. Other services - OpenCC
   1. Deployment: service-opencc-deployment.yaml
   2. Service: service-opencc-deployment.yaml
4. Other services - mercury
   1. Deployment: service-mercury-deployment.yaml
   2. Service: service.mercury-service.yaml

The specific K8S yaml content can be found here:ttrss/k8s · east4ming/rsshub-ttrss-k8s-deploy - CodeCloud - Open Source China (gitee.com)

📝 Note:

ttrss/k8s/ingress-rss-service.yaml · east4ming/rsshub-ttrss-k8s-deploy - CodeCloud - Open Source China (gitee.com) The configuration is inherited from another of my articles:Traefik-based Radical TLS Security Configuration Practice - Dongfeng Weiming Technology Blog (e-whisper.com)》

3. Deployment

use kubectl Deploy:

kubectl -n rss create -f ./k8s/

4. Configure DNS Record

In the DNS provider console (in this example, DNSPod), configure the DNS Record for the corresponding domain name <ttrss.e-whisper.com>:

• ttrss, A record, pointing to the public network address corresponding to the K8S cluster Ingress

5. Access Authentication

Visit the home page:https://ttrss.e-whisper.com/ , Default Account:admin Password:password, please change it as soon as possible.

The effect is as follows:

RSSHub build

The steps for building RSSHub are almost the same as for Tiny Tiny RSS. The details are as follows:

1. Modifications docker-compose

use kompose Conversion, before conversion, needs to docker-compose.yml Supplement the relevant information to ensure the success of the K8S service conversion, specifically by adding it to each Docker Compose service ports Field.

2. Use kompose conversion

The command is as follows:

at docker-compose.yml Execute in the directory:

kompose convert -o ./k8s/ --pvc-request-size 2Gi

📝 Note:

--pvc-request-size 2Gi Adjust as needed.

After conversion, the directory structure is as follows:

.
├── docker-compose.yml
└── k8s
    ├── browserless-deployment.yaml
    ├── browserless-service.yaml
    ├── redis-data-persistentvolumeclaim.yaml
    ├── redis-deployment.yaml
    ├── redis-service.yaml
    ├── rsshub-deployment.yaml
    └── rsshub-service.yaml

1 directory, 8 files

In addition, you need to manually create an ingress for external exposure services (in this case, Traefik CRD - IngressRoute).

1. RssHub
   1. Deployment: rsshub-deployment.yaml
   2. Service: rsshub-service.yaml (Used to connect with Ingress and provide external services)
   3. IngressRoute: rsshub-ingress.yaml
2. Browserless - Chrome
   1. Deployment: browserless-deployment.yaml
   2. SVC: browserless-service.yaml (RssHub connects to Browserless through this SVC.)
3. Redis
   1. Deployment: redis-deployment.yaml
   2. Service: redis-service.yaml
   3. PVC: redis-data-persistentvolumeclaim.yaml

The specific K8S yaml content can be found here:rsshub/k8s · east4ming/rsshub-ttrss-k8s-deploy - CodeCloud - Open Source China (gitee.com)

📝 Note:

rsshub/k8s/rsshub-ingress.yaml · east4ming/rsshub-ttrss-k8s-deploy - CodeCloud - Open Source China (gitee.com) The configuration is inherited from another of my articles:Traefik-based Radical TLS Security Configuration Practice - Dongfeng Weiming Technology Blog (e-whisper.com)》

3. Deployment

use kubectl Deploy:

kubectl -n rss create -f ./k8s/

4. Configure DNS Record

In the DNS provider console (in this case, DNSPod), configure the DNS Record for the corresponding domain name <rss.e-whisper.com>:

• rss, A record, pointing to the public network address corresponding to the K8S cluster Ingress

5. Access Authentication

If you see this page, it proves that the deployment has been successful:

You can subscribe to the RssHub feed through TTRss’s page to verify that RssHub is running normally, as shown below:

After clicking Subscribe, it is successful, as shown below:

Proof that RssHub is up and running.

🎉🎉🎉

summary

With the above configuration, we can subscribe and read messages through our browser-based Tiny Tiny RSS reader, and can convert a wide variety of information into subscribeable RSS routes through RssHub.

Resources

• RSS (Simple Information Aggregation)_Baidu Encyclopedia (baidu.com)
• Introduction | RSSHub
• 🐋 Awesome TTRSS | 🐋 Awesome TTRSS (henry.wang)