Forging a scalable, maintainable and distributable journal with full control, test coverage and monitoring. In the very first post we'll outline the general setup details as well as create a plan to forge and ship a fully operational Django Sidewinder blog with self-hosted CI infrastructure, satisfactory design, test automation coverage across all environments and two live pre-production servers along with a Docker container and post teaser mirroring to a Telegram channel through a DRF API.
Prerequisites:
- Using Linux
- Be decent in Python
- Be decent in CLI commands
The hardware used:
- Any Linux desktop or laptop PC for the dev environment. I use Arch, by the way.
- Any Linux desktop or laptop PC for the self-hosted runner (running Debian 13 as one of the most stable and easier in setup and maintenance).
- Any Linux desktop or laptop PC for the live pre-production server (running Debian 13 as one of the most stable and easier in setup and maintenance) - could be the same machine as in point 2 if you don't want to use a separate one - and that's what is used here.
- Any Linux desktop or laptop PC for the Allure Docker server (running Debian 13 as one of the most stable and easier in setup and maintenance) - could be the same machine as in point 3 if you don't want to use a separate one - and that's what is used here.
- Any tiny Raspberry Pi or other microcomputer to serve the distilled version via nginx and ready to post to Github Pages, as well as for final UI testing. Running Debian 13 ARM, since it unlocks the 64-bit capabilities of an old RPi 3B+ board's CPU in my case. You may have any other one, the key here is that you don't need to use a lot of processing power for this env.
As you can see, no VPS is needed - the whole infrastructure can be hosted in your own home network.
So first, we build the blog itself on the local dev env. It's the main product and the starting point.
Installation and configuration in Sidewinder docs may seem straightforward, but, as it is the case with many open-source projects, actually isn't. Some critical parts, like the initial database setup, are still missing and they will be covered in the next part of this article series.
After successful configuration and first run, we create the directory and the actual project inside the appname dir. Choosing this way, because renaming everything like the docs suggest is an incredible PITA, don't try it - way easier to add one nested dir to configs.
Django uses the Model-View-Template architecture, so we need to create each of these parts for our blog, as well as wire them in all the appropriate configs (apps.py, admin.py, urls.py (local and general), as well as general settings.py) and perform the migrations. Models and Views are easy to create, but for templates we might require some AI assistance, unless you're a frontend dev, of course.
Then we add markdown support for inline code and code blocks, as well as distillation and fixture loading mechanisms to display the actual content of the posts across all environments.
Second, we build the infrastructure as the skeleton of it all.
A point worth noting: we're using Github Actions CI processes for a Github Pages-based blog. Sure, it could be moved and/or mirrored to Gitlab Pages which has much less draconian quotas and pricing, but that wouldn't be worth the effort. We won't even need to utilize the 2000 minutes/month quota, since it's all gonna be run on a free to use self-hosted runner.
- Local dev environment - we just develop here, no servers need to be hosted on it.
- A Docker container to test the build process and correctness afterwards, as well as distill the website with django-distill and export the artifact downstream, for only static sites can be hosted on Github Pages (still can have JavaScript though). Using the full Python container image because it has git and stuff. The Python version is consistent across all environments which is facilitated by a separate prerequisite CI job. Any failure here blocks the downstream deploy. Using Docker with sidecars and not bare metal because Github Actions doesn't allow to run any systems for setup-python except Ubuntu. Oh well.
- The live pre-production environment with dj-admin and PostgreSQL, Huey + Redis task queuing and a DRF API, as well as a separate manual management command to make Telegram mirror posting easier and more controllable. Here's where the posts will be written from and here's where the integration tests happen, because everything here is live. The self-hosted Github Actions runner is also here, in a separate user directory with limited permissions and no sudo access, as well as Allure Docker service for test reports.
- The Raspberry Pi nginx environment. Here's where the static site will be served and final UI tests happen.
- The Github pages site. A separate public repo which has only the static distilled version hosted. The main repo with the full website is a private one with permission and posting restrictions, even with .envs in .gitignore. Even Github itself recommends it.
- The Telegram channel mirror. Nice to have, but still optional. A post teaser in generated and published the via the Telegram API through the local DRF endpoint on live preprod. A bot via BotFather is needed along with a generated token and channel link.
Third, we formulate a testing strategy and a test plan for each of the environments, write and include tests in the CI pipeline:
- The Docker container tests correctness
- The full live preprod server tests integration of the components
- The RPi nginx based server tests UI and real use cases
Overall, the master plan is:
- Build the blog itself on the local dev env
- Build the infra to be able to add, edit and delete posts
- Add tests into the CI across all environments, for each one serves a separate purpose
P.S. Don't see any reason to use any kind of shift-left stuff here, for it's only useful in big enterprise, when the infra and the actual product is already built and served. We're building from the ground up here:
- Without the product, nothing can be passed through the CI
- Without the CI set up, there's nowhere to put the tests in
- Without the tests, our product might become brittle
Stay tuned for new posts soon (tm) to come.