Table of Contents
There are two common ways to make sure your hardware and software works as it should: collecting logs and collecting metrics. I use a self-hosted Elasticsearch database to store logs and I use Kibana to visualize it. It’s pretty easy to set up and it requires little maintenance, but the things aren’t that bright when it comes to collecting and plotting metrics data.
Elastic stack is good at handling logs and there is a reason for that: it’s main purpose is to process text and everything else is a kind of a hack. That’s why I use InfluxDB as a metrics solution: it’s focused on time-series data such as metrics events and it handles it really well. The data can be visualized with a separate tool such as Grafana and they play well together.
So, InfluxDB + Grafana seems like a good solution for all of my metrics aggregation needs but, it turns out, I won’t need Grafana pretty soon, thanks to InfluxDB v2. The second major release of InfluxDB comes with a nice frontend that can be used to set up all of my metrics using a simple web interface so there is no need to install anything else.
Oh, and it looks pretty cool:
Self-hosting enables us to take back control over our data but you might end up shooting yourself in the foot if you’re not careful. Things happen: hardware failures, house fires, floods, tornadoes, earthquakes, you name it. We don’t like to think about the bad stuff but people who’re serious about self-hosting should really try to think like sysadmins and it means that data redundancy should be one of your biggest concerns.
Storing data on a single device is risky and the simplest way to mitigate the risk of hardware failure is to buy an additional “backup” HDD or SSD, depending on the amount of data you need to backup and the amount of money in your wallet. A more advanced user can automate this routine with a simple script and by setting up a local NAS. Such a scheme can protect you from an accidental storage device failure but some events, such as house fire or tornado, can destroy both of your storage devices simultaneously and that’s why we also need “location redundancy”.
Storing your data in a separate location is not hard but it comes with a different set of concerns. With a remote backup, you better have a good internet connection and you should also trust the remote side, at least to a certain degree.
So, here comes the need to minimize that trust. Of course, it’s better to choose a reliable data storage provider and most of them won’t lose your data because they’re professionals and they know how to handle your data but they can snoop on it or they can be hacked so your data might end up in the wrong hands. That’s the real risk and that’s why it’s important not to store your backups in plaintext, anything that leaves your house should be encrypted in order to make it unreadable by anyone except yourself.
That’s quite a long introduction, I generally like to be verbose, it helps me to systematize my thoughts. So, apparently, there is a need for a remote backups which are unreadable by a remote storage provider. Is there a tool for that?
Here comes duplicity. This tool can do incremental backups and it also encrypts the data before syncing it with the remote servers. It supports many storage providers, including S3-based offerings such as this nice and shiny bunker 25 meters below Paris. I played with this tool for a few days and I really like how it works.
My password database requires hardware token to access it, which adds an extra layer of security but it also means spending more time to access the passwords I frequently need.
To solve this issue, I started to “cache” the least critical and most frequently used passwords with a tool called pass. I like it a lot: it’s open source, it has an easy to use CLI with auto-completion and it doesn’t store passwords in plaintext.
Podcast: What Bitcoin Did
I’ve checked out this podcast a couple of weeks ago and it seems to be balanced enough to serve as one of the main sources of bitcoin news for a person with an interest in bitcoin high enough to be in the loop but without getting too technical.
Podcast: 2.5 Admins
2.5 Admins is a new podcast by Joe Ressington from (now cancelled) Linux Action News. There is only one full episode available at the moment but I liked it and I think that the next episodes will be interesting too. This podcast might become a good source of tech news and sysadmin gossip.
Movie: Tiger King
It looks like the whole world have been watching this movie lately and there are zero reasons not to. I still can’t believe that this “Tiger King” guy is real. Movies like this keep me from canceling my Netflix subscription, those guys are onto something and they just continue to up their game.
Movie: Operation Odessa
I avoided clicking on this movie in my recommendations for a couple of years because, for some reason, I thought that it’s some kind of a political drama. Far from it! It’s a criminal documentary and it’s actually pretty funny.