Table of Contents
I finished cleaning up the codebase and I’m working on a few performance optimizations for the most frequent tasks, such as:
- Fetching preview images for entries
- Importing feeds from OPML files
- Fetching new and updated entries in standalone mode
Some of those tasks aren’t using concurrency at all and some of them aren’t using it to its fullest potential. The initial prototype didn’t use concurrency, but I quickly switched to batch processing in order to speed things up a bit. The problem with splitting HTTP queries in batches is the fact that you still need to process those batches one by one, so every batch has to wait for it’s slowest query to complete. That might sound like a minor downside, but it can cause massive damage to app performance.
The better approach is to have a few concurrent workers crunching through a dynamic pool of jobs. For instance, this website has ~100 posts and fetching their Open Graph images takes ~20 seconds when I process them in batches of 10. Using a shared pool of jobs and 5 concurrent workers helped me to do the same task in just 2 seconds.
The key difference between those approaches is that the latter one guarantees that all the workers will be utilized all the time. A couple of slow queries can block some of those workers but they can’t paralyze the whole pipeline, which leads to a huge performance boost.
There are many ways to implement such a feature, but I used this situation as an excuse to play with Kotlin Channels. Here is the quote from the doc:
Multiple coroutines may receive from the same channel, distributing work between themselves
This approach is called fan-out and this term seems to be borrowed from digital electronics. I like this word, and this approach ended up working pretty well for the uses mentioned above.
I have a lot of Android phones and my favorite one is Xiaomi Mi A1. It’s a 5-year-old phone and it’s running the latest Android, thanks to LineageOS. This phone stopped getting official updates years ago, but its hardware has no issues running modern Android apps. It’s still blazing fast, which is a good reminder that 2-year and even 5-year support windows are indefensible, we should be able to use our hardware as long as it works, which can be close to 10 years or even more.
I’ve been studying US history recently, and I must say, it’s pretty different from what Russian education system frames it to be. It’s no surprise, but some takes are just unbelievably cynical. For instance, the Soviet and post-Soviet version of US History claims that Americans are descendants of criminals who tried to escape justice in the New World. This may sound funny until you try to fully realize how dangerous those delusions may be if they are held by hundreds of millions of people.
Of course, not everyone buys this narrative and most people don’t really care, but reading about more objective takes on US history with that in mind is quite an amusing experience. I didn’t know that some original British colonies were in fact deeply religious communities with an extremely low crime rate.
Book: Beej’s Guide to Network Programming
I don’t use C professionally, but I enjoy reading about low-level APIs and low-level documentation and code usually do a much better job at explaining how things work, and why they work in certain ways. This book is focused on Internet sockets and the modern ways of processing them in batches. Things like
libevent are the secret sauce which enables higher-level languages to have non-blocking IO. Those primitives power some cool things such as Kotlin coroutines and async Rust, so this book can be of interest to anyone who’re curious to see how things work under the hood.