This post is not going to be about what’s wrong with individual databases. There are so many databases and so many individual API issues, operational problems, and arbitrary limitations that it would be take forever to cover all of them. This post is about what’s wrong with databases as a collection. It’s about what’s wrong … Continue reading Everything wrong with databases and why their complexity is now unnecessary

There are a number of properties that are non-negotiable in any software system, such as no data loss, no stalling, and timely recovery from faults. These properties are particularly difficult to achieve in a distributed system, as the vast number of permutations of timings, event orderings, and faults makes reasoning about a distributed system extremely … Continue reading How Rama is tested: a primer on testing distributed systems

Today we’ve released Rama’s Clojure API, including detailed reference documentation and API docs. Information about how to add it as a dependency in your projects is on this page. Rama is a new programming platform for building high-performance scalable backends, integrating and generalizing data ingestion, processing, indexing, and querying. It’s so general-purpose that it can … Continue reading Introducing Rama’s Clojure API: build end-to-end scalable backends in 100x less code

I’m going to cover a lot of ground in this post, so here’s the TLDR: We built a Twitter-scale Mastodon instance from scratch in only 10k lines of code. This is 100x less code than the ~1M lines Twitter wrote to build and scale their original consumer product, which is very similar to Mastodon. Our … Continue reading How we reduced the cost of building Twitter at Twitter-scale by 100x

At Red Planet Labs we’ve been quietly developing a new kind of developer tool for many years. Our tool reduces the cost of building large-scale end-to-end applications by multiple orders of magnitude, and Clojure is a big reason why we’ve been able to tackle such an ambitious project with a small team. Our codebase consists … Continue reading Tour of our 250k line Clojure codebase

Testing distributed systems is hard. You probably already knew that. If you’ve spent even a little time writing distributed systems, you know that the relevant algorithms and techniques are complex and nuanced. Distributed systems end up full of gotchas and corner cases. The tests for these systems are even worse! If you actually care that … Continue reading Where we’re going, we don’t need threads: Simulating Distributed Systems

Dynamic, functional programming languages like Clojure are sometimes considered "slower" than their statically-typed and/or OO counterparts, due to facilities like dynamic function dispatch and immutable-orientation, etc. But this comparison is too general. Clojure is readily optimizable in cases where utmost performance is key. Clojure embraces the "Get it working, then make it fast" approach to … Continue reading Clojure, Faster

We use Nippy at Red Planet Labs whenever we need general purpose serialization / deserialization of Clojure objects. It's easy to use, fast, and covers virtually everything you could ever want to serialize in a Clojure program. One notable exception is Clojure fns. There are several places in the distributed computing application we're developing where … Continue reading Serializing and Deserializing Clojure Fns with Nippy