Riak Core on Partisan on Elixir Tutorial: Introduction

What is Riak Core?

(From the Partisan website)

Riak Core is a distributed programming framework written in Erlang and based on the Amazon Dynamo system that influenced the design of the distributed database Riak, Apache Cassandra, and the distributed actor framework Akka.

In Riak Core, a distributed hash table is used to partition a hash space across a cluster of nodes. These virtual nodes—the division of the hash space into N partitions—are claimed by a node in the cluster, and the resulting ownership is stored in a data structure known as the ring that is periodically gossiped to all nodes in the cluster.

Requests for a given key are routed to a node in the cluster based on the current partitioning of virtual nodes to cluster nodes in the ring structure using consistent hashing, which minimizes the impact of reshuffling when nodes join and leave the cluster. Background processes are used for cluster maintenance; ownership handoff, (transferring virtual node ownership) metadata anti-entropy (an internal KVS for configuration metadata) and ring gossip (information about the cluster’s virtual node to node mapping).

What is Partisan?

(From the Partisan website)

Partisan is the design of an alternative runtime system for improved scalability and reduced latency in actor applications.

Partisan provides:

• Better scalability by leveraging different network topologies for communication
• Reduced latency through efficient parallel message scheduling for actor-to-actor communication

Partisan is provided as a user library in Erlang and achieves up to an order of magnitude increase in the number of nodes the system can scale to through runtime overlay selection, up to a 34.96x increase in throughput, and up to a 13.4x reduction in latency over Distributed Erlang.

Why Partisan?

I think it's an interesting project, and providing an easy way to set up a useful example on top of it may help more people learn about it, take advantage of it and maybe contribute to it.

Check the Scaling Riak Core section on the Partisan page for more information about the benefits of using Partisan.

I should mention that this fork is not the official one, so consider this if you want to put something based on this stack in production.

How to Follow this Tutorial

The tutorial provides the commands and code you need to add and change, so you can follow it by copying and pasting it in the right place.

It also provides a repository with one tag for each step, the tag corresponding to each step will be linked near the code changes.

Following the tutorial it's recommended that you don't change the project's name unless you want to make lot of renames and maybe get some errors if you forgot one. Of course for a real project based on this code you can rename it, usually you have to find and replace all mentions of 'civile', 'Civile' and 'civiledb'.

Here's the link to the project: https://gitlab.com/marianoguerra/civiledb/

Here's the link to the project tags: https://gitlab.com/marianoguerra/civiledb/-/tags

To follow the tutorial from the example repository you will need to have git installed, check the Git Website for instructions on how to install it.

You can clone the repository with the following command:

git clone https://gitlab.com/marianoguerra/civiledb.git

To change to another tag (replace start-1 with the tag you want to change to):

git checkout start-1

To go back to the last change:

git checkout master

Resources

This tutorial is based on many resources around the web, here's an incomplete list of them, some of them provide more details, theory and explanation, some are in Erlang, some don't run any more, but you can get something extra out of each of those.

• GPad's "Create a riak_core application in Elixir" Blog Series
• Riak Core Tutorial
• Little Riak Core Book
• Ryan Zezeski's Riak Core Blog Series

Next post: Riak Core on Partisan on Elixir Tutorial: Setup.