On the beginning…

I wrote this blog post for myself. Myself from 10 years ago, to be precise. This is a post I wish I could send back in time and show to myself. I’m sure it would change my direction and career. If you have heard about Erlang but are not sure if it is worth giving it a try…or you just love Erlang and would like to hear my praise for it, please read on. I really hope this will be pleasant and fun read as it was for me writing it.

I have been professional software developer for a long time now. Double this time for developing for fun, hobby and well… just myself. In a recent years I have questioned myself and have realized that pretty much all of the languages that I used or currently use are imperative ones. That question led to another - why is that so?

The answer is kinda complicated but it is so mostly because this is how it used to be for me. It was my status quo. This has to do with very few books that were accessible for me when I was beginning, the university courses, the languages that everyone around used…you get the point. This was clear, understandable and directly mapped into how computers work, use memory and process information. It was easy and natural, right? I learned it, learned it well and it was easy for me to learn any new imperative language. My world was warm and cozy until I felt some urge, some calling, hard to describe really. Suddenly I wanted to learn and see something completely different. Of course I knew about different paradigms, mostly functional programming, but just never had a chance and motivation to dig into them. Why would I? And oh my, was I wrong not doing that many years ago. I took the red pill, took the risk. I joined a bunch of Erlang and Elixir magicians working on world class product. A scale I’ve never ever had a chance to even imagine in my previous projects. I saw something incredible and amazing, a living system, so stable, scalable, readable and maintainable that completely changed my perspective and knowledge. I had a pleasure to add some big chunks of new functionality, do refactors, write hundreds of tests and ultimately manage it on production as part of DevOps culture.

This post is about my mind shift and about my thoughts and notes from that experience. I hope you’ll enjoy it as much as I do and if you are thinking about trying Erlang, it will make your decision easier.

On simplicity…

The first thing that really struck me was how easy and simple Erlang language is. It was especially visible as before using Erlang I had been trying to catch up to the newest C++ revisions (17 and newer). In Erlang, there are less than 10 data types: something for numbers, for booleans, symbols (called atoms), maps, lists, tuples of arbitrary sizes and binary/bit storage that suits our string and binary needs. Yet these data types hold their ground and values without any problems. I never had any second thoughts, problems or hesitation what to use. You can combine them to produce more complex structures and they still keep their simplicity. This maps directly to how readable the code and concepts are. The only thing that made me think were atoms. They are interesting.. basically constant pointers into table which contains immutable words (or strings). By default you can have 1,048,576 of them. They are not garbage collected and live with your application for its whole life. They can be directly used in code as literal constants to provide you with domain names for structures, identifiers for whatever you want, return codes, enumerations…well they are ubiquitous. They are so simple yet so powerful. I really bow before the people who invented this concept. For example let’s say that we want a tuple that holds tire size of a vehicle. We can create this tuple simply by:

{ 305 }.
And some cars have different sizes of front and rear tires, for example:
{ 265, 305 }.
These tuples are correct but we can and should enrich them with atoms to provide much more meaning and usability when processing them later on:
{tire_sizes, {front, 265}, {rear, 305}}.
We can construct tagged tuples matching our needs and they will be very obvious to the reader:
{tire_sizes, {front_and_rear, 305}}.
{ok, Result}, {error, Reason}.
Here we have a tuple with 2 internal tuples. All of them are identified by atoms (tire_sizes, front, rear and front_and_rear). This is much more readable and very easy to extract and use. The only small downside of atoms is that they are just literals. You scatter them all over your code base, you need to remember them and it is quite easy to misspell them, which won’t be detected by compilation. This issue can be partially addressed by Dialyzer and typespecs. More on that later. Good tests however mitigate it and as Erlang is a dynamically typed language, tests are absolutely mandatory. I guess you can slip by without tests in Java (not that you should), but in Erlang in my opinion this is not an option. And writing tests in Erlang is a pleasure, but more on that later on…

On modelling functional domains…

Ok, so language is easy. What about code structure? Are there classes? How can we model and express our domain? Again, things are simple. There are modules which are single files in which we define functions and our custom data types. Module name which is the same as file name becomes point of encapsulation because we decide what functions we want to export to be visible to other modules. Module also becomes namespace further encapsulating functions. To call function from module you just write ModuleName:function and that’s it. There are no classes, no inheritance. Do I miss them? Hell NO!. To be honest, Erlang brought me a gust of fresh air. No classes, no inheritance, no polymorphism in a classical-heavy sense (we can still easily get dynamic dispatch and polymorphic execution thou), just simple modules with encapsulation. Honest opinion continued: It is so big relief after years with SOLID, GRASP and thinking about countless patterns and idioms. In my opinion regarding Object Oriented Analysis and Design people start to think too much. OOA&D has been present for many years and in so many projects that a lot of best practices and patterns emerged and yet still good programmers and architects want to extend it, to contribute, to add something new that would make their name in the industry. This is understandable but it complicates things even more. Not to mention that there are fundamentalists that are eager to say that some solution is not conforming to standards or idioms. Programming becomes like maneuvering around minefield of rules. Code Reviews become opportunities to show noobs how bad they designed something, waiting for anyone to slip, to attack.

Then there is a whole world of Domain Driven Design. It is massively complicated and you can get so many things wrong. So designs are getting bigger, set of rules longer and longer, there are traps everywhere, languages are becoming more complex - simple things stopped being expressed as simple solutions. Now, don’t get me wrong - I’m not negating these techniques and rules - they are here for a reason - if done right they provide clear, solid, maintainable and testable product. Complicated domain expressed in DDD is a pleasure to work with. But I’m really talking about the cost and complexity to reach this state. Yet Erlang does not have all of this and still manages to achieve that. How? By keeping things simple. Where I lost fun and enjoyment in quite rigid procedural world environment, I found them in functional one of Erlang’s simplicity. This is what probably hit me the most - that complicated domain which ultimately required quite big project could be expressed so nicely and cleanly.

To further explore this topic I strongly recommend watching this video in which Bryan Hunter presents CQRS in Erlang. Watch how simpler and more beautiful than OOP it is in Erlang:

You can find Scott’s book at The Pragmatic Bookshelf.

Complicated abstractions around automatic Dependency Injection and constructors/setters? Not needed. Big heavy frameworks to provide functionality? Not needed. Strange, complex, brittle hierarchies? Not needed. Thinking and worrying about state and multithreading? Well, simplified by a magnitude :). Ability to test stuff completely? At will and without any problems. All in all, creating and expressing functional domain, while being hard at the beginning turned out to be just perfect.

On functional flow…

Simple, effective, side-effects-free functions that comprise Erlang program quickly became my new thinking mode. I have found out that I create them almost automatically. Of course the most important thing here is proper naming - it is super important to keep function names to express their operations. Seeing experienced functional developers at work I saw that “the functional flow” can be much more effective than mine, and it was great and a big motivation! Functions should be simple - for the same input they must produce the same output - this is one of fundamentals of functional programming (referential transparency). They shall not depend on any external state nor modify it. Of course this can be problematic in practice but Erlang is a pragmatic language. It allows to bend the rules slightly, for example with ETS which is kinda shared storage. It may seem to be like static public shared data table and it can easily be misused. But one of the few Erlang design patterns describes proper usage of ETS and well, when you stick to it, it will be most useful in creation of simple and robust modules.

Is domain designed in Erlang by composing bunch of functions better than properly designed domain in OOP/DDD? By better I mean more readable, more aligned to real domain and more maintainable? In my opinion, no, it is not, but it’s very very close and by not bringing any problems to the table it gives back so much excellent things from functional programming world.

On immutability…

Erlang variables can be only in 2 states: Unbound which is waiting to be assigned a value from pattern match or assigned (Bound). Once assigned it can never change. You need new value - you need to create new variable. This is fundamental aspect of functional programming. It makes programming harder at first and seems ineffective but bear with it and it will become apparent how beneficial this is. Program analysis and debugging becomes so easy - nothing can change variable value so you don’t have to look for any external forces - you analyse a couple of functions. In the long run I suspect this was the biggest-effect factor of functional programming that I loved. Reasoning about functionality or finding bugs becomes straightforward if this is in place.

On the Actor Model…

Erlang’s actor model requires new approach to the thinking and modelling of solution to the problem we are trying to solve. In short, this boils down to divide and conquer the problem to simple processes and modeling how they interact. When you get some experience with that it can be quite fun and powerful. Mapping to the code then is direct. In the times of multicore processors this concurrency model is most probably one of the best. I don’t want to bash Java again but yeah, I saw custom, home made, low level multithreaded code too many times. Of course it rarely worked as expected. In Erlang, you model processes, messages, workflows. Then compose process supervision trees, decide what happens if things go wrong and let it spin. Process owns its data which is accessible only from inside of it, and messages are processed sequentially. There is no need for synchronization, critical sections or any other low level constructs. In result, it is much harder to make mistakes, regarding multithreaded code and synchronization in actor model. Of course language has direct support for sending and receiving messages, which makes everything smooth and readable. Great help comes from error handling which is very well designed in Erlang. Usually you are expected to program with expectations for everything to go well (happy path). All other bad things…you let it crash. The process will be killed but then if you designed your model well, it will be detected by other processes and can be restarted. In most cases this is exactly what you want. Supervision trees wrap your clean business logic and react on proper events. As usual, hard to get at the start, but much appreciated after that. And actor model with message passing is a natural representation of the surrounding world - and how we communicate/use elements of it. I like the Joe Armstrong’s comparison to the physics and concept of time.

On the language itself…

Erlang is dynamically typed. This has both pros and cons. You can easily create sections that dynamically execute modules based on some conditions. Overall code is simpler and more concise. You spend less time refactoring and working on boilerplate stuff. Mocking in tests is a breeze. On the other hand you have a possibility to introduce more errors which could only be detected at runtime. Auch! And code analysis at times becomes harder as you need to jump a lot between modules to read the code. The former issue as I mentioned before is (and must!) be mitigated by suite of unit tests. I already said that and I’ll repeat: you need unit tests for Erlang program. Period. The latter issue can be more nice to live with by using some full fledged IDE like for example IntelliJ IDEA with Erlang plugin or VIM/Emacs with bunch of plugins for Erlang syntax, navigation and stuff. Erlang has some helpers for making types more explicit and even catch errors during compilation. It is called typespec and requires quite a lot of additional code in which we, by using grammar notation, define types along with inputs and outputs of functions. This is very handy because it can catch errors during compilation by using Dialyzer tool and for humans reading code as well. However, it was hard for me to create those typespecs and keep them up to date. Sometimes when I wanted to quickly create some new functions I had found out that I had left behind some of them without specs…and it required backtracking and some shortcuts. I guess this is a price for flexibility and we are still to see some tools that would be able to deduce and create such typespecs for us. I know that this is very hard algorithmic and mathematical problem to solve. Anyhow I strongly recommend to have typespecs and have them complete and up to date as in the long run they will make development faster and more pleasant.

On the pattern matching…

This is often mentioned as one of Erlang’s strongest points and for a very good reason. For me it was jaw-dropping feature. So powerful and so expressive. I’ll be honest, it destroyed all other languages for me and I have hard times switching now. But to the point. Let’s see some examples:

Function heads allow to cleanly express different processing for different input parameters. This results in short, precise and testable functions. Of course this can be expressed as if or case statements but I think that going with such small functions works much better:

handle_call({purchase, Item}, From, State) ->
handle_call({refund, Item}, From, State) when is_record(Item, computer) ->
handle_call(terminate, From, State) ->

Matching in lists (head, tail):

process_list([H|T])  %% The function expects a List with at least one element and at the same time it
                     %% will bound H variable to first element and T to the rest
                     %% (empty list if there is only one element).

%% What if you want to match a list with 2 elements and extract them at the same time?
process_list([First, Second|Tail])  %% Easy. Magic.

Matching in tuples.

%% This nice function expects tuple with 2 inside tuples of 3 elements.
%% At the same time it will bound it to the DateTime variable.
expecting_tuple(DateTime = {{_, _, _}, {_, _, _}})
This is especially useful for tagged tuples:
%% will match only when called with tire sizes tuple and will bind tire sizes of front and rear. Nice.
expecting_tuple({tire_sizes, {front, FrontTireSize}, {rear, RearTireSize}}).

**Matching in records.**Lets say we have following record:

-record(job, {id, kind, deadline, complexity}).
We can match it easily with:
%% This will be only called when passing job record and it will bind kind and deadline to the variables.
%% Other fields we don’t care about we can skip. This is very readable.
process(#job{kind=JobKind, deadline=JobDeadline}) ->

**Matching in binaries.** This is extremely powerful (and complicated) pattern matching. So, Erlang has a concept of binary with which we can very cleanly represent bits of data. For example we have a stream of bits in Stream variable:

<<SyncByte:8, Count:32, CRC:8, Rest/binary>> = Stream.
We can match Stream to our desired representation and bind our expected standard fields: sync byte, count of bytes in the stream, CRC sum and the rest payload. Working with network packets is so fun with Erlang and of course usages are much broader.

**Matching in receive.** This is almost the same as function heads or case statements, but language build support for actor model and messages makes it really readable and simple to handle messages sent to the process:

main_loop(State) ->
    message_a -> main_loop(process_message_a(State));
    message_b -> main_loop(process_message_b(State));
    shutdown -> exit(shutdown);
    _ -> log(), main_loop(State)
This shows how easy we can receive messages and act on them. And as a bonus it shows tail recursion optimization, in this context how we can implement main loop that potentially runs forever. There is more and I still find out some new clever usages of pattern matching..it is so powerful.

On the runtime…

And this is supercool. Let’s say we have working Erlang program. It works on a BEAM VM. We can attach to it and then, at real time analyse what is going on. We can use REPL to query state of our application, check variable values (and remember they never change, so you can’t get intermittent responses). But the most epic thing is when you use tools like Redbug or Recon. They allow to capture and trace specific function calls and see what were parameters. Tracing on production becomes a reality and I have used it many times with great successes. You can check some other posts on our blog:

  1. Introduction to Erlang Tracing
  2. Dark Corners of Erlang and Elixir Tracing

Some praise should also go to doing releases. Language and runtime take care of about deployments out of the box. The result is that this process is well designed and standardized. You can find out more on official Erlang documentation.

On the development…

I like the approach to modules and export/import of functions. In general you specify what you want to export from module and treat module as a namespace, not using import at all. When you want to call exported function you just use ModuleName:function. This is simple, expressive and is not cluttering modules with imports.

Unit testing is essential in modern software development and Erlang is no different. Being dynamically typed language makes it even more important to have proper suite of tests. I have been using EUnit and so far it has been very good framework. Tests are readable, can be grouped, labeled, properly wrapped by setup and cleanup. There are some macros and helpers that make code even more compact and readable. Separation of concerns because of processes, easy to understand DI (passing an argument to a function), pure functions (referential transparency + isolation of side effects) and immutability - all of these aspects help us with creation of unit tests.

What is very nice is mocking. Meck allows to create mocks on the fly with ease. There are only a couple of base Erlang modules that can’t be mocked and everything else is easily mockable. I found it to be one of the cleanest mocking frameworks I’ve used.

On more advanced testing concepts, Erlang lends itself well to property based testing. You can check Fred Hebert’s book for more details.

I have found out that working in TDD cycle is kinda awesome in Erlang. Using dynamic language, you can do fast iterations (red-green-refactor) in one particular module. There is no need to recompile whole project when working on single test case and module. Only when you are done you can recompile and update consuming modules. For me it was massive timesaver and one element of my perception of excellent development speed in Erlang. Comparing to Java, I had a feeling like being at least 3 times more productive and faster.

What about 3rd party libs and tools? Generally, I never had any problems with finding external libs for my needs. Erlang is alive with helpful open source community. There was a need to slightly modify a couple of libraries but all in all it was fun and provided space to learn new things. Special section should be dedicated to OTP (Open Telecom Platform). This is a mine of useful tools, libraries and middlewares. Everything battle tested and broadly used. It abstracts away a lot of details and I highly recommend digging and learning how it does that. It will make you a better Erlang programmer as you will learn about proper design patterns for concurrency.

I also like that Erlang takes very pragmatic approach. It takes what is good from functional programming but when necessary takes some shortcuts. Good example can be ETS which is like built-in, in-memory storage for shared data. If that raised some alerts that’s good, because it is dangerous to have shared state. However at times such capability can be extremely useful and make code much more compact. If you keep it isolated and use it according to a couple of rules there will be no problems.

All of these things make me feel very productive when developing. Code is simple. I generally don’t have to think about idioms, I can quickly do TDD cycle, number of bugs is lower.. And I just feel happier :)

On the negative side…

Is there anything I don’t like about Erlang?

Surprisingly not a lot. Error messages could be more verbose and easier to understand. Especially when test fails, in EUnit, I was spending too much time trying to understand what happened. I’m sure that experienced Erlang devs have no issues with that, but for someone at the start it can be hard. I’m comparing to Java’s Spring error messages by the way.

I have a feeling that the whole ecosystem is not as much mature as Java. This of course is a result of much smaller community. There is not that many developers and contributors. Libraries and tools sometimes lack documentation and can be rough around the edges.

Erlang’s string processing is tricky. There are 2 different types of Strings and one is slow but you can manipulate it easily, the second is fast but more cumbersome to use. They have very different syntax and it happens that you need to switch between representations in different places. Unfortunately there is no way to live without string manipulation and this aspect could be better.

Dynamic typing takes its toll as well. I’ve spent too many hours chasing bugs I created due to passing wrong types. Dialyzer errors can be absolute pain in the ass. It happens more than I like, that issue from 3rd party library causes errors being thrown from your code and they are not easily understandable. I saw experienced BEAMers scratching their heads as well.

I was a bit surprised to see that it was a common practice to download code of external libs, put it in your repo, modify it to your needs and use it directly. This of course has big advantages like tailoring lib to your needs and knowing what is going on “under the hood”. But I have found out that many of such changes rarely end up being pushed back into open-source. Another issue is that it quickly diverges from origin/master and creates problem with syncing. I know that latest changes in Rebar3 improve things a lot. By the way, Rebar3 is: Erlang build tool that makes it easy to compile and test Erlang applications and releases.. Strongly recommend to put it to good use for all Erlang projects.

Lastly, I don’t like syntax of anonymous functions which we use to create lambdas and closures. They are very common, and I could live with somewhat shorter syntax as now they look like this:

 lists:map(fun(X) -> X + 1 end, L).)

On the summary…

I have probably used the word simple too many times in this blog post. Sorry for that, but this is not a coincidence. It may be counterintuitive, but on the foundation of this simplicity, Erlang unleashes its power. And actually this is how I feel when working in Erlang: free, empowered, fast and just happy. Just like many years ago, when I started and was not overwhelmed by rules and boundaries.

Without a doubt, for me, Erlang is the best language I have ever used and I want and will use it as much as possible. It is just so perfect for production ready, massive scale systems. Cheers!

At the end I would like to recommend 2 books by Fred Hebert. They are free and are an excellent starting point on journey to learning and mastering Erlang:

  1. Learn You Some Erlang for Great Good
  2. Stuff Goes Bad: Erlang in Anger

And (not free) Joe Armstrong’s Programming Erlang: 2nd Edition

Veteran Elixir/Erlang Team Available

Are looking for Elixir or Erlang experts? You are in the right place! We truly love working with that technology, and as a side effect, it turned out that we have mastered it.

Schedule a call with our expert