Erlang Vm

What is wrong with gen_event?

gen_event confused me from the beginning, so I wanted to investigate the topic more deeply. I did that here. Then I left that topic, and it returned recently to me when I was wondering how the situation changed since then. Here is the updated version of the initial investigation, which started with the following statement: I never used a gen_event, I think it is a bad pattern. At first, it may look like a controversial statement, but I heard a lot of those complaints from other people.

Continue reading

Dark Corners of Erlang and Elixir Tracing

Observability-first Approach Tracing as a tool is immensely productive. It cannot be overestimated - especially in production, but also in development when debugging our own mistakes. In the previous article I covered why we should use that facility and why they invented it in the first place. After first sight and couple of rounds, you fall in love with that tool. Right after you gain some experience, more questions start to arise.

Continue reading

How would you debug a highly concurrent system?

Debugging Concurrency Let me tell you a story. We have deployed a new version of software on the whole fleet. Hundreds of machines in 4 different regions around the world. We observed our metrics, inspected our logs - nothing there. Success, another deployment without any issues. Fast forward two days, and we received a customer support ticket to investigate. One of our integrations complained that they are observing a significant amount of errors when calling our endpoints, in two distinct regions.

Continue reading