When you can lose messages in Kafka
Kafka is speedy and fault-tolerant distributed streaming platform. However, there are some situations when messages can disappear. It can happen due to misconfiguration or misunderstanding Kafka’s internals. In this article, I’ll explain when the data loss can happen and how to prevent it. Publisher: Acknowledgment When a message is sent to the publisher, the publisher waits for an acknowledgment (ACK) from the broker. There are three configuration options which can be used:
Golang Tips & Tricks #4 - internal folders
While developing a library, we create a directory structure to keep the code organized. However, some exported functions or struct should not be used by users of the library. The achieve that, call the package internal. . ├── bar │ ├── bar.go │ └── internal │ └── foobar.go ├── internal │ └── foo.go └── main.go In the example above, the foo.go can be included only in the main.go. What’s more, only the bar.
Learning On Mistakes
There are many situations which can cause unavailability. One of them can be a bug in a software, bad architecture design decisions or even a human error. Depending on how the numbers are calculated, from 22% to even 70% of outages are caused by human error. Software engineers, DevOps or administrators cannot prevent all the outages but we can learn from ourselves to improve the stability and reliability of systems we are creating.
Golang Tips & Tricks #3 - graceful shutdown
In the microservices’ world, one thing what’s worth considering is a graceful shutdown. This is important to not lose data while shutting down a container. The container orchestrator like Kubernetes can restart the container by sending SIGTERM or SIGINT signal. Those signals can be handled to safely close all connections and finish background tasks. Signals are propagated using os.Signal channel. You can add the above code to your main. var gracefulStop = make(chan os.
Golang Tips & Tricks #2 - interfaces
When it comes to interfaces, a good practice is to create an interface where you’ll use it. Creating interfaces in advanced is not recommended in Go. There are two exceptions: you’re creating a library which will be used in different projects you’ll have more than 1 implementation In the example below, we have a storage implementation. type inMemoryStorage struct { mutex *sync.Mutex storage map[string]*Value } func NewStorage() *inMemoryStorage { return &inMemoryStorage{ storage: map[string]*Value{}, mutex: &sync.
Golang Tips & Tricks #1 - errors
You should use the package github.com/pkg/errors instead of errors package for errors in your applications. The default package lacks a few things like: stack trace easy appending message to the error and more It helps with debugging a lot. Below you can find an example error message with the stack trace. An important thing to remember is that you should wrap every error which is from any external library or your freshly created error you want to return.
Go deeper – Database connection pool
Golang uses a connection pool to manage opened connections for us. As a result, new connections are used when no free connection left and reuses them when golang finds an idle connection. The most important thing is that when two queries are called one by one it does not mean that the queries will use the same connection. It may be true if not in every case. In the example below, you can find two queries which may seem to be executed in one connection.
Be aware of copying in Go
Some bugs are very hard to find and to reproduce but easy to fix. To avoid them, it’s helpful to know how the tools we’re using work under the hood. From this article, you’ll learn what shallow and deep copy are and which errors you can avoid thank’s the knowledge about them. Can you find a problem with the code below? q1 := NewQuestion(1, "How to be cool?") q1 = q1.
Books I read in 2018
I’m glad that this year I read so many valuable books. I want to share with you with the most interesting items which you may find interesting too. Getting Things Programmed This book is only available in Polish. I would say that this is Getting Things Done but for developers. I think that the similarity of titles is not an accident. In this book, you can find answers on questions like:
How to send multiple variables via channel in golang?
Channels in golang are referenced type. It means that they are references to a place in the memory. The information can be used to achieve the goal. Firstly, let’s consider using structs as the information carrier. This is the most intuitive choice for the purpose. Below you can find an example of a struct which will be used today. type FuncResult struct { Err error Result int } func NewFuncResult(result int) FuncResult { return FuncResult{Result: result} } The idea is to create a channel from the struct, pass the channel to a function and wait for the result.