Ljupcho Apostolov /data/devs/

Golang Microservices Starter Kit

Golang microservices starter kit is my idea on how to architecturalize services in order to find the optimal way to a scalable solution and at same time abstractions to allow code reusability and easier development. This would be something to get me started on the right foot and give me a way to break down services. In microservices terminology we can distinguish three entities: api, worker and consumer. Sometimes I spend quite some time making a decision where some logic belongs and how to move the data. Those decisions lead up to a more robust system and more scalable solution and if not done in the right way might lead up to unnecessary processing and requests, services hard to work with and as the times goes on even harder to break down. Starter kit offers monitoring with prometheus and grafana. Deployment is done with helm charts.


Continue reading...

The cost of a grpc call

In this blog post I am looking more into inner-server communication between golang microservices. Obviously there is a cost to having multiple services that need to exchange data rather than having one bigger service. But, not going too much into the micro-services discussion here, I would prefer having smaller services for lots of reasons. Structuring them is an art on its own and deciding what should be service and what kind. I would say there are three main types of services: api, worker and consumer. Having a MQ to load balance between the same service that gets autos-laced and fan-out on the same topic for different services. That’s all there is to it.


Continue reading...

Scaling worker environment with k8s

After having set up a consumer app as a service and setting it up under kubernetes cluster, I have set a minimum of 2 replicas with the possibility of horizontal scaling up to 8. The consumers, or worker service on start up makes a connection to postgres and rabbitmq and have some queueing jobs to be run. The api which is a separate service does the publishing of the jobs to the rabbitmq and the consumer services picks them up, based on the job type and payload builds the appropriate job and runs it.


Continue reading...

Path to golang microservices

I have one application that has a lot of things like modules, internal packages/libs, migrations, cli commands, controllers/handlers, services, repositories, utils functions etc. I wanted to extract some of those into services that will make more sense so they will be handled in a better way both in terms of working with them and deployments. I am still not sold on the microservices bandwagon, but I for one know that carrying on with a monolith is not the way. Kubernetes offers a lot of flexibility for building service, more than I need really, all I want is to have a couple of services split in a way that are independent regardless of data share so k8s will load balance between multiple instances/pods. Regarding the data of-course most companies deviate towards having a managed service from aws, google cloud or elsewhere will proper backups, data-warehouse solutions etc. So, I began splitting some of the folders/code from my monolith into modules and making services from some of them. I would have entities, libs, consumers, migrations, seeds and app (what’s left of the monolith) as modules and will make services out of them except for the entities and libs. The reason for that is because I want to share the code as much as possible, but I’d rather extract them as modules or even duplicate some of the functions instead of running grps calls between the services. Out of these services migrations and seeds will be tasks instead of pods, so once they are called and run they will free up space and memory.


Continue reading...

Rust vs Golang

Yet another one rust vs go comparison, but I wanted to check for myself and have a bit of the rust dev experience. The idea is to create a simple endpoint and test it out with benchmark tools. Of-course rust would be better just wanted to have my first rust dev experience and let me tell you it took me a few weeks to get it running. The biggest problems developers have today is the ability to unlearn things that they considered to be solid and live by and learn new concepts. It seems to me it is hard to let go of things that took you a lot of time to learn and now that you have finally felt comfortable using the inheritance and design patterns with your classes and building concepts with them you would need to forget all of that. Simple things like DI, dependency injection that is more or less present in all languages and widely accepted might be troublesome to implement using Rust and even more trying to have the same code structure of handlers/controllers, services or repositories.


Continue reading...