To be honest, some kubernetes distributions make the cluster operations minimal (I use k0s managed via ansible)!
Either way, the moment you go from N containers on one box to N containers on M boxes you need to start considering how to handle stateful applications, load balancing, etc. And that in general requires knowledge on a domain which is different from having simply applications wrapped in containers locally.
I’ve tried Docker Swarm because Kubernetes seemed like an overkill for a cluster of 4 small-ish servers. There have been several issues (networking for example) that took me two days to solve - by reinstalling the machine completely.
There are some hoops and hurdles along the way, some command will just literally brick your cluster without any notice whatsoever (like removing the second manager, leaving only one and cluster stops responding, but you get no warning that’s gonna happen).
Also secrets, where there is no simple way to manage them, or replace them. You can’t just replace a secret, you have to remove and recreate it. Which means turning off the service or creating a new secret with a different name and do a rolling update, which is just annoying to do every time unless you can afford a robust CI CD pipeline code that does it automatically.
Even then, there’s dockerswarm.rocks (linking directly to tutorial to show how easy it is!)
I really thought swarm was dead :)
To be honest, some kubernetes distributions make the cluster operations minimal (I use k0s managed via ansible)!
Either way, the moment you go from N containers on one box to N containers on M boxes you need to start considering how to handle stateful applications, load balancing, etc. And that in general requires knowledge on a domain which is different from having simply applications wrapped in containers locally.
I’ve tried Docker Swarm because Kubernetes seemed like an overkill for a cluster of 4 small-ish servers. There have been several issues (networking for example) that took me two days to solve - by reinstalling the machine completely.
There are some hoops and hurdles along the way, some command will just literally brick your cluster without any notice whatsoever (like removing the second manager, leaving only one and cluster stops responding, but you get no warning that’s gonna happen).
Also secrets, where there is no simple way to manage them, or replace them. You can’t just replace a secret, you have to remove and recreate it. Which means turning off the service or creating a new secret with a different name and do a rolling update, which is just annoying to do every time unless you can afford a robust CI CD pipeline code that does it automatically.