KubernetesKubernetes is an open source container orchestration framework (create and manage multiple docker containers).
- By Google.
- Huge GitHub activity.
- Runs everywhere (on-premise clusters or on cloud with support for AWS,GCE and Azure out of the box).
- REST API
- Builds the cluster and maintains it: network abstraction and management, scale adding nodes, remove nodes, etc.
- Service upgrade and rollback.
- Resource abstraction and management: frees you from the machine. It does the segregation of the services in the machines. You only define the resource needs of each service in CPU, GB or percentages.
- Storage abstraction and management: docker volumes, upgrades/restarts,
- Standalone: http://blog.madhukaraphatak.com/scaling-spark-with-kubernetes-part-1/
- Kubernetes as native scheduler: https://datapipes.org/2017/04/05/native-spark-jobs-on-kubernetes/ https://www.slideshare.net/databricks/apache-spark-on-kubernetes-anirudh-ramanathan-and-tim-chen
- Kubernetes in Mesos: https://thenewstack.io/mesosphere-now-includes-kubernetes-for-managing-clustered-containers/
- Mesos based.
- Container technology agnostic but now centered on Docker.
- Non containerized applications can also be deployed.