Wednesday, June 28, 2017

Container Ochestrators

https://netsil.com/blog/kubernetes-vs-docker-vs-mesosphere/

Swarm


Kubernetes

Kubernetes is an open source container orchestration framework (create and manage multiple docker containers).
  • By Google.
  • Huge GitHub activity.
  • Runs everywhere (on-premise clusters or on cloud with support for AWS,GCE and Azure out of the box).
  • Declarative
  • REST API
  • Builds the cluster and maintains it: network abstraction and management, scale adding nodes, remove nodes, etc.
  • Service upgrade and rollback.
  • Resource abstraction and management: frees you from the machine. It does the segregation of the services in the machines. You only define the resource needs of each service in CPU, GB or percentages.
  • Storage abstraction and management: docker volumes, upgrades/restarts, 

Spark support

DC/OS 

  • Mesos based.
  • Container technology agnostic but now centered on Docker.
  • Non containerized applications can also be deployed.

Thursday, June 1, 2017

Kafka Connect

Standalone 
bin/connect-standalone worker.properties connector1.properties connector2.properties ...

Converters
Connectors
http://www.landoop.com/kafka/connectors/

Tasks
A single call to "put" may include messages from different topics.
"flush" is invoked periodically (offset.flush.interval.ms).
Both put and flush may be invoked with zero messages.

Custom connector
https://github.com/mcandre/hello-kafka-connect