Kapsule APIv1beta4

Kubernetes Kapsule by Scaleway is a managed, production-ready environment for deploying, managing and scaling containerized applications. It installs and operates Kubernetes Clusters, allowing you to get up and running with them in no time.

"Kubernetes is a portable, extensible open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available." -- What is kubernetes

The product is currently available as a beta version. This means that:

• changes may be introduced in product API,
• product may be discontinued,
• you agree to use the product at your own risk,
• the product is free of charge.

During this phase, we will do our best to ensure the production-like quality of the service but we cannot guarantee its continuity.

• Persistant Volume Claims (PVC) are available through Scaleway Block Volumes.
• Pool autoscaling and autohealing is available using the API.
• Multiple container runtimes available (Only docker is officially supported, containerd and crio are experimental. NB: containerd ships with gvisor for untrusted workloads).
• Kubernetes auto upgrades features is available.

• Kapsule is only available in PAR1 datacenter
• Clusters can't be isolated in a VPC (Virtual Private Cloud).
• During the beta phase the maximum number of nodes per cluster is 20.
• During the beta phase all GP1 instances can be allocated in new and default pools.
• The maximum of pod per node is 110.
• Kubernetes Kapsule is currently in beta and no service continuity is guarantee
• When replacing a node : the node is deleted and a new one is created in parallel.

This API allows you to:

• create, update and delete a Kubernetes cluster
• manage cluster resources using node pools

The Cluster Master directs the worker nodes, making scheduling decisions, responding to events, implementing changes, and monitoring the Cluster. Its components (kube-apiserver, kube-scheduler and kube-controller-manager) are managed by Scaleway to ensure the Master remains highly available. Consequently, you cannot set your own parameters on the Kubernetes api-server, controller-manager and scheduler. Scaleway takes care of your control plane: its health is continuously monitored and watched by a dedicated team. Its availability enables the Master to get upgraded or rebooted without any downtime, and the team can perform maintenance operations on your clusters in total transparency.

Component on the master that exposes the Kubernetes API. It is the front-end for the Kubernetes control plane. It is managed by Scaleway.

Component on the master that watches newly created pods that have no assigned node, and selects a node for them to run on. It is managed by Scaleway.

Component on the master that runs controllers of Kubernetes. It is managed by Scaleway.

Scaleway Load Balancers are used to route requests toward the multiple control-plane replicas. This enables high availability for your Cluster.

Consistent and highly-available key value store used as Kubernetes backing store for all cluster data.

The Cloud Controller Manager runs controllers that interact with Scaleway cloud, allowing to create load balancer objects and to retrieve information about your cluster (node public IP, node location, node commercial type...)

Deamon that reports nodes statuses to the orchestrator, allowing -for instance- healing actions when a node has been down for a certain period of time.

Cluster resources (i.e. nodes) are managed within pools. A pool has a name (optional), a size (its current number of nodes), and nodes number limits (min, max). Changing those limits increases/decreases the size of a pool. Thus, when autoscaling is enabled, the pool will grow or shrink inside those limits, depending on its cluster load. A "default pool" is automatically created with every Cluster and can't be deleted.

A Node (short for Worker Node) is an abstraction for a Scaleway instance. It is part of a pool and is instantiated by Scaleway, making Kubernetes software installed and configured automatically on it. Please note that Kubernetes nodes cannot be accessed with ssh.

Container Network Interface (https://github.com/containernetworking/cni) is a standard designed to ease configuration of container networking at their creation/deletion. Various projects have been released to address Kubernetes networking requirements and provide the networking features that cluster administrators require. Kubernetes Kapsule supports the following CNI plugins : cilium (https://cilium.io/), flannel (https://github.com/coreos/flannel), calico (https://github.com/projectcalico/cni-plugin) & weave (https://www.weave.works/oss/net/). Flannel plugin is installed by default, but yo can customize your cluster picking the one that suits your need.

Below is a step by step guide explaining how to create a cluster, deploy an application on it, use the cluster wildcard DNS to request the application, extend a pool and then delete the cluster.

In order to use the Scaleway API you need to have curl installed. It is also a good idea to have jq, which will help you to read and parse JSON output. Make sure you have these two tools before you begin. Otherwise use your package manager to install them.

To call the Scaleway API, you will need an X-Auth-Token. If you don't have one yet, you can create one on credentials page of your Scaleway account.

Using this token, get your organization ID:

$ export TOKEN="<Secret key of your token>"
$ export REGION="<choose your location (nl-ams/fr-par)>"

$ curl -H "X-Auth-Token: $TOKEN" https://account.scaleway.com/organizations | jq '.organizations[] | "orga_id: " + .id + " orga_name: " + .name'

$ export ORGANIZATION_ID="<your organization ID>"

You can also find your organization ID in Scaleway web console.

Customize the name, description, tags and set your organization ID

$ curl -X POST "https://api.scaleway.com/k8s/v1beta4/regions/$REGION/clusters" -H "Accept: application/json" -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/json" \
-d "{
        \"cni\": \"flannel\",
        \"enable_dashboard\": true,
        \"default_pool_config\": {
            \"autoscaling\": true,
            \"node_type\": \"gp1_xs\",
            \"max_size\": 3,
            \"min_size\": 1,
            \"size\": 3,
        },
        \"description\": \"my first cluster\",
        \"ingress\": \"nginx\",
        \"name\": \"cluster1\",
        \"organization_id\": \"${ORGANIZATION_ID}\",
        \"tags\": [
            \"tag1\",
            \"tag2\"
        ],
        \"version\": \"1.14.1\"
   }"

{"region":"fr-par","id":"4a31ee90-71eb-4539-a647-7b1ad3bcde67","organization_id":"82833329-b242-4cf8-9193-8d1600aff0c1","created_at":"2019-04-30T12:37:51.006332102Z","updated_at":"2019-04-30T12:37:51.006332102Z","name":"cluster1","description":"my first cluster","current_node_count":0,"status":"creating","version":"1.14.1","cni":"flannel","tags":["tag1","tag2"],"current_core_count":0,"current_mem_count":0,"cluster_url":"https://4a31ee90-71eb-4539-a647-7b1ad3bcde67.api.k8s.fr-par.scw.cloud:6443","dns_wildcard":"*.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud","autoscaler_config":{"scale_down_disabled":false,"scale_down_delay_after_add":"10m","estimator":"binpacking","expander":"random","ignore_daemonsets_utilization":false,"balance_similar_node_groups":false,"expendable_pods_priority_cutoff":-10},"dashboard_enabled":true,"ingress":"nginx"}

Copy the id field of the response to use in the next steps. For the sake of simplicity we will save the ID to a variable, which we will use in the following examples:

$ export CLUSTER_ID="4a31ee90-71eb-4539-a647-7b1ad3bcde67"

Wait for the cluster to be in ready state:

$ curl -Ss -X GET "https://api.scaleway.com/k8s/v1beta4/regions/${REGION}/clusters/${CLUSTER_ID}" -H "Accept: application/json" -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/json" | jq -r '.status'
ready

In order to be able to manage you cluster retrieve your kubeconfig file:

$ curl -Ss -X GET "https://api.scaleway.com/k8s/v1beta4/regions/${REGION}/clusters/${CLUSTER_ID}/kubeconfig?dl=1" -H "Accept: application/json" -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/json" > /tmp/mycluster.kubeconfig

Use the previously downloaded kubeconfig file to manage your cluster. Wait for all the nodes to join the cluster (we allow you to connect to the control plane before all the nodes are ready and it can take some time to create it).

$ export KUBECONFIG=/tmp/mycluster.kubeconfig
$ kubectl get nodes
NAME                                             STATUS   ROLES    AGE    VERSION
scw-cluster1-default-5aa9167001624d5498033c647   Ready    <none>   104s   v1.14.1
scw-cluster1-default-60a2fd61a09e442ca2c5a905f   Ready    <none>   2m4s   v1.14.1
scw-cluster1-default-d82d9a85b32c47f6ab489ea1c   Ready    <none>   97s    v1.14.1
$ kubectl cluster-info
Kubernetes master is running at https://4a31ee90-71eb-4539-a647-7b1ad3bcde67.api.k8s.fr-par.scw.cloud:6443
CoreDNS is running at https://4a31ee90-71eb-4539-a647-7b1ad3bcde67.api.k8s.fr-par.scw.cloud:6443/api/v1/namespaces/kube-system/services/coredns:dns/proxy
kubernetes-dashboard is running at https://4a31ee90-71eb-4539-a647-7b1ad3bcde67.api.k8s.fr-par.scw.cloud:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy

In this step we will show you how to deploy your first application on Kubernetes and how to leverage the free wildcard DNS automatically created by Scaleway with your new cluster.

Create the cafe application. This application is very simple and will show "coffee" if you query a coffee url, and "tea" if you query a tea url.

$ kubectl create -f https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/master/examples/complete-example/cafe.yaml
deployment.extensions/coffee created
service/coffee-svc created
deployment.extensions/tea created
service/tea-svc created

By a default a free wildcard dns is given to you at the cluster creation, you can use it to create your ingress objects. This wildcard point to all the public ips of you cluster. If you want something more robust you can choose to deploy a load balancer (but it in this case you have to manage the dns by yourself). The free wildcard looks like that *.<CLUSTER_UUID>.nodes.k8s.fr-par.scw.cloud

You can check the wildcard is working.

$ host coffee.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud
coffee.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud has address 51.15.196.44
coffee.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud has address 51.158.110.219
coffee.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud has address 212.47.243.47

$ host tea.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud
tea.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud has address 51.15.196.44
tea.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud has address 51.158.110.219
tea.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud has address 212.47.243.47

Then create the ingress object matching tea or coffee url:

$ cat << EOF > /tmp/myingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: cafe-ingress
spec:
  rules:
  - host: coffee.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud
    http:
      paths:
      - backend:
          serviceName: coffee-svc
          servicePort: 80
  - host: tea.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud
    http:
      paths:
      - backend:
          serviceName: tea-svc
          servicePort: 80
EOF

$ kubectl create -f /tmp/myingress
ingress.extensions/cafe-ingress created

Just query both url using curl and check the application is working.

$ curl cafe.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud
Server address: 100.64.2.3:80
Server name: coffee-bbd45c6-m6nck
Date: 30/Apr/2019:12:56:42 +0000
URI: /
Request ID: 7ad4396d26e472626fdf16b3b1a41d99

$ curl tea.4a31ee90-71eb-4539-a647-7b1ad3bcde67.nodes.k8s.fr-par.scw.cloud
Server address: 100.64.2.4:80
Server name: tea-5857f7786b-glxp8
Date: 30/Apr/2019:12:56:48 +0000
URI: /
Request ID: b1568ac01c0d0040f3ef9357fce4268c

Get the default pool id.

$ curl -Ss -X GET "https://api.scaleway.com/k8s/v1beta4/regions/$REGION/clusters/$CLUSTER_ID/pools" -H "Accept: application/json" -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/json" | jq -r '.pools[]|select(.name|contains("default"))|.id'
e95f9a3a-5b95-4490-8b5c-adcd14abedb0
$ export POOL_ID=e95f9a3a-5b95-4490-8b5c-adcd14abedb0

Extend the pool for 3 to 4

$ curl -Ss -X PATCH "https://api.scaleway.com/k8s/v1beta4/regions/$REGION/pools/$POOL_ID" -H "Accept: application/json" -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/json" -d "{

               \"autoscaling\": true,
               \"size\": \"4\",
               \"min_size\": \"1\",
               \"max_size\": \"4\"
             }"

{"region":"fr-par","id":"e95f9a3a-5b95-4490-8b5c-adcd14abedb0","cluster_id":"4a31ee90-71eb-4539-a647-7b1ad3bcde67","created_at":"2019-04-30T12:37:51.008981Z","updated_at":"2019-04-30T13:14:18.428172545Z","name":"default","current_node_count":3,"status":"ready","version":"1.14.1","node_type":"gp1_xs","autoscaling":true,"size":4,"min_size":1,"max_size":4,"current_core_count":12,"current_mem_count":25769803776,"container_runtime":"docker","autohealing":false,"tags":[],"placement_group_id":null}

Wait for the pool to be extended.

$ kubectl get nodes
NAME                                             STATUS     ROLES    AGE   VERSION
scw-cluster1-default-5aa9167001624d5498033c647   Ready      <none>   35m   v1.14.1
scw-cluster1-default-60a2fd61a09e442ca2c5a905f   Ready      <none>   35m   v1.14.1
scw-cluster1-default-86c266b168604d1a83a182cf2   NotReady   <none>   1s    v1.14.1
scw-cluster1-default-d82d9a85b32c47f6ab489ea1c   Ready      <none>   35m   v1.14.1
$ kubectl get nodes
NAME                                             STATUS   ROLES    AGE   VERSION
scw-cluster1-default-5aa9167001624d5498033c647   Ready    <none>   35m   v1.14.1
scw-cluster1-default-60a2fd61a09e442ca2c5a905f   Ready    <none>   36m   v1.14.1
scw-cluster1-default-86c266b168604d1a83a182cf2   Ready    <none>   22s   v1.14.1
scw-cluster1-default-d82d9a85b32c47f6ab489ea1c   Ready    <none>   35m   v1.14.1

$ curl -Ss -X DELETE "https://api.scaleway.com/k8s/v1beta4/regions/$REGION/clusters/$CLUSTER_ID" -H "Accept: application/json" -H "X-Auth-Token: $TOKEN" -H "Content-Type: application/json"