why kubernetes networking driver weave generating big log file

Question

I want to ask why the Kubernetes networking driver weave container is generating a lot of logs?

The log file size is 700MB after two days.

How can I solve that?

Answer 1

Logs in kubernetes

As it was said in comment, kubernetes is not responsible for log rotation. This is from kubernetes documentation:

An important consideration in node-level logging is implementing log rotation, so that logs don't consume all available storage on the node. Kubernetes is not responsible for rotating logs, but rather a deployment tool should set up a solution to address that. For example, in Kubernetes clusters, deployed by the kube-up.sh script, there is a logrotate tool configured to run each hour. You can also set up a container runtime to rotate an application's logs automatically.

As proposed option, this can be managed on container's runtime level.

Please refer to Logging at the node level.

Reducing logs for Weave CNI

There are two containers in each pod. Weave itself and weave-npc (which is a network policy controller).

By default weave's log level is set to INFO. This can be changed to WARNING to see only exceptions. This can be achieved by adding --log-level flag through the EXTRA_ARGS environment variable for the weave:

$ kubectl edit daemonset weave-net -n kube-system

So weave container part should look like:

spec:
  containers:
  - command:
    - /home/weave/launch.sh
    env:
    - name: HOSTNAME
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: spec.nodeName
    - name: EXTRA_ARGS # this was added with value below!
      value: --log-level=warning
    - name: INIT_CONTAINER
      value: "true"
    image: docker.io/weaveworks/weave-kube:2.8.1
    imagePullPolicy: IfNotPresent
    name: weave 

Weave - logs level.

A lot of logs go from Weave NPC, there's an option that allows to disable NPC. However based on documentation this is a paid option based on their documentation - cloud.weave.works

Weave - Changing configuration options