Docker Swarm - Connection problem with the kafka service

Question

I have a problem with a docker-compose in swarm. This is my docker-compose:

    version: "3"



services:

  zookeeper:
    image: 'bitnami/zookeeper:latest'
    deploy:
      replicas: 1
    ports:
      - "2181:2181"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
    networks:
      - ertest

  kafka:
    image: 'bitnami/kafka:latest'
    deploy:
      replicas: 1
    ports:
      - 9093:9092
    depends_on:
      - zookeeper
    environment:
      KAFKA_CFG_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_CFG_LISTENERS: INTERNAL://:9093,OUTSIDE://:9092
      KAFKA_CFG_ADVERTISED_LISTENERS: INTERNAL://kafka:9093,OUTSIDE://sub.domain.ltd:9092
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_CFG_INTER_BROKER_LISTENER_NAME: INTERNAL
      ALLOW_PLAINTEXT_LISTENER: "yes"
    networks:
      - ertest

  igi-interface:
    image: nsdevuti01.netstudio.it:8083/er/igi-interface:test
    depends_on:
      - kafka
    ports:
      - 8091:8091
    environment:
      - MP_MESSAGING_CONNENCTOR_SMALLRYE-KAFKA_SSL_TRUSTSTORE_LOCATION=/deployments/ssh/igi-gateway.server.truststore.jks
      - MP_MESSAGING_CONNENCTOR_SMALLRYE-KAFKA_SSL_TRUSTSTORE_PASSWORD=changeit
      - MP_MESSAGING_CONNENCTOR_SMALLRYE-KAFKA_SSL_KEYSTORE_LOCATION=/deployments/ssh/igi-gateway.server.keystore.jks
      - MP_MESSAGING_CONNENCTOR_SMALLRYE-KAFKA_SSL_KEYSTORE_PASSWORD=changeit
      - KAFKA_BOOTSTRAP_SERVERS=kafka:9092
    volumes:
      - ./config/igi-gateway:/deployments/ssh
    networks:
      - ertest

networks:
  ertest:
    driver: overlay
  ingress:
    external: true

The services are up (for this try they are deployed on the same node), but i receive the following error when logging the igi-interface related container:

14:54:46 WARN  [or.ap.ka.cl.NetworkClient] (kafka-producer-network-thread | kafka-producer-user-message) [Producer clientId=kafka-producer-user-message] Bootstrap broker kafka:9092 (id: -1 rack: null) disconnected
14:54:46 WARN  [or.ap.ka.cl.NetworkClient] (kafka-producer-network-thread | kafka-producer-account-message) [Producer clientId=kafka-producer-account-message] Connection to node -1 (kafka/10.0.1.10:9092) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue.

Is that a problem with the kafka container generated with the image bitnami/kafka?

this is the log of the kafka service:

[2022-05-27 15:05:53,780] WARN [SocketServer listenerType=ZK_BROKER, nodeId=1001] Unexpected error from /10.0.1.66 (channelId=10.0.1.70:9092-10.0.1.66:37934-372); closing connection (org.apache.kafka.common.network.Selector)

org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 369296129 larger than 104857600)

at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:105)

at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452)

at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402)

at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674)

at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:576)

at org.apache.kafka.common.network.Selector.poll(Selector.java:481)

at kafka.network.Processor.poll(SocketServer.scala:1144)

at kafka.network.Processor.run(SocketServer.scala:1047)

at java.base/java.lang.Thread.run(Thread.java:829)

these are the container on the network created:

"Containers": {
        "05b840fca9768baa737f251b8687027c7c7129e629d9edfab1c79eaab019bc79": {
            "Name": "test_igi-interface.1.gf1zbtaajohfhj09370mugqsn",
            "EndpointID": "299107c675dc38275f8a4eda6c6e94155c9047bbada7740b12f60170b9fe96ea",
            "MacAddress": "02:42:0a:00:01:48",
            "IPv4Address": "10.0.1.72/24",
            "IPv6Address": ""
        },
        "6285eb948cfedf917da28982ed08c8105712a3d6f6b0159b18e0606110320508": {
            "Name": "test_kafka.1.jpawj3nwf2q68niev5be7h8dm",
            "EndpointID": "7700a38302942549f464b99bfc48ed49e6d216d1d5e744368c65c0385cdfe2b9",
            "MacAddress": "02:42:0a:00:01:46",
            "IPv4Address": "10.0.1.70/24",
            "IPv6Address": ""
        },
        "a7f8fde2f5bcdd2abb6f74447b6922cc9d1f694bbc9f9c085935277bbf7404c3": {
            "Name": "test_zookeeper.1.bumghhajiejes9z1c9x3r4aak",
            "EndpointID": "5b2cf22296a4e85b6d83a37ca2b5ff93a3f29a5daf9e20cd45495097b3bcb773",
            "MacAddress": "02:42:0a:00:01:4a",
            "IPv4Address": "10.0.1.74/24",
            "IPv6Address": ""
        },
        "lb-test_ertest": {
            "Name": "test_ertest-endpoint",
            "EndpointID": "ed8f4e31d95fc4d9090010cd0d185e708183f7ceaf0738e3055fe7fc378f6fd5",
            "MacAddress": "02:42:0a:00:01:42",
            "IPv4Address": "10.0.1.66/24",
            "IPv6Address": ""
        }
    }

Kafka have an issue with test_ertest-endpoint, but i don't know why. How can i fix this problem? the container are under the same network (ertest) and this is an overlay network.

Thanks for your time.