[Q&A] efk is up via docker compose but hitting the kibana url on port 5601 via browser or wget just keeps awaiting response #5040

alkuma · 2025-07-29T07:06:34Z

alkuma
Jul 29, 2025

What is a problem?

We followed instructions to install at https://docs.fluentd.org/container-deployment/docker-compose - for fluentd version 1.0 to the letter

$ tree
.
├── docker-compose.yml
└── fluentd
    ├── conf
    │   └── fluent.conf
    └── Dockerfile

The command to bring up the components gives us

[+] Running 5/5
 ✔ Network efk5_default      Created                                       0.0s 
 ✔ Container elasticsearch   Healthy                                      21.1s 
 ✔ Container efk5-kibana-1   Started                                      21.2s 
 ✔ Container efk5-fluentd-1  Started                                      21.2s 
 ✔ Container efk5-web-1      Started                                      21.4s

This can be verified via

$ docker compose stats
CONTAINER ID   NAME             CPU %     MEM USAGE / LIMIT   MEM %     NET I/O   BLOCK I/O   PIDS 
ef4a3de9d8a9   efk5-web-1       --        -- / --             --        --        --          -- 
ea081c3665bd   efk5-fluentd-1   --        -- / --             --        --        --          -- 
77c918387617   efk5-kibana-1    --        -- / --             --        --        --          -- 
460c49d94f78   elasticsearch    --        -- / --             --        --        --          --

Now when we try to follow the next step in the instructions, ie to generate logs for httpd (apache)

$ curl http://localhost:8080/
<html><body><h1>It works!</h1></body></html>

So the apache is up thanks to the docker compose up -d

Now we come to step 4 of the documentation which asks us to navigate to http://localhost:5601/app/discover#/

Same with wget

$ wget localhost:5601
--2025-07-29 12:23:34--  http://localhost:5601/
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:5601... connected.
HTTP request sent, awaiting response...

The service is indeed listening at port 5601

$ netstat -an | grep 5601 | grep LISTEN
tcp        0      0 0.0.0.0:5601            0.0.0.0:*               LISTEN     
tcp6       0      0 :::5601                 :::*                    LISTEN

The documentation on that page doesn't mention any next diagnostic steps

Describe the configuration of Fluentd

The docker-compose.yml is as follows

services:
  web:
    image: httpd
    ports:
      - "8080:80"
    depends_on:
      - fluentd
    logging:
      driver: "fluentd"
      options:
        fluentd-address: localhost:24224
        tag: httpd.access

  fluentd:
    build: ./fluentd
    volumes:
      - ./fluentd/conf:/fluentd/etc
    depends_on:
      # Launch fluentd after that elasticsearch is ready to connect
      elasticsearch:
        condition: service_healthy
    ports:
      - "24224:24224"
      - "24224:24224/udp"

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.17.1
    container_name: elasticsearch
    hostname: elasticsearch
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false # Disable security for testing
    healthcheck:
      # Check whether service is ready
      test: ["CMD", "curl", "-f", "http://localhost:9200/_cluster/health"]
      interval: 10s
      retries: 5
      timeout: 5s
    ports:
      - 9200:9200

  kibana:
    image: docker.elastic.co/kibana/kibana:8.17.1
    depends_on:
      # Launch fluentd after that elasticsearch is ready to connect
      elasticsearch:
        condition: service_healthy
    ports:
      - "5601:5601"

The fluentd/Dockerfile is as follows:

# fluentd/Dockerfile

FROM fluent/fluentd:edge-debian
USER root
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.4.3"]
USER fluent

The fluentd/conf/fluent.conf file is as follows:

# fluentd/conf/fluent.conf

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match *.**>
  @type copy

  <store>
    @type elasticsearch
    host elasticsearch
    port 9200
    logstash_format true
    logstash_prefix fluentd
    logstash_dateformat %Y%m%d
    include_tag_key true
    type_name access_log
    tag_key @log_name
    flush_interval 1s
  </store>

  <store>
    @type stdout
  </store>
</match>

Describe the logs of Fluentd

We are not certain where these logs will be hence skipping this, happy to provide them if needed.

Environment

- Fluentd version:Created from the documentation and code at https://docs.fluentd.org/container-deployment/docker-compose for `1.0`
- TD Agent version: Created from the documentation and code at https://docs.fluentd.org/container-deployment/docker-compose for `1.0`
- Fluent Package version: Created from the documentation and code at https://docs.fluentd.org/container-deployment/docker-compose for `1.0`
- Docker image (tag): Created from the documentation and code at https://docs.fluentd.org/container-deployment/docker-compose for `1.0`
- Operating system: Ubuntu 24.04.2 LTS
- Kernel version: 6.14.0-24-generic

Answered by Watson1978

Aug 5, 2025

I think your issue is related to fluent/fluentd-docs-gitbook#594 (comment)

By an error in data transmission from Fluentd, the data may not be displayed on the dashboard.

I think you can solve the issue when you rewrite Dockerfile with following.

FROM fluent/fluentd:edge-debian
USER root

# ADD
RUN ["gem", "install", "elasticsearch", "--no-document", "--version", "8.19.0"]

RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.4.3"]
USER fluent

View full answer

alkuma · 2025-08-04T04:08:06Z

alkuma
Aug 4, 2025
Author

OP here again. Any suggestions on what to look at (logs, folders, configurations) to understand what's going on will help.

1 reply

Watson1978 Aug 5, 2025
Collaborator

I think your issue is related to fluent/fluentd-docs-gitbook#594 (comment)

By an error in data transmission from Fluentd, the data may not be displayed on the dashboard.

I think you can solve the issue when you rewrite Dockerfile with following.

FROM fluent/fluentd:edge-debian
USER root

# ADD
RUN ["gem", "install", "elasticsearch", "--no-document", "--version", "8.19.0"]

RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.4.3"]
USER fluent

Answer selected by alkuma

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Q&A] efk is up via docker compose but hitting the kibana url on port 5601 via browser or wget just keeps awaiting response #5040

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment 1 reply

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

[Q&A] efk is up via docker compose but hitting the kibana url on port 5601 via browser or wget just keeps awaiting response #5040

Uh oh!

alkuma Jul 29, 2025

What is a problem?

Describe the configuration of Fluentd

Describe the logs of Fluentd

Environment

Replies: 1 comment · 1 reply

Uh oh!

alkuma Aug 4, 2025 Author

Uh oh!

Uh oh!

Watson1978 Aug 5, 2025 Collaborator

alkuma
Jul 29, 2025

Replies: 1 comment 1 reply

alkuma
Aug 4, 2025
Author

Watson1978 Aug 5, 2025
Collaborator