Can't Access WebHDFS

[rgreen13]

I can't access WebHDFS via Curl or using Python HDFS. For instance, the response of curl -i "http://localhost:50070/webhdfs/v1/user/?op=GETFILESTATUS" is curl: (52) Empty reply from server.

When I log into the namenode and check to see if port 50070 is in use, all I can see is the following output:

I've double checked to ensure that HDFS_CONF_dfs_webhdfs_enabled=true is also set across the board. This carries through and is accurately set in the config file.

Any suggestions?

[OneCricketeer]

My repo is outdated, but it did work when I tried setting up HUE FileBrowser

https://github.com/OneCricketeer/docker-stacks/tree/master/hadoop-spark

[aurma97]

Hello, i don't know if it's late, i got the same problem and the fix was just by adding subnetworks.

Find bellow my config, i also add hostname and it's the ip of container...

networks:
  frontend:
    ipam:
      config:
        - subnet: 172.20.0.0/24

services:
  namenode:
    image: bde2020/hadoop-namenode:2.0.0-hadoop3.2.1-java8
    container_name: namenode
    hostname: "172.20.0.6"
    restart: always
    networks:
      frontend:
          ipv4_address: 172.20.0.6
    ports:
      - 9870:9870
      - 9000:9000
    volumes:
      - hadoop_namenode:/hadoop/dfs/name
    environment:
      - CLUSTER_NAME=test
    env_file:
      - ./hadoop.env

  datanode:
    image: bde2020/hadoop-datanode:2.0.0-hadoop3.2.1-java8
    container_name: datanode
    hostname: "172.20.0.7"
    restart: always
    networks:
      frontend:
          ipv4_address: 172.20.0.7
    volumes:
      - hadoop_datanode:/hadoop/dfs/data
    environment:
      SERVICE_PRECONDITION: "namenode:9870"
    env_file:
      - ./hadoop.env
  
  resourcemanager:
    image: bde2020/hadoop-resourcemanager:2.0.0-hadoop3.2.1-java8
    container_name: resourcemanager
    restart: always
    networks:
      frontend:
          ipv4_address: 172.20.0.8
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode:9864"
    env_file:
      - ./hadoop.env

  nodemanager1:
    image: bde2020/hadoop-nodemanager:2.0.0-hadoop3.2.1-java8
    container_name: nodemanager
    restart: always
    networks:
      frontend:
          ipv4_address: 172.20.0.9
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode:9864 resourcemanager:8088"
    env_file:
      - ./hadoop.env
  
  historyserver:
    image: bde2020/hadoop-historyserver:2.0.0-hadoop3.2.1-java8
    container_name: historyserver
    restart: always
    networks:
      frontend:
          ipv4_address: 172.20.0.10
    environment:
      SERVICE_PRECONDITION: "namenode:9000 namenode:9870 datanode:9864 resourcemanager:8088"
    volumes:
      - hadoop_historyserver:/hadoop/yarn/timeline
    env_file:
      - ./hadoop.env
  
volumes:
  hadoop_namenode:
  hadoop_datanode:
  hadoop_historyserver:

[minhdc]

In my situation, I just mapped the corresponding ports to the outside, for example:

  datanode:
    image: bde2020/hadoop-datanode:2.0.0-hadoop3.2.1-java8
    container_name: datanode
    restart: always
    ports:
      - 9864:9864
    volumes:
      - hadoop_datanode:/hadoop/dfs/data
    environment:
      SERVICE_PRECONDITION: "namenode:9870"
    env_file:
      - ./hadoop.env

The default docker-compose file didn't include the port mapping for containers except the name node

[wangdian]

Though it's late, but hadoop had changed port of namenode from 50070 to 9870 on 3.x.x
But seems this project just updated docker-compose but not updated the swarm config...
Plus there are couple of problems, for example, upload/download file outside (it will direct to datanode, which cannot be resolved outside docker)