TLS errors when hassio tries to connect to this broker

[gbrayut]

Problem/Motivation

I tried setting up MQTT on hassio today using tls (Lets Encrypt certificate via another plugin), but I was not able to get it to work. The homeassistant logs don't give much detail:

 docker logs homeassistant >& /tmp/logs.txt; grep -i mqtt /tmp/logs.txt
2018-12-16 18:33:00 INFO (MainThread) [homeassistant.loader] Loaded mqtt from homeassistant.components.mqtt
2018-12-16 18:33:00 INFO (MainThread) [homeassistant.setup] Setting up mqtt
2018-12-16 18:33:00 INFO (MainThread) [homeassistant.setup] Setup of domain mqtt took 0.0 seconds.
2018-12-16 18:33:02 INFO (Thread-2) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
2018-12-16 18:33:02 INFO (MainThread) [homeassistant.loader] Loaded mqtt.discovery from homeassistant.components.mqtt.discovery
2018-12-16 18:33:03 INFO (Thread-2) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
2018-12-16 18:33:05 INFO (Thread-2) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
2018-12-16 18:33:09 INFO (Thread-2) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server
2018-12-16 18:33:17 INFO (Thread-2) [homeassistant.components.mqtt] Successfully reconnected to the MQTT server

but it is not getting any messages. The broker works fine using http://workswithweb.com/mqttbox.html and the Hivemq web ui, so this might be an issue with home assistant or hassio, or possibly the tls certificate validation (although I also tried disabling that).

Expected behavior

Home assistant should be able to publish messages from the https://hassio:8123/dev-mqtt

Actual behavior

I can connect via mqttbox but publishing via dev-mqtt doesn't work, and any sensor created via mqtt also don't work. The following errors are displayed in the mqtt addon logs:

docker logs addon_a0d7b954_mqtt -f
INFO: Starting NGINX for the MQTT client...
1545010299: New client connected from 192.168.0.4 as mqttbox1545009967693 (c1, k10, u'mqtt_iot').
2018/12/16 18:31:42 [notice] 1075#1075: using the "epoll" event method
2018/12/16 18:31:42 [notice] 1075#1075: nginx/1.14.1
2018/12/16 18:31:42 [notice] 1075#1075: OS: Linux 4.14.66-v8
2018/12/16 18:31:42 [notice] 1075#1075: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2018/12/16 18:31:42 [notice] 1075#1075: start worker processes
2018/12/16 18:31:42 [notice] 1075#1075: start worker process 1144
1545010317: Client connection from 192.168.0.30 failed: error:140260FC:SSL routines:ACCEPT_SR_CLNT_HELLO:unknown protocol.
1545010317: New connection from 192.168.0.30 on port 4883.
1545010364: OpenSSL Error: error:140260E5:SSL routines:ACCEPT_SR_CLNT_HELLO:ssl handshake failure
1545010364: Socket error on client <unknown>, disconnecting.
1545010380: New connection from 192.168.0.30 on port 4883.
1545010380: OpenSSL Error: error:140260FC:SSL routines:ACCEPT_SR_CLNT_HELLO:unknown protocol
1545010380: Socket error on client <unknown>, disconnecting.
1545010382: New connection from 192.168.0.30 on port 4883.
1545010383: OpenSSL Error: error:140260E5:SSL routines:ACCEPT_SR_CLNT_HELLO:ssl handshake failure
1545010383: Socket error on client <unknown>, disconnecting.
1545010383: New connection from 192.168.0.30 on port 4883.
1545010383: OpenSSL Error: error:140260FC:SSL routines:ACCEPT_SR_CLNT_HELLO:unknown protocol
1545010383: Socket error on client <unknown>, disconnecting.
1545010383: New connection from 192.168.0.30 on port 4883.
1545010385: OpenSSL Error: error:140260E5:SSL routines:ACCEPT_SR_CLNT_HELLO:ssl handshake failure
1545010385: Socket error on client <unknown>, disconnecting.
1545010385: New connection from 192.168.0.30 on port 4883.
1545010385: OpenSSL Error: error:140260FC:SSL routines:ACCEPT_SR_CLNT_HELLO:unknown protocol
1545010385: Socket error on client <unknown>, disconnecting.
1545010385: New connection from 192.168.0.30 on port 4883.
1545010389: OpenSSL Error: error:140260E5:SSL routines:ACCEPT_SR_CLNT_HELLO:ssl handshake failure
1545010389: Socket error on client <unknown>, disconnecting.
1545010389: New connection from 192.168.0.30 on port 4883.
1545010389: OpenSSL Error: error:140260FC:SSL routines:ACCEPT_SR_CLNT_HELLO:unknown protocol
1545010389: Socket error on client <unknown>, disconnecting.
1545010389: New connection from 192.168.0.30 on port 4883.
1545010397: OpenSSL Error: error:140260E5:SSL routines:ACCEPT_SR_CLNT_HELLO:ssl handshake failure
1545010397: Socket error on client <unknown>, disconnecting.
1545010397: New connection from 192.168.0.30 on port 4883.
1545010397: OpenSSL Error: error:140260FC:SSL routines:ACCEPT_SR_CLNT_HELLO:unknown protocol
1545010397: Socket error on client <unknown>, disconnecting.
1545010397: New connection from 192.168.0.30 on port 4883.
1545010413: OpenSSL Error: error:140260E5:SSL routines:ACCEPT_SR_CLNT_HELLO:ssl handshake failure
1545010413: Socket error on client <unknown>, disconnecting.
1545010413: New connection from 192.168.0.30 on port 4883.
1545010413: OpenSSL Error: error:140260FC:SSL routines:ACCEPT_SR_CLNT_HELLO:unknown protocol
1545010413: Socket error on client <unknown>, disconnecting.
1545010413: New connection from 192.168.0.30 on port 4883.
1545010445: OpenSSL Error: error:140260E5:SSL routines:ACCEPT_SR_CLNT_HELLO:ssl handshake failure
1545010445: Socket error on client <unknown>, disconnecting.
1545010445: New connection from 192.168.0.30 on port 4883.
1545010445: OpenSSL Error: error:140260FC:SSL routines:ACCEPT_SR_CLNT_HELLO:unknown protocol
1545010445: Socket error on client <unknown>, disconnecting.
1545010445: New connection from 192.168.0.30 on port 4883.
1545010471: OpenSSL Error: error:140260E5:SSL routines:ACCEPT_SR_CLNT_HELLO:ssl handshake failure
1545010471: Socket error on client <unknown>, disconnecting.
1545010487: New connection from 172.30.32.1 on port 1883.
1545010487: New client connected from 172.30.32.1 as home-assistant (c1, k60, u'mqtt_hass').
2018/12/16 18:35:11 [info] 1144#1144: *1 [lua] ha-auth.lua:59: authenticate(): Authenticated user against Home Assistant., client: 192.168.0.4, server: _, request: "GET / HTTP/1.1", host: "hassio:5713"

Steps to reproduce

Install hassio 0.83.3 and use the following settings

# use the default settings for mqtt addon
"broker": {
    "enabled": true,
    "enable_ws": false,
    "enable_mqtt": false,
    "enable_ws_ssl": true,
    "enable_mqtt_ssl": true,
    "allow_anonymous": false
  },

# configuration.yaml
mqtt:
  username: mqtt_hass
  password: !secret mqtt_password
  client_id: home-assistant
  discovery: true
  protocol: 3.1.1 #Same result using 3.1
  broker: 127.0.0.1
  port: 4883
  tls_version: '1.2' #Same results using 1.1
  tls_insecure: true

Instead of 127.0.0.1 I also tried the FQDN for the domain, and confirmed that the certificate was valid in the container:

docker exec -it homeassistant bash
bash-4.4# openssl s_client -connect hassio.example.com:4884 -tls1_2
CONNECTED(00000003)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = hassio.example.com
verify return:1

Workaround

It works if you switch "enable_mqtt": false to true in the plugin config, and then use the following mqtt config in home assistant:

mqtt:
  username: mqtt_hass
  password: !secret mqtt_password
  client_id: home-assistant
  discovery: true
  protocol: 3.1.1
  broker: 127.0.0.1
  port: 1883

[addons-assistant]

👋 Thanks for opening your first issue here! If you're reporting a 🐛 bug, please make sure you include steps to reproduce it. Also, logs, error messages and information about your hardware might be usefull.

[ludeeus]

If it works everywhere else, then the issue is with how Home Assistant connect to it.

[addons-assistant]

This thread has been automatically locked because it has not had recent activity. Please open a new issue for related bugs and link to relevant comments in this thread.