Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Check CAA record and /var/lib/ACME/accounts before cmdeploy run / cmdeploy dns #317

Closed
hagenest opened this issue Jun 4, 2024 · 21 comments
Closed

Check CAA record and /var/lib/ACME/accounts before cmdeploy run / cmdeploy dns #317

hagenest opened this issue Jun 4, 2024 · 21 comments

Comments

@hagenest
Copy link
Contributor

hagenest commented Jun 4, 2024

related to #311 (comment).

If a user wants to redeploy their chatmail instance they currently only get a long ACMEtool error. I think it'd be better if we instead warn them, when checking DNS, that they have to make a change to their CAA record.
@missytake
Copy link
Contributor

missytake commented Jun 4, 2024 •
edited
Loading

E.g., a check in the beginning of cmdeploy run:

  • if there is a CAA record,
  • but
    • acmetool account-url returns a different one,
    • or the command isn't installed yet (e.g. in the CI) in /var/lib/acme/certs/*/url is no fitting account ID,
  • recommend to delete the CAA record first.

@link2xt
Copy link
Contributor

link2xt commented Jun 21, 2024

Just make sure not to reintroduce #288.
The server may be not deployed at all yet, e.g. there may be no dig (and no unbound, but it is less of a problem) on the server, so trying to query CAA record might fail.

@abixadamj
Copy link

I have fresh Debian 12 on VPS:

root@deltachat:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)
Release:        12
Codename:       bookworm
root@deltachat:~# 

root@deltachat:~# acmetool --version
go version go1.19.8 linux/amd64 gc cgo=true
acmetool Debian version 0.2.2-1+b4
root@deltachat:~# dig -v
DiG 9.18.24-1-Debian
root@deltachat:~# python3 -V
Python 3.11.2


root@deltachat:~# ls -lR /var/lib/acme/
/var/lib/acme/:
total 28
drwx------ 2 root root 4096 Jul 10 03:38 accounts
drwxr-xr-x 2 root root 4096 Jul 10 02:59 certs
drwxr-xr-x 2 root root 4096 Jul 10 02:59 conf
drwxr-xr-x 2 root root 4096 Jul 10 02:59 desired
drwx------ 2 root root 4096 Jul 10 02:59 keys
drwxr-xr-x 2 root root 4096 Jul 10 02:59 live
drwx------ 2 root root 4096 Jul 10 02:59 tmp

/var/lib/acme/accounts:
total 0

/var/lib/acme/certs:
total 0

/var/lib/acme/conf:
total 0

/var/lib/acme/desired:
total 0

/var/lib/acme/keys:
total 0

/var/lib/acme/live:
total 0

/var/lib/acme/tmp:
total 0

When I try do deploy, I got en Error:

adasiek@adasiek-neon:~/chatmail$ scripts/cmdeploy run
[ssh] login to deltachat.jurkiewicz.chat
Checking DNS entries .20240710033914 [CRITICAL] acmetool: fatal: get account URL: HTTP error: 400 Bad Request
(problem (type "urn:ietf:params:acme:error:accountDoesNotExist") (instance "") (id <nil>) (title ""): (detail "No account exists with the provided key"))
...Traceback (most recent call last):
  File "/home/adasiek/chatmail/venv/bin/cmdeploy", line 8, in <module>
    sys.exit(main())
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 299, in main
    res = args.func(args, out, **kwargs)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 56, in run_cmd
    retcode, remote_data = show_dns(args, out)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/dns.py", line 21, in show_dns
    remote_data = sshexec(remote_funcs.perform_initial_checks, mail_domain=mail_domain)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/sshexec.py", line 16, in __call__
    code, data = self._remote_cmdloop_channel.receive(timeout=self.timeout)
  File "/home/adasiek/chatmail/venv/lib/python3.10/site-packages/execnet/gateway_base.py", line 934, in receive
    raise self._getremoteerror() or EOFError()
execnet.gateway_base.RemoteError: Traceback (most recent call last):
  File "<string>", line 1291, in executetask
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 104, in <module>
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 42, in perform_initial_checks
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 59, in get_ip_address
OSError: [Errno 101] Network is unreachable

I am little confised.... Can you give me some advise, what could I check?

@hpk42
Copy link
Contributor

hpk42 commented Jul 10, 2024 via email

@abixadamj
Copy link

Yes.. 'main' branch.
My firewall on Debian is simple:

root@deltachat:~# ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
Anywhere                   ALLOW IN    188.68.237.191            
Anywhere                   ALLOW IN    172.30.0.0/24             
80                         ALLOW IN    Anywhere                  
443                        ALLOW IN    Anywhere                  
587                        ALLOW IN    Anywhere                  
465                        ALLOW IN    Anywhere                  
143                        ALLOW IN    Anywhere                  
993                        ALLOW IN    Anywhere                  
22/tcp                     ALLOW IN    Anywhere                  
80 (v6)                    ALLOW IN    Anywhere (v6)             
443 (v6)                   ALLOW IN    Anywhere (v6)             
587 (v6)                   ALLOW IN    Anywhere (v6)             
465 (v6)                   ALLOW IN    Anywhere (v6)             
143 (v6)                   ALLOW IN    Anywhere (v6)             
993 (v6)                   ALLOW IN    Anywhere (v6)             
22/tcp (v6)                ALLOW IN    Anywhere (v6)

I have made modification...:

def get_ip_address(typ):
    print("Trying to connect....0 notifications.delta.chat, 443")
    sock = socket.socket(typ, socket.SOCK_DGRAM)
    sock.settimeout(0)
    print("Trying to connect....1 notifications.delta.chat, 443")
    sock.connect(("notifications.delta.chat", 443))
    return sock.getsockname()[0]

I can connect from my Debian:

root@deltachat:~# curl notifications.delta.chat
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head><title>Permanently Moved</title></head>
<body><h1>Permanently Moved</h1>
<p>This resource has <strong>moved permanently</strong> to
 <a href="https://notifications.delta.chat/">https://notifications.delta.chat/</a>.</p>
</body></html>root@deltachat:~# 
root@deltachat:~# 
root@deltachat:~# curl https://notifications.delta.chat
Hello, world!root@deltachat:~#

And now the error is:

adasiek@adasiek-neon:~/chatmail$ scripts/cmdeploy run
[ssh] login to deltachat.jurkiewicz.chat
Checking DNS entries .20240710065842 [CRITICAL] acmetool: fatal: get account URL: HTTP error: 400 Bad Request
(problem (type "urn:ietf:params:acme:error:accountDoesNotExist") (instance "") (id <nil>) (title ""): (detail "No account exists with the provided key"))
...Traceback (most recent call last):
  File "/home/adasiek/chatmail/venv/bin/cmdeploy", line 8, in <module>
    sys.exit(main())
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 299, in main
    res = args.func(args, out, **kwargs)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 56, in run_cmd
    retcode, remote_data = show_dns(args, out)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/dns.py", line 21, in show_dns
    remote_data = sshexec(remote_funcs.perform_initial_checks, mail_domain=mail_domain)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/sshexec.py", line 16, in __call__
    code, data = self._remote_cmdloop_channel.receive(timeout=self.timeout)
  File "/home/adasiek/chatmail/venv/lib/python3.10/site-packages/execnet/gateway_base.py", line 934, in receive
    raise self._getremoteerror() or EOFError()
execnet.gateway_base.RemoteError: Traceback (most recent call last):
  File "<string>", line 1291, in executetask
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 106, in <module>
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 42, in perform_initial_checks
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 61, in get_ip_address
OSError: [Errno 101] Network is unreachable

The strange behavior is - I cannot see any of my print from code.... what next?

@abixadamj
Copy link

Maybe there is a problem, that, I run cmdeploye few weeks ago on Ubuntu 24.04 and that domain - so there is already a Letsencrypt SSL cert generated for domain: 'jurkiewicz.chat' ?

@hpk42
Copy link
Contributor

hpk42 commented Jul 10, 2024 via email

@missytake
Copy link
Contributor

Maybe there is a problem, that, I run cmdeploye few weeks ago on Ubuntu 24.04 and that domain - so there is already a Letsencrypt SSL cert generated for domain: 'jurkiewicz.chat' ?

this should not be a problem. dig CAA jurkiewicz.chat doesn't show a record; if there was a record, this might prevent letsencrypt from giving you a new SSL/TLS certificate, but if there is no CAA record and you didn't run into the rate limit, it should be fine.

To be sure, you can delete your let's encrypt state with rm -rf /var/lib/acme; but a few messages before it was empty anyway, right?

@abixadamj
Copy link

OK, the output is:

adasiek@adasiek-neon:~/chatmail$ scripts/cmdeploy dns -v
[ssh] login to deltachat.jurkiewicz.chat
Checking DNS entries 
$ acmetool account-url
20240710082910 [CRITICAL] acmetool: fatal: get account URL: HTTP error: 400 Bad Request
(problem (type "urn:ietf:params:acme:error:accountDoesNotExist") (instance "") (id <nil>) (title ""): (detail "No account exists with the provided key"))
$ dig
$ unbound-control flush_zone deltachat.jurkiewicz.chat
$ openssl rsa -in /etc/dkimkeys/opendkim.private -pubout 2>/dev/null | awk '/-/{next}{printf("%s",$0)}'
Trying to connect....0 notifications.delta.chat, 443
Trying to connect....1 notifications.delta.chat, 443
Trying to connect....0 notifications.delta.chat, 443
Trying to connect....1 notifications.delta.chat, 443
Traceback (most recent call last):
  File "/home/adasiek/chatmail/venv/bin/cmdeploy", line 8, in <module>
    sys.exit(main())
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 299, in main
    res = args.func(args, out, **kwargs)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 86, in dns_cmd
    retcode, remote_data = show_dns(args, out)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/dns.py", line 21, in show_dns
    remote_data = sshexec(remote_funcs.perform_initial_checks, mail_domain=mail_domain)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/sshexec.py", line 16, in __call__
    code, data = self._remote_cmdloop_channel.receive(timeout=self.timeout)
  File "/home/adasiek/chatmail/venv/lib/python3.10/site-packages/execnet/gateway_base.py", line 934, in receive
    raise self._getremoteerror() or EOFError()
execnet.gateway_base.RemoteError: Traceback (most recent call last):
  File "<string>", line 1291, in executetask
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 106, in <module>
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 42, in perform_initial_checks
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 61, in get_ip_address
OSError: [Errno 101] Network is unreachable

I check openssl on chat server - ERROR:

root@deltachat:~# openssl rsa -in /etc/dkimkeys/opendkim.private -pubout
Could not open file or uri for loading private key from /etc/dkimkeys/opendkim.private
4097F64EA97F0000:error:16000069:STORE routines:ossl_store_get0_loader_int:unregistered scheme:../crypto/store/store_register.c:237:scheme=file
4097F64EA97F0000:error:80000002:system library:file_open:No such file or directory:../providers/implementations/storemgmt/file_store.c:267:calling stat(/etc/dkimkeys/opendkim.private)

I don't have /etc/opendkim directory on server!

@missytake
Copy link
Contributor

Yes, cmdeploy run needs to complete successfully first.

@abixadamj
Copy link

abixadamj commented Jul 10, 2024 •
edited
Loading

@missytake - but it failed:


adasiek@adasiek-neon:~/chatmail$ scripts/cmdeploy run -v
[ssh] login to deltachat.jurkiewicz.chat
Checking DNS entries 
$ acmetool account-url
20240710083754 [CRITICAL] acmetool: fatal: get account URL: HTTP error: 400 Bad Request
(problem (type "urn:ietf:params:acme:error:accountDoesNotExist") (instance "") (id <nil>) (title ""): (detail "No account exists with the provided key"))

@missytake
Copy link
Contributor

okay, it's a problem on our side. We will do a fix and get back to you :)

@abixadamj
Copy link

abixadamj commented Jul 10, 2024 •
edited
Loading

Write directly on Matrix:

https://matrix.to/#/@adam.jurkiewicz:matrix.org

I will pull repo and test.

@hpk42 hpk42 mentioned this issue Jul 10, 2024
Closed
@hpk42
Copy link
Contributor

hpk42 commented Jul 10, 2024 via email

@abixadamj
Copy link

abixadamj commented Jul 10, 2024 via email

@abixadamj
Copy link

@hpk42 ...

adasiek@adasiek-neon:~/chatmail$ git status
Na gałęzi hpk/debug317
nic do złożenia, drzewo robocze czyste


adasiek@adasiek-neon:~/chatmail$ scripts/cmdeploy run
[ssh] login to deltachat.jurkiewicz.chat
Checking DNS entries .20240710120822 [CRITICAL] acmetool: fatal: get account URL: HTTP error: 400 Bad Request
(problem (type "urn:ietf:params:acme:error:accountDoesNotExist") (instance "") (id <nil>) (title ""): (detail "No account exists with the provided key"))
...Traceback (most recent call last):
  File "/home/adasiek/chatmail/venv/bin/cmdeploy", line 8, in <module>
    sys.exit(main())
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 299, in main
    res = args.func(args, out, **kwargs)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 56, in run_cmd
    retcode, remote_data = show_dns(args, out)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/dns.py", line 21, in show_dns
    remote_data = sshexec(remote_funcs.perform_initial_checks, mail_domain=mail_domain)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/sshexec.py", line 16, in __call__
    code, data = self._remote_cmdloop_channel.receive(timeout=self.timeout)
  File "/home/adasiek/chatmail/venv/lib/python3.10/site-packages/execnet/gateway_base.py", line 934, in receive
    raise self._getremoteerror() or EOFError()
execnet.gateway_base.RemoteError: Traceback (most recent call last):
  File "<string>", line 1291, in executetask
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 104, in <module>
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 42, in perform_initial_checks
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 59, in get_ip_address
OSError: [Errno 101] Network is unreachable

adasiek@adasiek-neon:~/chatmail$ scripts/cmdeploy dns -v
[ssh] login to deltachat.jurkiewicz.chat
Checking DNS entries 
$ acmetool account-url
20240710120853 [CRITICAL] acmetool: fatal: get account URL: HTTP error: 400 Bad Request
(problem (type "urn:ietf:params:acme:error:accountDoesNotExist") (instance "") (id <nil>) (title ""): (detail "No account exists with the provided key"))
$ dig
$ unbound-control flush_zone deltachat.jurkiewicz.chat
$ openssl rsa -in /etc/dkimkeys/opendkim.private -pubout 2>/dev/null | awk '/-/{next}{printf("%s",$0)}'
Traceback (most recent call last):
  File "/home/adasiek/chatmail/venv/bin/cmdeploy", line 8, in <module>
    sys.exit(main())
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 299, in main
    res = args.func(args, out, **kwargs)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 86, in dns_cmd
    retcode, remote_data = show_dns(args, out)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/dns.py", line 21, in show_dns
    remote_data = sshexec(remote_funcs.perform_initial_checks, mail_domain=mail_domain)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/sshexec.py", line 16, in __call__
    code, data = self._remote_cmdloop_channel.receive(timeout=self.timeout)
  File "/home/adasiek/chatmail/venv/lib/python3.10/site-packages/execnet/gateway_base.py", line 934, in receive
    raise self._getremoteerror() or EOFError()
execnet.gateway_base.RemoteError: Traceback (most recent call last):
  File "<string>", line 1291, in executetask
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 104, in <module>
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 42, in perform_initial_checks
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 59, in get_ip_address
OSError: [Errno 101] Network is unreachable

@hpk42
Copy link
Contributor

hpk42 commented Jul 10, 2024 •
edited
Loading

could you "git pull" all recent changes?
the "get_ip_address" function failing here has been removed some hour ago so i suspect you are not up to date in your checkout.

@abixadamj
Copy link

OK, I removed dir and clone once again....

adasiek@adasiek-neon:~$ git clone https://github.com/deltachat/chatmail
Klonowanie do „chatmail”...
remote: Enumerating objects: 6186, done.
remote: Counting objects: 100% (2177/2177), done.
remote: Compressing objects: 100% (741/741), done.
remote: Total 6186 (delta 1635), reused 1480 (delta 1404), pack-reused 4009
Pobieranie obiektów: 100% (6186/6186), 2.17 MiB | 12.60 MiB/s, gotowe.
Rozwiązywanie delt: 100% (3450/3450), gotowe.
adasiek@adasiek-neon:~$ cd chatmail
adasiek@adasiek-neon:~/chatmail$ git checkout -b hpk/debug317
Przełączono na nową gałąź „hpk/debug317”
adasiek@adasiek-neon:~/chatmail$ git status
Na gałęzi hpk/debug317
nic do złożenia, drzewo robocze czyste
adasiek@adasiek-neon:~/chatmail$ git log
commit 4b0600a45392b36b94ff4587765b2015932b6a8d (HEAD -> hpk/debug317, origin/main, origin/HEAD, main)
Author: holger krekel <holger@merlinux.eu>
Date:   Tue Jul 9 23:52:23 2024 +0200

    be a bit more lenient on keeping old users

commit f1c10cac2b4b69ef0bc1b7193723141c7112958f
Author: holger krekel <holger@merlinux.eu>
Date:   Tue Jul 9 23:24:28 2024 +0200

    chunked deletion

Then I have made initenv and:

adasiek@adasiek-neon:~/chatmail$ scripts/cmdeploy init deltachat.jurkiewicz.chat
created config file for deltachat.jurkiewicz.chat in chatmail.ini
adasiek@adasiek-neon:~/chatmail$ scripts/cmdeploy run
[ssh] login to deltachat.jurkiewicz.chat
Checking DNS entries .20240710132049 [CRITICAL] acmetool: fatal: get account URL: HTTP error: 400 Bad Request
(problem (type "urn:ietf:params:acme:error:accountDoesNotExist") (instance "") (id <nil>) (title ""): (detail "No account exists with the provided key"))
...Traceback (most recent call last):
  File "/home/adasiek/chatmail/venv/bin/cmdeploy", line 8, in <module>
    sys.exit(main())
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 299, in main
    res = args.func(args, out, **kwargs)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/cmdeploy.py", line 56, in run_cmd
    retcode, remote_data = show_dns(args, out)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/dns.py", line 21, in show_dns
    remote_data = sshexec(remote_funcs.perform_initial_checks, mail_domain=mail_domain)
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/sshexec.py", line 16, in __call__
    code, data = self._remote_cmdloop_channel.receive(timeout=self.timeout)
  File "/home/adasiek/chatmail/venv/lib/python3.10/site-packages/execnet/gateway_base.py", line 934, in receive
    raise self._getremoteerror() or EOFError()
execnet.gateway_base.RemoteError: Traceback (most recent call last):
  File "<string>", line 1291, in executetask
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 104, in <module>
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 42, in perform_initial_checks
  File "/home/adasiek/chatmail/cmdeploy/src/cmdeploy/remote_funcs.py", line 59, in get_ip_address
OSError: [Errno 101] Network is unreachable

I don't understand...

@hpk42
Copy link
Contributor

hpk42 commented Jul 10, 2024

git checkout -b hpk/debug317 is missing to checkout the branch where things have been modified.
sorry if i wasn't clear enough on this.

@hpk42
Copy link
Contributor

hpk42 commented Jul 10, 2024

i suggest we discuss/feedback further on the PR #355

@hpk42
Copy link
Contributor

hpk42 commented Jul 28, 2024

#355 is merged and initial checks should be better now, also no IP-address detection takes place anymore.

@hpk42 hpk42 closed this as completed Jul 28, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

streamline DNS checks further chatmail/server
5 participants
@hpk42 @abixadamj @link2xt @hagenest @missytake