Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

3094bbc broke fragments checking: InvalidBaseJoin #1574

Open
autoantwort opened this issue Nov 27, 2024 · 2 comments · May be fixed by #1624
Open

3094bbc broke fragments checking: InvalidBaseJoin #1574

autoantwort opened this issue Nov 27, 2024 · 2 comments · May be fixed by #1624
Labels
bug Something isn't working

Comments

@autoantwort
Copy link
Contributor

autoantwort commented Nov 27, 2024 •
edited
Loading

Before and after:

➜  lychee git:(87d5b56) cargo run -- ../network-documentation/services/zfs.md --include-fragments --base ../network-documentation/ --mode=emoji --timeout=5
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.87s
     Running `target/debug/lychee ../network-documentation/services/zfs.md --include-fragments --base ../network-documentation/ --mode=emoji --timeout=5`
  6/6 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                                                                                                                           
🔍 6 Total (in 0s) ✅ 6 OK 🚫 0 Errors
➜  lychee git:(87d5b56) git checkout -
Previous HEAD position was 87d5b56 Box Octocrab error as it is too large (#1543)
HEAD is now at 3094bbc Add support for relative links (#1489)
➜  lychee git:(3094bbc) cargo run -- ../network-documentation/services/zfs.md --include-fragments --base ../network-documentation/ --mode=emoji --timeout=5
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 16.22s
     Running `target/debug/lychee ../network-documentation/services/zfs.md --include-fragments --base ../network-documentation/ --mode=emoji --timeout=5`
[WARN] Error creating request: InvalidBaseJoin("#scrubs-wichtig")
[WARN] Error creating request: InvalidBaseJoin("#unterschiedliche-platten-in-einem-vdevpool")
  4/4 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                                                                                                                            
🔍 4 Total (in 0s) ✅ 4 OK 🚫 0 Errors

You can also reproduce that with the fragments folder in this repo:
Before: cargo run -- fixtures/fragments/ --include-fragments --mode=emoji --base .

  15/15 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                                                                                                                          Issues found in 2 inputs. Find details below.

[fixtures/fragments/file1.md]:
❌ file:///Users/leanderSchulten/git_projekte/lychee/fixtures/fragments/file2.md#missing-fragment
❌ file:///Users/leanderSchulten/git_projekte/lychee/fixtures/fragments/file1.md#missing-fragment

[fixtures/fragments/file.html]:
❌ file:///Users/leanderSchulten/git_projekte/lychee/fixtures/fragments/file.html#in-the-end

🔍 15 Total (in 0s) ✅ 12 OK 🚫 3 Errors

After:

⠁ 0/0 ━━━━━━━━━━━━━━━━━━━━ Extracting links                                                                                                                                                                     [WARN] Error creating request: InvalidBaseJoin("file1.md#lets-wear-a-hat-être")
[WARN] Error creating request: InvalidBaseJoin("#fragment-2")
[WARN] Error creating request: InvalidBaseJoin("file2.md#fragment-1")
[WARN] Error creating request: InvalidBaseJoin("#missing-fragment")
[WARN] Error creating request: InvalidBaseJoin("file2.md#missing-fragment")
[WARN] Error creating request: InvalidBaseJoin("#code-heading")
[WARN] Error creating request: InvalidBaseJoin("#explicit-fragment")
[WARN] Error creating request: InvalidBaseJoin("file.html#a-word")
[WARN] Error creating request: InvalidBaseJoin("file2.md#custom-id")
[WARN] Error creating request: InvalidBaseJoin("#kebab-case-fragment")
[WARN] Error creating request: InvalidBaseJoin("#kebab-case-fragment-1")
[WARN] Error creating request: InvalidBaseJoin("empty_file#fragment")
[WARN] Error creating request: InvalidBaseJoin("file1.md#fragment-1")
[WARN] Error creating request: InvalidBaseJoin("#in-the-beginning")
[WARN] Error creating request: InvalidBaseJoin("#in-the-end")
  0/0 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                                                                                                                            🔍 0 Total (in 0s) ✅ 0 OK 🚫 0 Errors
@eishundo eishundo mentioned this issue Dec 5, 2024
Closed
@trask
Copy link
Contributor

trask commented Dec 6, 2024

I suspect this issue relates to --base modeling Base URL, while probably what you want in this case is the upcoming --root-dir

@MichaIng
Copy link
Member

MichaIng commented Dec 8, 2024 •
edited
Loading

Not only fragment URLs are broken, but all relative URL checks:

   [WARN ] Error creating request: InvalidBaseJoin("../../software/vpn/")
   [WARN ] Error creating request: InvalidBaseJoin("../../software/webserver_stack/")
   [WARN ] Error creating request: InvalidBaseJoin("../")
   [WARN ] Error creating request: InvalidBaseJoin("../software_installation/")
   [WARN ] Error creating request: InvalidBaseJoin("../system_configuration/")
   [WARN ] Error creating request: InvalidBaseJoin("../system_maintenance/")
   [WARN ] Error creating request: InvalidBaseJoin("./")
   [WARN ] Error creating request: InvalidBaseJoin("#useful-dietpi-shell-functions")
   [WARN ] Error creating request: InvalidBaseJoin("#dietpi-banner")
   [WARN ] Error creating request: InvalidBaseJoin("#dietpi-cpu-info")

We use --base build with the HTML file structure all in ./build. Previously an URL ../../software/vpn/ in the HTML file ./build/docs/releases/v9_7/index.html was correctly resolved to the directory ./build/docs/software/vpn/. I actually wonder why --base was/is required at all in such case, as relative URLs can be always just resolved relative to the directory of the checked document. The help/docs say --base is for relative URLs, but isn't such needed for absolute URLs only, at least when checking for local files and directories?
EDIT: Or is the wording "relative URL" used for absolute file paths, i.e. relative to the base URL/domain, not relative to the directory structure?

However, when checking from any directory on the system and aiming to check relative path URLs against an online https://-like URLs, indeed the solution to introduce --root-dir and rename --base to --base-url seems the best solution:

  • --root-dir should allow to resolve all absolute path URLs like /path/to to local files and directories relative to this root directory.
  • --base-url allows to resolve absolute path URLs against online/scheme URLs, and in combination with --root-dir also relative URLs, resolving the path first relative to the --root-dir, then appending it to the given --base-url.

Checking relative path URLs, which include fragment-only URLs, against local files/dirs however should not require any of the two options, but just checking for a local file/dir relative to the directory of the checked document.

Based on #1489 and the linked issues, where relative path URLs never checked before? I though they were 🤔.
EDIT: Ah I see now in #1480 the problem:

  • OP uses a relative path, which was always resolved against the given --base URL, not taking into account the possible path elements of the input URL.
  • I am not sure why, but the fix broke relative local file resolution instead, including pure fragment URLs. Is the --base supposed to be given differently after the recent change, or is it a bug?

@mre mre added the bug Something isn't working label Dec 10, 2024
@mre mre mentioned this issue Dec 18, 2024
Closed
@grahamc grahamc mentioned this issue Dec 18, 2024
Closed
@MichaIng MichaIng mentioned this issue Jan 6, 2025
Closed
@mre mre mentioned this issue Jan 6, 2025
Closed
@mre mre linked a pull request Feb 11, 2025 that will close this issue
Draft
@chinatsu chinatsu mentioned this issue Mar 16, 2025
Merged
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix relative links lycheeverse/lychee
4 participants
@mre @trask @MichaIng @autoantwort