Automatically transform in-source links to in-wiki links

[ribtoks]

Hi

Thanks for developing github-wiki-action. There's a small problem with cross-links:

Steps to reproduce:

• create docs/First.md and docs/Second.md
• Add a link [Second](./Second.md) to the First.md
• The link works in GitHub UI
• Deploy Wiki
• The link does not work

[Andrew-Chen-Wang]

Sorry, I don't want to leave you hanging for too long like last time, but I've got finals coming up so I won't be able to return back until mid December when they're over. Apologies in advance.

[ribtoks]

@Andrew-Chen-Wang No problem and thanks for replying. Will be looking forward to your return.

[Andrew-Chen-Wang]

@ribtoks The error is not on this repository. Instead, GitHub wiki apparently follows MediaWiki's link syntax: https://stackoverflow.com/a/8972756 https://docs.github.com/en/communities/documenting-your-project-with-wikis/editing-wiki-content#adding-links

I've tested this with your example. Instead of [First](./First.md), you instead have to do [[First|First]]. You can test this out in the edit pane.

[ribtoks]

@Andrew-Chen-Wang Thanks for investigating. I do not agree about usage of MediaWiki example right in the .md files because then links in the .md files stop to work. The whole point of this GitHub Action is to deploy .md files into Wiki. If you would ruin your .md files right away (by inserting links that work only in deployed Wiki), then there's no difference from writing Wiki files right away. The purpose is to have both working .md files and Wiki.

[Andrew-Chen-Wang]

Hm in this case, a potential "loophole" is that in this action, I can create a quick regex and optionally change the links to be of MediaWiki format and commit the changes to the wiki git.

[ribtoks]

@Andrew-Chen-Wang Yes, changing links before the deploy sounds like a good idea!

[xabolcs]

I've tested this with your example. Instead of [First](./First.md), you instead have to do [[First|First]]. You can test this out in the edit pane.

The problem is not the markdown type linking! MediaWiki type linking also doesn't work for [[First|First.md]]!
The problem is that there is an extension (.md) in the link!

GitHub's wiki shows the page source if there is an extension in the link, like in [link to Second page with extension](./Second.md).
If one drops the extensions, like in [link to Second page without extension](Second) then it will work on the GitHub wiki.

If one would strip the extensions from the links with a script, then the title functionality should taken into account!
E.g.

This is a test with a [link to other page](Second.md "a nice title with an .md inside" ).

[Andrew-Chen-Wang]

yes but [link to other page] is annoying sinve we'd want to use relative paths.

[xabolcs]

Thats why I wrote: the deploy script should just remove the extension from the link.
No need to reformat it to MediaWiki.

[jcbhmr]

@ribtoks do you have an opinion on whether you think this should be a separate preprocessing step/action or included in this Andrew-Chen-Wang/github-wiki-action ?

- uses: octocat/strip-md-links@v1
  with:
    path: wiki/**.md
- uses: Andrew-Chen-Wang/github-wiki-action@v4

vs

- uses: Andrew-Chen-Wang/github-wiki-action@v4
  with:
    strip_md_links: true

Related to #21

/cc @xabolcs

[ribtoks]

I'd vote for the latter strip_md_links: true. It's possible that the octocat/strip-md-links will be gone, etc, but github-wiki-action will have it's own implementation, thus the user interface will not change.

[jcbhmr]

@ribtoks Maybe I'm misunderstanding, what do you mean by "change"? Do you mean that the action will auto-update? Have new params? I don't think it does that since it's pinned at @v1, and that would, according to semver, mean that it doesn't break. But I guess you could also pin it to a specific commit if you wanted to be sure like user/strip-md-links@abcdef123456

I guess the whole discussion comes down to how much coupling/complexity you, a user, want in your options and how much coupling/complexity, Andrew-Chen-Wang, the main dev is OK maintaining.

[jcbhmr]

I assume that the issue outlined here is this:

Check out our [How it works](./How-it-works.md) page!

That this 👆 doesn't work in the deployed GitHub wiki tab. That's because the URLs that GitHub wikis use are /wiki/How-it-works, not /blob/main/wiki/How-it-works.md. Thus, you need something like:

https://github.com/marketplace/actions/strip-markdown-extensions-from-links-action

for (const file of readDirSync(filesPath)) {
  const filename: string = file.toString()
  const oldContent: string = readFileSync(filename, 'utf8')
  const newContent: string = replacer.transformMarkdownLinks(oldContent)

  if (oldContent != newContent) {
    debug(filename + ' updated')
    writeFileSync(filename, newContent)
  }
}

to solve the problem. Am I right?

I think that this problem is a great thing that we should highlight in our examples in the readme! We can point out https://github.com/marketplace/actions/strip-markdown-extensions-from-links-action like this:

Preprocessing

You may wish to strip the [link](page.md) .md suffix from your links to make them viewable in GitHub source view (with the .md) as well as in GitHub wiki (without the .md; pretty URLs!). You can use a preprocessing action like Strip MarkDown extensions from links action to remove those .md suffixes before using this action. Here's an example:

publish-wiki:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v3
    - uses: impresscms-dev/strip-markdown-extensions-from-links-action@v1.0.0
      with:
        path: wiki
    - uses: Andrew-Chen-Wang/github-wiki-action@v4

❤️ If you have an awesome preprocessor action that you want to add here, let us know! We'd love to add an example.

see a potential version of it: https://github.com/Andrew-Chen-Wang/github-wiki-action/tree/jcbhmr#preprocessing

@ribtoks does this meet with your expectations of this action? thoughts? 🤔

[jcbhmr]

As time has passed, my opinion on whether or not this is in-scope for this action have changed.

Ok so I've somewhat lessened my opposition on this issue purely due to this project's name. This is "github-wiki-action" after all, so it makes sense to include everything, I guess. In not thrilled about tackling seemingly separate projects all in one thing, but if it fits the goal, i guess it fits the goal. 😕

SO in summary, while it was good to get v4 out and in the wild, I guess now it's time to revisit the goal of what "github-wiki-action" means. #26 Taken maximally, it does include this feature builtin.

[jcbhmr]

To do this, it would seem that it's relatively basic:

1. loop through each file
2. match /\[([^\]]+)?\]\(([^)]+)\)/g or /^\[(.*)\]:\s+(.*?)\s*(".*"|'.*'|\(.*\))?$/gm
3. $2 += .md
4. replace it!

Actually, that will fail with codeblocks or other stuff like [](./not-me) since it's a regex, not a true markdown parser...

To do it the right way, it looks like I need to use something like this:

const visit = require('unist-util-visit');

module.exports = function (options) {

  function visitor(node) {
    if (/\.md$/.test(node.url)) {
      node.url = node.url.replace(/\.md$/, "")
    }
  }

  function transform(tree) {
    visit(tree, ['link', 'linkReference'], visitor);
  }

  return transform;
};

https://github.com/Pondorasti/remark-img-links
https://github.com/zslabs/remark-relative-links