Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fix website build failures & CIPs' assets copy #295

Merged
merged 1 commit into from
Jul 13, 2022

Conversation

KtorZ
Copy link
Member

@KtorZ KtorZ commented Jul 13, 2022

First of all, I greatly dislike this hacky and dirty build scripts.
I'll eventually bite the bullet and redo all this with a proper static
website generator.

There were a few issues:

(a) Assets and annexes were copied, but in a wrong folder
(e.g. cips/cip-0003/xxx instead of cips/cip3/xxx).

  I still don't understand why it was decided to remove the trailing
  zeros from the uri path. It would have been WAY easier to leave
  them and now, makes it difficult to change (because of all links
  already sent out in the wild).

(b) Some annexes are markdown files, and if copied and served as such,
are simply interpreted by the browser as binary / text files (and
thus, it offers to download them; browsers don't have native
markdown renderers). So for those, we have to render them as html
files and copy the html file over, not the markdown. Had to tweak
then all existing CIPs before rendering to make sure that links
would point to html files instead (possibly, could have left an
'.md' extension and hoped that browsers would be smart enough to
still interpret the result as html based on the body content..
but, still I can't really be sure of that behavior across all
browsers I am playing it safe).

(c) Some CIPs, like CIP-0052 were numbered with leading zeros in the
front-matter header (0052 instead of 52) causing the underlying
engine to interpret the number in octal base 0052 = 42 and thus
doing some funky stuff with the final location of that CIP.
Resulting in the url /cip/cip52 resolving to a 404 not found.

(a) & (b) fixes #109.

This is what the public folder now looks like after all the changes:

.
├── all
│   └── index.html
├── assets
│   ├── css
│   │   └── styles.css
│   └── images
│       └── logo.svg
├── cips
│   ├── cip1
│   │   ├── CIP-0001.md.html
│   │   ├── CIP_Flow.png
│   │   ├── index.html
│   │   └── LICENSE
│   ├── cip10
│   │   ├── CIP-0010.md.html
│   │   ├── index.html
│   │   ├── registry.json
│   │   └── registry.schema.json
│   ├── cip11
│   │   ├── CIP-0011.md.html
│   │   └── index.html
│   ├── cip12
│   │   ├── CIP-0012.md.html
│   │   ├── index.html
│   │   └── schema.json
│   ├── cip13
│   │   ├── CIP-0013.md.html
│   │   └── index.html
│   ├── cip14
│   │   ├── CIP-0014.md.html
│   │   └── index.html
│   ├── cip15
│   │   ├── CIP-0015.md.html
│   │   ├── index.html
│   │   ├── schema.cddl
│   │   └── test-vector.md.html
│   ├── cip16
│   │   ├── CIP-0016.md.html
│   │   ├── index.html
│   │   └── LICENSE
│   ├── cip17
│   │   ├── CIP-0017.json
│   │   ├── CIP-0017.md.html
│   │   └── index.html
│   ├── cip18
│   │   ├── CIP-0018.md.html
│   │   └── index.html
│   ├── cip1852
│   │   ├── CIP-1852.md.html
│   │   └── index.html
│   ├── cip1853
│   │   ├── CIP-1853.md.html
│   │   └── index.html
│   ├── cip1854
│   │   ├── CIP-1854.md.html
│   │   └── index.html
│   ├── cip1855
│   │   ├── CIP-1855.md.html
│   │   └── index.html
│   ├── cip19
│   │   ├── CIP-0019-byron-addresses.cddl
│   │   ├── CIP-0019-cardano-addresses.abnf
│   │   ├── CIP-0019.md.html
│   │   └── index.html
│   ├── cip2
│   │   ├── CIP-0002.md.html
│   │   └── index.html
│   ├── cip20
│   │   ├── CIP-0020.md.html
│   │   └── index.html
│   ├── cip21
│   │   ├── CIP-0021.md.html
│   │   └── index.html
│   ├── cip22
│   │   ├── CIP-0022.md.html
│   │   └── index.html
│   ├── cip23
│   │   ├── CIP-0023.md.html
│   │   ├── index.html
│   │   └── minfees.php
│   ├── cip24
│   │   ├── CIP-0024.md.html
│   │   └── index.html
│   ├── cip25
│   │   ├── cddl
│   │   │   ├── version_1.cddl
│   │   │   └── version_2.cddl
│   │   ├── CIP-0025.md.html
│   │   └── index.html
│   ├── cip26
│   │   ├── index.html
│   │   └── schema.json
│   ├── cip27
│   │   ├── CIP-0027.md.html
│   │   └── index.html
│   ├── cip28
│   │   └── index.html
│   ├── cip29
│   │   ├── CIP-0029.md.html
│   │   ├── index.html
│   │   ├── phase-1-monetary-scripts.cddl
│   │   └── phase-1-monetary-scripts.json
│   ├── cip3
│   │   ├── Byron.md.html
│   │   ├── CIP-0003.md.html
│   │   ├── Icarus.md.html
│   │   ├── index.html
│   │   └── Ledger_BitBox02.md.html
│   ├── cip30
│   │   └── index.html
│   ├── cip31
│   │   └── index.html
│   ├── cip32
│   │   └── index.html
│   ├── cip33
│   │   └── index.html
│   ├── cip34
│   │   ├── index.html
│   │   ├── registry.json
│   │   └── schema.json
│   ├── cip35
│   │   └── index.html
│   ├── cip36
│   │   ├── index.html
│   │   ├── schema.cddl
│   │   └── test-vector.md.html
│   ├── cip4
│   │   ├── CIP-0004.md.html
│   │   └── index.html
│   ├── cip40
│   │   └── index.html
│   ├── cip42
│   │   └── index.html
│   ├── cip5
│   │   ├── CIP-0005.md.html
│   │   ├── index.html
│   │   ├── LICENSE
│   │   └── README.md.orig
│   ├── cip52
│   │   ├── index.html
│   │   └── Tx-spec.md.html
│   ├── cip54
│   ├── cip55
│   │   └── index.html
│   ├── cip59
│   │   └── index.html
│   ├── cip6
│   │   ├── CIP-0006.md.html
│   │   ├── index.html
│   │   └── schema.json
│   ├── cip7
│   │   ├── CIP-0007.md.html
│   │   ├── index.html
│   │   └── rewards.php
│   ├── cip8
│   │   ├── CIP-0008.md.html
│   │   └── index.html
│   └── cip9
│       ├── CIP-0009.md.html
│       └── index.html
├── index.html
├── informational
│   └── index.html
├── process
│   └── index.html
├── standards
│   └── index.html
└── standards-track
    └── index.html

Screenshot from 2022-07-13 13-16-20

Screenshot from 2022-07-13 13-16-29

  First of all, I greatly dislike this hacky and dirty build scripts.
  I'll eventually bite the bullet and redo all this with a proper static
  website generator.

  There were a few issues:

  (a) Assets and annexes were copied, but in a wrong folder
      (e.g. `cips/cip-0003/xxx` instead of `cips/cip3/xxx`).

      I still don't understand why it was decided to remove the trailing
      zeros from the uri path. It would have been WAY easier to leave
      them and now, makes it difficult to change (because of all links
      already sent out in the wild).

  (b) Some annexes are markdown files, and if copied and served as such,
      are simply interpreted by the browser as binary / text files (and
      thus, it offers to download them; browsers don't have native
      markdown renderers). So for those, we have to render them as html
      files and copy the html file over, not the markdown. Had to tweak
      then all existing CIPs before rendering to make sure that links
      would point to html files instead (possibly, could have left an
      '.md' extension and hoped that browsers would be smart enough to
      still interpret the result as html based on the body content..
      but, still I can't really be sure of that behavior across all
      browsers I am playing it safe).

  (c) Some CIPs, like CIP-0052 were numbered with leading zeros in the
      front-matter header (`0052` instead of `52`) causing the underlying
      engine to interpret the number in octal base 0052 = 42 and thus
      doing some funky stuff with the final location of that CIP.
      Resulting in the url `/cip/cip52` resolving to a 404 not found.

  (a) & (b) fixes #109.

  This is what the public folder now looks like after all the changes:

  ```
  .
  ├── all
  │   └── index.html
  ├── assets
  │   ├── css
  │   │   └── styles.css
  │   └── images
  │       └── logo.svg
  ├── cips
  │   ├── cip1
  │   │   ├── CIP-0001.md.html
  │   │   ├── CIP_Flow.png
  │   │   ├── index.html
  │   │   └── LICENSE
  │   ├── cip10
  │   │   ├── CIP-0010.md.html
  │   │   ├── index.html
  │   │   ├── registry.json
  │   │   └── registry.schema.json
  │   ├── cip11
  │   │   ├── CIP-0011.md.html
  │   │   └── index.html
  │   ├── cip12
  │   │   ├── CIP-0012.md.html
  │   │   ├── index.html
  │   │   └── schema.json
  │   ├── cip13
  │   │   ├── CIP-0013.md.html
  │   │   └── index.html
  │   ├── cip14
  │   │   ├── CIP-0014.md.html
  │   │   └── index.html
  │   ├── cip15
  │   │   ├── CIP-0015.md.html
  │   │   ├── index.html
  │   │   ├── schema.cddl
  │   │   └── test-vector.md.html
  │   ├── cip16
  │   │   ├── CIP-0016.md.html
  │   │   ├── index.html
  │   │   └── LICENSE
  │   ├── cip17
  │   │   ├── CIP-0017.json
  │   │   ├── CIP-0017.md.html
  │   │   └── index.html
  │   ├── cip18
  │   │   ├── CIP-0018.md.html
  │   │   └── index.html
  │   ├── cip1852
  │   │   ├── CIP-1852.md.html
  │   │   └── index.html
  │   ├── cip1853
  │   │   ├── CIP-1853.md.html
  │   │   └── index.html
  │   ├── cip1854
  │   │   ├── CIP-1854.md.html
  │   │   └── index.html
  │   ├── cip1855
  │   │   ├── CIP-1855.md.html
  │   │   └── index.html
  │   ├── cip19
  │   │   ├── CIP-0019-byron-addresses.cddl
  │   │   ├── CIP-0019-cardano-addresses.abnf
  │   │   ├── CIP-0019.md.html
  │   │   └── index.html
  │   ├── cip2
  │   │   ├── CIP-0002.md.html
  │   │   └── index.html
  │   ├── cip20
  │   │   ├── CIP-0020.md.html
  │   │   └── index.html
  │   ├── cip21
  │   │   ├── CIP-0021.md.html
  │   │   └── index.html
  │   ├── cip22
  │   │   ├── CIP-0022.md.html
  │   │   └── index.html
  │   ├── cip23
  │   │   ├── CIP-0023.md.html
  │   │   ├── index.html
  │   │   └── minfees.php
  │   ├── cip24
  │   │   ├── CIP-0024.md.html
  │   │   └── index.html
  │   ├── cip25
  │   │   ├── cddl
  │   │   │   ├── version_1.cddl
  │   │   │   └── version_2.cddl
  │   │   ├── CIP-0025.md.html
  │   │   └── index.html
  │   ├── cip26
  │   │   ├── index.html
  │   │   └── schema.json
  │   ├── cip27
  │   │   ├── CIP-0027.md.html
  │   │   └── index.html
  │   ├── cip28
  │   │   └── index.html
  │   ├── cip29
  │   │   ├── CIP-0029.md.html
  │   │   ├── index.html
  │   │   ├── phase-1-monetary-scripts.cddl
  │   │   └── phase-1-monetary-scripts.json
  │   ├── cip3
  │   │   ├── Byron.md.html
  │   │   ├── CIP-0003.md.html
  │   │   ├── Icarus.md.html
  │   │   ├── index.html
  │   │   └── Ledger_BitBox02.md.html
  │   ├── cip30
  │   │   └── index.html
  │   ├── cip31
  │   │   └── index.html
  │   ├── cip32
  │   │   └── index.html
  │   ├── cip33
  │   │   └── index.html
  │   ├── cip34
  │   │   ├── index.html
  │   │   ├── registry.json
  │   │   └── schema.json
  │   ├── cip35
  │   │   └── index.html
  │   ├── cip36
  │   │   ├── index.html
  │   │   ├── schema.cddl
  │   │   └── test-vector.md.html
  │   ├── cip4
  │   │   ├── CIP-0004.md.html
  │   │   └── index.html
  │   ├── cip40
  │   │   └── index.html
  │   ├── cip42
  │   │   └── index.html
  │   ├── cip5
  │   │   ├── CIP-0005.md.html
  │   │   ├── index.html
  │   │   ├── LICENSE
  │   │   └── README.md.orig
  │   ├── cip52
  │   │   ├── index.html
  │   │   └── Tx-spec.md.html
  │   ├── cip54
  │   ├── cip55
  │   │   └── index.html
  │   ├── cip59
  │   │   └── index.html
  │   ├── cip6
  │   │   ├── CIP-0006.md.html
  │   │   ├── index.html
  │   │   └── schema.json
  │   ├── cip7
  │   │   ├── CIP-0007.md.html
  │   │   ├── index.html
  │   │   └── rewards.php
  │   ├── cip8
  │   │   ├── CIP-0008.md.html
  │   │   └── index.html
  │   └── cip9
  │       ├── CIP-0009.md.html
  │       └── index.html
  ├── index.html
  ├── informational
  │   └── index.html
  ├── process
  │   └── index.html
  ├── standards
  │   └── index.html
  └── standards-track
      └── index.html
  ```
@KtorZ KtorZ added the Bi-Weekly Notes / Editorial Housekeeping Publishing Bi-weekly meetings minutes / Mintor edits of public surrounding information label Jul 13, 2022
@KtorZ KtorZ added the Correction Fixing minor issue or typo label Jul 13, 2022
Copy link
Collaborator

@rphair rphair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm grateful that at least one person has a proper understanding how all this stuff works. I'm curious how you were able to debug & test this without influencing the production environment, but if switching to a better method later then maybe there's no point (?)

From the change to CIP-0059/README.md in this PR I'm assuming all relative links to asset files in the same directory need to be prefixed by ./... please correct me if I'm wrong.

Thanks & looking forward to seeing this in action 😍

@KtorZ
Copy link
Member Author

KtorZ commented Jul 13, 2022

you were able to debug & test this without influencing the production environment,

The setup is pretty easy to run locally actually. yarn build is all you need, it generates files in a public directory and then it's just a static website, so any webserver does the trick to serve them.

From the change to CIP-0059/README.md in this PR I'm assuming all relative links to asset files in the same directory need to be prefixed by ./... please correct me if I'm wrong.

If we want them to be rendered (i.e markdown files) that's correct. For images and other static files, it may not be necessary.

All-in-all, I want to get rid of these scripts and use a proper tool that'll handle this correctly and without us bothering too much. Plus, it may be the opportunity to do some redesign of the CIP website which is a bit... bland.

@rphair
Copy link
Collaborator

rphair commented Jul 13, 2022

great, similar to the developer portal... CIPs are already being rendered there, so maybe Docusaurus would also work here as a static site generator? Also working an issue there I found pandoc works well for markdown conversion if necessary.

@KtorZ
Copy link
Member Author

KtorZ commented Jul 13, 2022

I was aiming for simpler than Docusaurus. Been using it in a few other projects and, it's a bit heavy for the job (I mean, doing server-side rendering for rendering a static website, that's slightly overkill). Something more like hugo which just works great and has a minimal footprint (and integrated markdown renderer).

@KtorZ KtorZ merged commit c70f95e into master Jul 13, 2022
@KtorZ KtorZ deleted the issue-109-and-build-fixes branch July 13, 2022 17:58
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Bi-Weekly Notes / Editorial Housekeeping Publishing Bi-weekly meetings minutes / Mintor edits of public surrounding information Correction Fixing minor issue or typo
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Broken cross-CIP links on derived website
3 participants