Snapshot tests for output using code samples #945
spenserblack
started this conversation in
Ideas
Replies: 1 comment 1 reply
-
To confirm, the plan would be to have separate code snippets for each language in the We would also have a script that can create a repository from a single code snippet, which would be used in individual tests for each language. The tests would capture a snapshot of the entire CLI output. Is this the overall idea? |
Beta Was this translation helpful? Give feedback.
1 reply
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
-
Starting this out as a discussion, we can convert to an issue if it sounds like a good idea. This may be unrealistic, but I think it would be pretty nice to have 🙂
I think it would be nice to get snapshot tests of the CLI output. ASCII, stats, coloring, everything. Here's a general idea of what that might look like. Using
cat path/to/snapshot
or evengit diff
gives a nice view of what the output looks like in your terminal.This can be helpful since the contents of the raw string can change, while keeping visual output the same. For example, reordering ANSI codes. Generating a snapshot is much easier than manually writing the expected string 🙂 Also, if we do something like add a new stat, we certainly would want to automate adding that change to all tests 😅
I'm thinking that, like github/linguist, we can have a
samples/
directory. To make adding snapshot tests as painless as possible, a tool should iterate through the samples, slugify their filenames, and create a test for each one. E.g.Jupyter Notebooks.ipynb
➡️fn jupyter_notebooks_snapshot()
. It may be tempting to just have 1 test that iterates through each sample in a for loop, but it will make it harder to tell which snapshot is failing, and if multiple are failing, if we don't have a test function for each sample.Additionally, I think it would be reasonable to use one git repo setup script that all tests can share.
I think we can leave it up to the community to provide code samples. github/linguist is very strict about requiring real-world examples, but I think we can be a bit more lenient and get away with reasonably formatted Hello Worlds.
Beta Was this translation helpful? Give feedback.
All reactions