Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

refactor: Add workflow spans and every cli logging #603

Merged
merged 8 commits into from
Mar 13, 2024

Conversation

bgins
Copy link
Contributor

@bgins bgins commented Mar 7, 2024

Description

This PR implements the following changes:

  • Add initial workflow and task execution spans
  • Add workflow initialize, start, and end logs
  • Add computed and replayed receipt logs
  • Add custom logging format and filter for EveryCLI
  • Minor re-wording of existing logs

Link to issue

Implements spans needed in #457

Type of change

  • New feature (non-breaking change that adds functionality)
  • Refactor (non-breaking change that updates existing functionality)

Test plan (required)

Run Homestar with EVERY_CLI set to true to see the simplified logs:

EVERY_CLI=true cargo run -- start

Check that RUST_LOG works when EVERY_CLI is false or not set:

EVERY_CLI=false RUST_LOG=info cargo run -- start

Screenshots

Running test-workflow-add-one.json with EVERY_CLI logs:

CleanShot 2024-03-12 at 22 10 28@2x

Same workflow, on replay:

CleanShot 2024-03-12 at 22 12 12@2x

A different workflow with WASI logging at trace level:

CleanShot 2024-03-12 at 22 17 42@2x

Error case where an arg with an unexpected type is passed to a workflow task:

CleanShot 2024-03-12 at 22 14 17@2x

@bgins bgins added observability Anything related to metrics, tracing, etc dx Developer experience applications and improvements labels Mar 7, 2024
@bgins bgins requested a review from a team as a code owner March 7, 2024 16:30
@bgins bgins changed the title feat: Add workflow spans and every cli logging refactor: Add workflow spans and every cli logging Mar 7, 2024
Copy link

codecov bot commented Mar 7, 2024

Codecov Report

Attention: Patch coverage is 58.82353% with 28 lines in your changes are missing coverage. Please review.

Project coverage is 69.88%. Comparing base (4118586) to head (45cb6cf).
Report is 45 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #603      +/-   ##
==========================================
+ Coverage   69.37%   69.88%   +0.51%     
==========================================
  Files          97       98       +1     
  Lines       13193    13394     +201     
==========================================
+ Hits         9152     9360     +208     
+ Misses       4041     4034       -7     
Files Coverage Δ
homestar-runtime/src/tasks/wasm.rs 100.00% <100.00%> (ø)
homestar-runtime/src/worker/resolver.rs 56.89% <100.00%> (+5.04%) ⬆️
homestar-wasm/src/wasmtime/host/helpers.rs 41.66% <100.00%> (-42.95%) ⬇️
homestar-wasm/src/wasmtime/world.rs 87.89% <100.00%> (-0.62%) ⬇️
homestar-runtime/src/runner.rs 90.65% <50.00%> (+1.33%) ⬆️
homestar-runtime/src/worker.rs 79.51% <87.50%> (-1.49%) ⬇️
homestar-runtime/src/logger.rs 70.58% <50.94%> (-29.42%) ⬇️

... and 24 files with indirect coverage changes

@bgins bgins self-assigned this Mar 7, 2024
Copy link
Contributor

@zeeshanlakhani zeeshanlakhani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a minor ask and that debug question remains, but it's not necessary. 👍🏽

@bgins bgins force-pushed the bgins/client-logging-filter branch 2 times, most recently from b7ad003 to 0e575a1 Compare March 12, 2024 15:15
@bgins bgins force-pushed the bgins/client-logging-filter branch from 0e575a1 to 4e30711 Compare March 13, 2024 05:21
Copy link
Contributor

@zeeshanlakhani zeeshanlakhani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple super minor comments, otherwise 👍🏽

@bgins bgins merged commit c3f9b4a into main Mar 13, 2024
33 checks passed
@bgins bgins deleted the bgins/client-logging-filter branch March 13, 2024 16:43
@release-plz-ipvm-wg release-plz-ipvm-wg bot mentioned this pull request Mar 13, 2024
bgins pushed a commit that referenced this pull request Mar 13, 2024
## 🤖 New release
* `homestar-runtime`: 0.2.0 -> 0.3.0 (⚠️ API breaking changes)
* `homestar-invocation`: 0.2.0 -> 0.3.0 (⚠️ API breaking changes)
* `homestar-wasm`: 0.2.0 -> 0.3.0 (⚠️ API breaking changes)
* `homestar-workflow`: 0.2.0 -> 0.3.0 (✓ API compatible changes)

### ⚠️ `homestar-runtime` breaking changes

```
--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.29.1/src/lints/enum_variant_added.ron

Failed in:
  variant Command:Init in /tmp/.tmpgzntbb/homestar/homestar-runtime/src/cli.rs:152
```

### ⚠️ `homestar-invocation` breaking changes

```
--- failure enum_tuple_variant_field_added: pub enum tuple variant field added ---

Description:
An enum's exhaustive tuple variant has a new field, which has to be included when constructing or matching on this variant.
        ref: https://doc.rust-lang.org/reference/attributes/type_system.html#the-non_exhaustive-attribute
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.29.1/src/lints/enum_tuple_variant_field_added.ron

Failed in:
  field 1 of variant Nonce::Nonce96 in /tmp/.tmpgzntbb/homestar/homestar-invocation/src/task/instruction/nonce.rs:39
  field 1 of variant Nonce::Nonce128 in /tmp/.tmpgzntbb/homestar/homestar-invocation/src/task/instruction/nonce.rs:41

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.29.1/src/lints/enum_variant_added.ron

Failed in:
  variant Error:FromMultibase in /tmp/.tmpgzntbb/homestar/homestar-invocation/src/error.rs:38
  variant Error:FromMultibase in /tmp/.tmpgzntbb/homestar/homestar-invocation/src/error.rs:38
```

### ⚠️ `homestar-wasm` breaking changes

```
--- failure auto_trait_impl_removed: auto trait no longer implemented ---

Description:
A public type has stopped implementing one or more auto traits. This can break downstream code that depends on the traits being implemented.
        ref: https://doc.rust-lang.org/reference/special-types-and-traits.html#auto-traits
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.29.1/src/lints/auto_trait_impl_removed.ron

Failed in:
  type State is no longer Sync, in /tmp/.tmpgzntbb/homestar/homestar-wasm/src/wasmtime/world.rs:41
  type State is no longer Sync, in /tmp/.tmpgzntbb/homestar/homestar-wasm/src/wasmtime/world.rs:41

--- failure enum_variant_missing: pub enum variant removed or renamed ---

Description:
A publicly-visible enum has at least one variant that is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.29.1/src/lints/enum_variant_missing.ron

Failed in:
  variant InterpreterError::MapType, previously in file /tmp/.tmpEeUFOH/homestar-wasm/src/error.rs:36
```

<details><summary><i><b>Changelog</b></i></summary><p>

## `homestar-runtime`
<blockquote>

##
[0.3.0](homestar-runtime-v0.2.0...homestar-runtime-v0.3.0)
- 2024-03-13

### Added
- log the creation of the key file in `init`
- default `key-file` path to output directory
- support generating PEM encoded ed25519 keys from `init`
- translate `InquireError` to `miette::Result` in `init`
- recursively create config directory on `init`
- output a cleaner error when an invalid seed is passed to `init`
- support configuring key using `init` command
- run `init` command non-interactively if a TTY isn't detected
- add `--no-input` to `init` command
- add `--force` to `init` command for forcing destructive operations
- add `--quiet` flag to `init` command
- support `--dry-run` for `init` command
- add `init` command for initializing a settings file
- load settings from a well-known config file
- finish interpreter
([#595](#595))

### Fixed
- cleanup empty key file when bailing out of generating secp256k1
- parse ed25519 keys using the old strategy as a fallback
- use `ed25519-dalek` for parsing PEM-encoded
PKCS[#8](#8) ed25519 keys
- only constrain `inquire` and `derive_builder` by minor version
- hide `IpfsSettings` behind "ipfs" feature
- set `truncate(true)` when forcefully overwriting the config
- Update nonce schema with IPLD bytes
([#593](#593))

### Other
- Add workflow spans and every cli logging
([#603](#603))
- *(schemas)* update OpenRPC API doc and JSON schemas
- handle nonce as incoming string/arraybuf
([#611](#611))
- [chore(cargo)](deps): Bump toml from 0.8.10 to 0.8.11
([#612](#612))
- document that a random seed will be chosen if `key-seed` is unset
- document that if unset, a default path is used with `key-file`
- update help text for `key-file` to say it'll generate a key
- prompt for the key file as a `String` instead of `PathBuf`
- add a test for writing the generated config file + key
- split `force` field out of `OutputMode::File`
- remove `KeyTypeArg` in favor of using `KeyType`
- remove unneeded `defaults.toml`
- add simple tests for `init` command
- remove out of date TODO in `init.rs`
- remove unneeded `#[allow(dead_code)]` in `settings.rs`
- wrap all `init` args in `InitArgs` and consolidate handling
- sort imports in `cli/init.rs`
- remove docs link to private `homestar_runtime::db::pool`
- improve error for passing `--no-input` to `init` with no key
- remove extraneous `...` destructuring of `Command::Init`
- change `--config` flag to `--output` for `init` command
- move handling of `init` command to `init.rs`
- fix comments listing supported public key types
- *(schemas)* update OpenRPC API doc and JSON schemas
- *(schemas)* update OpenRPC API doc and JSON schemas
- poll DHT in background when worker runs up a workflow + dual-stack
webserver ([#590](#590))
- [chore(cargo)](deps): Bump config from 0.13.4 to 0.14.0
([#588](#588))
- [chore(cargo)](deps): Bump nix from 0.27.1 to 0.28.0
([#587](#587))
</blockquote>

## `homestar-invocation`
<blockquote>

##
[0.3.0](homestar-invocation-v0.2.0...homestar-invocation-v0.3.0)
- 2024-03-13

### Added
- finish interpreter
([#595](#595))

### Fixed
- Update nonce schema with IPLD bytes
([#593](#593))

### Other
- handle nonce as incoming string/arraybuf
([#611](#611))
- test json/ipld/nonce
([#610](#610))
- poll DHT in background when worker runs up a workflow + dual-stack
webserver ([#590](#590))
</blockquote>

## `homestar-wasm`
<blockquote>

##
[0.3.0](homestar-wasm-v0.2.0...homestar-wasm-v0.3.0)
- 2024-03-13

### Added
- finish interpreter
([#595](#595))

### Other
- Add workflow spans and every cli logging
([#603](#603))
- handle incoming as Wit integer, but argument is float
([#609](#609))
- handle ref/non-ref case with rigor
([#608](#608))
- doc interpreter ([#607](#607))
- poll DHT in background when worker runs up a workflow + dual-stack
webserver ([#590](#590))
- wasmtime 17->18 ([#585](#585))
</blockquote>

## `homestar-workflow`
<blockquote>

##
[0.3.0](homestar-workflow-v0.2.0...homestar-workflow-v0.3.0)
- 2024-03-13

### Other
- handle nonce as incoming string/arraybuf
([#611](#611))
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/MarcoIeni/release-plz/).

---------

Signed-off-by: release-plz-ipvm-wg[bot] <144082651+release-plz-ipvm-wg[bot]@users.noreply.github.com>
Co-authored-by: release-plz-ipvm-wg[bot] <144082651+release-plz-ipvm-wg[bot]@users.noreply.github.com>
Co-authored-by: release-plz-ipvm-wg[bot] <release-plz-ipvm-wg[bot]@users.noreply.github.com>
@release-plz-ipvm-wg release-plz-ipvm-wg bot mentioned this pull request Mar 13, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
dx Developer experience applications and improvements observability Anything related to metrics, tracing, etc
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants