Skip to content

Expose Issue severity and isFailure as API #1075

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

suzannaratcliff
Copy link
Contributor

@suzannaratcliff suzannaratcliff commented Apr 14, 2025

Expose Issue Severity and isFailure as API

Motivation:

To allow users to create issues on their tests without causing the tests to fail, it would be helpful to support creating issues with a warning severity.

Test Warnings:
Severity:

  public enum Severity: Sendable {
    /// The severity level for an issue which should be noted but is not
    /// necessarily an error.
    ///
    /// An issue with warning severity does not cause the test it's associated
    /// with to be marked as a failure, but is noted in the results.
    case warning

    /// The severity level for an issue which represents an error in a test.
    ///
    /// An issue with error severity causes the test it's associated with to be
    /// marked as a failure.
    case error
  }

This will allow users to create issues but they will not have to fail the test

Example Usage:

Issue.record("My comment", severity: .warning)

Later on users can inspect the severity of their issue with the severity property.

/// The severity of the issue.
public var severity: Severity
  /// Whether or not this issue should cause the test it's associated with to be
  /// considered a failure.
  ///
  /// The value of this property is `true` for issues which have a severity level of
  /// ``Issue/Severity/error`` or greater and are not known issues via
  /// ``withKnownIssue(_:isIntermittent:sourceLocation:_:when:matching:)``.
  /// Otherwise, the value of this property is `false.`
  ///
  /// Use this property to determine if an issue should be considered a failure, instead of
  /// directly comparing the value of the ``severity`` property.
public var isFailure: Bool

This API will be useful for teams who would like to inspect if their issue is a failing issue
https://developer.apple.com/documentation/xctest/xctestcase/record(_:)

Modifications:

  • Add isFailure variable
  • Remove experimental spi from Severity

Checklist:

  • Code and documentation should follow the style of the Style Guide.
  • If public symbols are renamed or modified, DocC references should be updated.

public var severity: Severity

/// If the issues is a failing issue.
public var isFailure: Bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this member is necessary; whether or not a warning is a failure might depend on external configuration (some equivalent to -Werror perhaps.)

@grynspan grynspan added this to the Swift 6.x milestone Apr 14, 2025
@grynspan grynspan added enhancement New feature or request public-api Affects public API api-proposal API proposal PRs (documentation only) issue-handling Related to Issue handling within the testing library labels Apr 14, 2025
@suzannaratcliff suzannaratcliff force-pushed the suzannaratcliff/expose-severity-as-api branch from c383628 to 7616fc7 Compare April 15, 2025 18:09
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
api-proposal API proposal PRs (documentation only) enhancement New feature or request issue-handling Related to Issue handling within the testing library public-api Affects public API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants