Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Support XCTest, Expecta and Nimble assertions in Kiwi specs #636

Merged
merged 10 commits into from
Aug 25, 2015

Conversation

sharplet
Copy link
Contributor

This change allows the use of XCTest assertions in specs. This should hopefully include other matcher libraries implemented in terms of XCTest assertions!

To be clear, I'm thinking of adding some more functional specs in this PR to test out whether this will enable support for Expecta or Nimble matchers.

This now works:

#import <Kiwi/Kiwi.h>

SPEC_BEGIN(KWObjCXCTestAssertionTests)

describe(@"XCTest assertions in Objective C", ^{
    it(@"supports XCTAssert", ^{
        XCTAssert(1 + 1 == 3); // <- fails with a real test failure instead of an exception or warning!
    });
});

SPEC_END

  • Within +buildExampleGroups, defines a shadow self pointer that is of the type of the test case instance.
  • Unrecognized selectors sent to the test case class are forwarded to
    the current example, and then to the current test case instance.
  • Adds a test for the use of XCTest assertions within specs.

This should possibly be considered a breaking change, because any users depending on the type of self being their test case class (and for example invoking class methods on KWSpec different in a category) will now get compiler warnings (that would be compiler errors in the case of -Werror).

However:

  • This will just generate compiler warnings that could safely be ignored, as the tests themselves won't fail because self is still actually the class object at runtime!
  • It should be fairly straightforward to redefine any of those class methods as instance methods.
  • While valid code, it's not an encouraged or documented way of writing test helper methods.

To make this not a breaking change (at compile time), a workaround could be for this PR to change the type of self to id. This is probably even more accurate! However the convenience of the static type declaration is useful. I just don't know if this will be an issue in practice for Kiwi users.

XCTest assertions in Objective-C require self to point to an instance
of XCTestCase. Unfortunately Kiwi currently builds examples from within
a class method `+buildExampleGroups`, so self points to the class
XCTestCase, rather than the running test case instance.
Allows the use of XCTest assertions in specs. This should include other
matcher libraries implemented in terms of XCTest assertions.

- Within `+buildExampleGroups`, defines a shadow `self` pointer that is
  of the type of the test case instance.
- Unrecognized selectors sent to the test case class are forwarded to
  the current example, and then to the current test case instance.
- Adds a test for the use of XCTest assertions within specs.

describe(@"XCTest assertions in Objective C", ^{
it(@"supports XCTAssert", ^{
XCTAssert(1 + 1 == 2);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test passes. In fact, it probably would have passed without any of the other changes in this PR. 0_o

The important thing is actually not that this test passes, but that when it fails it fails in the right way. I don't actually know how to verify that. ¯_(ツ)_/¯

@supermarin
Copy link

👍 for the code.
Will give it a bit of thought on how to test your assertion

- Builds Expecta.framework via Carthage.
- Overrides `+respondsToSelector:` in `KWSpec` to delegate to the
  current example, allowing failures to be recorded via the current
  test case instance.
- Builds Nimble.framework via Carthage.
- Adds Swift and Objective-C specs for Nimble support.
@sharplet sharplet changed the title Support XCTest assertion macros in Kiwi specs Support XCTest, Expecta and Nimble assertions in Kiwi specs Aug 20, 2015
@sharplet
Copy link
Contributor Author

I've just pushed a couple more commits adding explicit support for both Expecta and Nimble matchers!

@sharplet
Copy link
Contributor Author

Been thinking on testing that assertions behave correctly, and I could potentially follow a similar approach to this: https://github.com/kiwi-bdd/Kiwi/blob/master/test_suite_configuration.rb.

Basically:

  • Create a separate test bundle for testing matcher failures, and move the XCTest, Expecta and Nimble specs there.
  • Write a script that matches on the output of running that test bundle.
  • Integrate the script into CI.

However, it would be a lot nicer to have it properly integrated into the existing test suite.

Introduces `KWFailureInterceptingSpec` which can intercept failures recorded
by XCTest. This enables a custom `haveFailed` matcher to be defined in order
to assert that a given block records a test failure.

Swift specs will need to be handled differently, mainly because Kiwi matchers
aren't available in Swift yet.
@sharplet
Copy link
Contributor Author

Yo @supermarin, came up with something for testing that assertions occur: a77c3e7.

@supermarin
Copy link

:shipit:

@sharplet
Copy link
Contributor Author

I looked into trying to do write the same kind of test in the Swift specs, but we don't even support full Kiwi matchers in Swift yet. And it's working really well in Objective-C, so I don't think it's necessary to duplicate the effort in this PR.

sharplet added a commit that referenced this pull request Aug 25, 2015
Support XCTest, Expecta and Nimble assertions in Kiwi specs
@sharplet sharplet merged commit 43b798f into master Aug 25, 2015
@sharplet sharplet deleted the swift-spec branch August 25, 2015 11:40
@aspcartman
Copy link
Contributor

It's not working for me

#import <Kiwi/Kiwi.h>
#import <Expecta/Expecta.h>

SPEC_BEGIN(TestSpec)

describe(@"Just a test", ^{
    it(@"Works",^{
        expect(1).to.equal(1);
    });
});

SPEC_END

It errors on Expecta/EXPMatchers+beNil.h on (3,21): expected ')'

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants