Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat(xUnitXmlPublisher): Include assertion message in test output #208

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

stefanbesler
Copy link
Contributor

@stefanbesler stefanbesler commented Mar 18, 2023

This PR write the actual assertion failure to the output of xUnitXmlPublisher. Previously only the message from test was written into message.

@dfreiberger fyi

@stefanbesler stefanbesler force-pushed the feature/xUnitXmlPublisher_AssertionFailure branch from 55a1946 to c4f6242 Compare November 18, 2023 16:37
@stefanbesler stefanbesler force-pushed the feature/xUnitXmlPublisher_AssertionFailure branch from c4f6242 to c7c8563 Compare November 18, 2023 16:40
@stefanbesler
Copy link
Contributor Author

stefanbesler commented Nov 18, 2023

Maybe some explanation on how to verify this:

  • Enable xUnit publishing by setting the following TcUnit parameters in TcUnitVerifier.plcproj
    • XUNITENABLEPUBLISH = TRUE

image

If any test fails, the C:\tcunit_xunit_testresults.xml will now correctly show the EXP and ACT values in the assertion message

<testsuites disabled="" failures="116" tests="144" time="6.1428994">
   <testsuite id="0" name="PRG_TEST.PrimitiveTypes" tests="46" failures="23" time="1.4893e-3">
      <testcase name="Test_ANY_Equals" classname="PRG_TEST.PrimitiveTypes" time="2.35e-5" status="PASS" />
      <testcase name="Test_ANY_Differ_DataType" classname="PRG_TEST.PrimitiveTypes" time="1.47e-5" status="FAIL">
         <failure message="Values differ; EXP: (Type class = INT), ACT: (Type class = UINT)" type="Array2D_LREAL" />
      </testcase>
      <testcase name="Test_BOOL_Equals" classname="PRG_TEST.PrimitiveTypes" time="3.8e-6" status="PASS" />
      <testcase name="Test_BOOL_Differ" classname="PRG_TEST.PrimitiveTypes" time="1.18e-5" status="FAIL">
         <failure message="Values differ; EXP: TRUE, ACT: FALSE" type="Array2D_LREAL" />
      </testcase>
      <testcase name="Test_BYTE_Equals" classname="PRG_TEST.PrimitiveTypes" time="4.4e-6" status="PASS" />
      <testcase name="Test_BYTE_Differ" classname="PRG_TEST.PrimitiveTypes" time="1.14e-5" status="FAIL">
         <failure message="Values differ; EXP: 0xAB, ACT: 0xCD" type="Array2D_LREAL" />
      </testcase>

Without the PR, failure > message only shows "Values differ"

@sagatowski
Copy link
Member

sagatowski commented Nov 23, 2023

I'll start to look into this as soon as your previous PR is fixed/completed.

See: #210 & #226

@sagatowski
Copy link
Member

@stefanbesler Have you noticed the problems described in #226?

@dfreiberger
Copy link
Contributor

@stefanbesler thanks for this, it will be good to have these more granular results.

@TimSalzmann
Copy link

@stefanbesler, Hi, I really want to have this feature. If I can help, please let me know

@sagatowski
Copy link
Member

I agree, this is a great feature.
I do however have to get the stuff fixed that will be included in version 1.3 first.
For this milestone (https://github.com/tcunit/TcUnit/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+1.3.0.0%22) there is only one ticket left to be solved, and this one has to be solved first before I look into integrating more PRs.

@dfreiberger & @TimSalzmann, you are of course welcome to help me figure out the problem!

@sagatowski sagatowski added this to the Release 1.4.0.0 milestone Nov 30, 2023
@tcunit tcunit deleted a comment from iadonkey Dec 2, 2023
@tcunit tcunit deleted a comment from iadonkey Dec 2, 2023
@tcunit tcunit deleted a comment from iadonkey Dec 2, 2023
@tcunit tcunit deleted a comment from iadonkey Dec 2, 2023
@sagatowski
Copy link
Member

Realized that the discussion in this PR went very OffTopic, cleared it up. Sorry for that!

@stefanbesler I'm working on the last pieces to release 1.3. After that, I will look into this PR.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants