Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Add 'heldout-test' support #300

Open
pdreiter opened this issue Apr 2, 2023 · 1 comment
Open

Add 'heldout-test' support #300

pdreiter opened this issue Apr 2, 2023 · 1 comment
Assignees
@pdreiter
Labels
feature

Comments

@pdreiter
Copy link
Collaborator

pdreiter commented Apr 2, 2023

During assessment, need to extend evaluation infrastructure to support heldout-tests.
@pdreiter pdreiter self-assigned this Apr 2, 2023
@pdreiter pdreiter pinned this issue Apr 2, 2023
@pdreiter
Copy link
Collaborator Author

pdreiter commented Apr 2, 2023

development is occurring on branch mpt_dj => subbranch heldout_test-dev

pdreiter added a commit that referenced this issue Apr 2, 2023
@pdreiter
set of changes related to new heldout-test feature (#300)
4d6cc46 
- tested with and without heldout-test content
- IMPACT: Changes the TestOutcome, s.t. these two cases:
  (without heldout tests) and (failing heldout tests) are indiscernible
pdreiter added a commit that referenced this issue Apr 7, 2023
@pdreiter
reimplementation of heldout-test evaluation structure based on @Chris…
ad084af 
…Timperley's feedback issue #300

- tested, but seeing some issues with test timeouts
@pdreiter pdreiter mentioned this issue Apr 7, 2023
Open
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@pdreiter pdreiter

Labels
feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@pdreiter