Save tests to separate dir

[qumusabel]

Added option to save :tests files to a separate dir
(saves to .tests by default)

[npanuhin]

Hello!

1. I tested this and I like this new feature.
2. @qumusabel, you forgot to duplicate the setting for Windows and OSX :)
3. @Jatana, can you please explain how the test cases are stored? I see that the test cases file is empty and has a weight of zero KB.

[Jatana]

Hi

@npanuhin, the tests are simply stored in files <program_name>:tests (except if a custom suffix is specified). If there is at least one non-empty input, then the test file should not be empty. Try to specify "tests_file_suffix": "__tests" and check the content and the size of <program_name>__tests(there have recently been some problems with colon on Windows).

[npanuhin]

@Jatana, I still can't find anything in these files.

This is the screenshot taken after executing 2020.11.19/A.cpp file, using __tests suffix.

Left: executable file
Center: FastOlympicCoding page
Right: test case file

Currently, it creates the following structure:

• A.cpp
• __tests
  • A.cpp

Which contradicts what you said:

... the tests are simply stored in files <program_name>:tests (except if a custom suffix is specified). If there is at least one non-empty input, then the test file should not be empty. Try to specify "tests_file_suffix": "__tests" and check the content and the size of <program_name>__tests(there have recently been some problems with colon on Windows).

I am wondering, where is it storing data because I can not find any file containing test cases :)

And another question:
When this new setting to change the suffix was not implemented, where the test cases were stored? There was no folder created in the directory with the executable file.

[npanuhin]

I figured out what was going on:
I had the tests_dir setting set to __tests, but not the tests_file_suffix.

As I can see, currently there is not tests_file_suffix setting in the default settings file at all. Is that how it should be?

And still, the answer to the last question remains unclear:

When this new setting to change the suffix was not implemented, where the test cases were stored? There was no folder created in the directory with the executable file.

[npanuhin]

I think the : character should be removed as a default parameter because Windows doesn't support it and I think that was the problem. I still don't know where the test files were saved on my computer all the time while I was using FOC ¯ \ _ (ツ) _ / ¯

[Jatana]

@npanuhin, probably when .tests/main.cpp:tests is created, Windows also creates an empty file .tests/main.cpp (if it wasn't created before). For some reason, files like .tests/main.cpp:tests are not shown by default, so you were seeing the empty file main.cpp and not main.cpp:tests where the actual tests were stored.

[npanuhin]

@Jatana you are right, I finally found a file with the :tests suffix. In fact, they were always stored near the executables, but I didn't see them in either the Sublime sidebar or Windows explorer.
I read that the colon in filenames on Windows works a little differently. Maybe : should be replaced with some other character (as the default suffix), for example #? Otherwise, others will wonder where all tests are saved, just like me :)

[qumusabel]

For me, it stored the :tests files right next to the sources. That quickly cluttered the working dir (on linux the :tests files are visible), so I decided to try to fix that

[npanuhin]

@Jatana, I have another two questions:
Is there any reason to keep the settings menu under Tools -> FastOlympicCoding and not the common Settings -> Package Settings?
Why is the package called CppFastOlympicCoding and not just FastOlympicCoding? Python and Java are currently implemented as well.

[npanuhin]

@qumusabel, yes, that is what I am talking about (on Windows I couldn't find them).
Also, I want to say, that your PR can be improved in this way:

It would be very cool if a user could specify a separate folder on the computer so that all test cases from each executable file on the computer would be saved there (e.g. "C:/cache"). To implement this it is needed to save the files with some ID-name so that the names are different for each file.

[Jatana]

@Jatana, I have another two questions:
Is there any reason to keep the settings menu under Tools -> FastOlympicCoding and not the common Settings -> Package Settings?

Actually, I haven't known about that menu. I think it is a good idea to move it there.

Why is the package called CppFastOlympicCoding and not just FastOlympicCoding? Python and Java are currently implemented as well.

There are specific functions that only work with C++, so I decided to indicate it there.

[npanuhin]

There are specific functions that only work with C++, so I decided to indicate it there.

Ok, understand.

Actually, I haven't known about that menu. I think it is a good idea to move it there.

I don't know how to commit to this PR, so I will make another PR when this one is approved.

PS: I meant Preferences -> Package Settings

[npanuhin]

I read that the colon in filenames on Windows works a little differently. Maybe : should be replaced with some other character (as the default suffix), for example #? Otherwise, others will wonder where all tests are saved, just like me :)

@Jatana, @qumusabel, how about changing the :tests suffix to#tests? I also want to add the tests_file_suffix settings to all the settings on the default page so that users can easily edit it without cloning or extracting the package and looking for it in the code.

And let me repeat: in Windows : the character behaves illogically and incorrectly, which should not remain that way.

[qumusabel]

If : behaves incorrectly on Windows, then it should definitely be changed. However I don't really like #tests, it looks weird to me. How about just .tests?

[npanuhin]

Windows and UNIX operating systems have restrictions on valid characters that can be used in a file name. The list of illegal characters for each operating system, however, is different. For example, a UNIX file name can use a colon (:), but a Windows file name cannot use a colon (:).

How about using ꞉ instead of :? :)
If you don't see the difference, watch this answer.

But maybe we shouldn't use such non-standard characters in file names. So . is a good choice because it represents the file extension {filename}.cpp.tests, which is good practice.

[Jatana]

Actually, sometimes it is useful to have two or more programs sharing a same test file. Currently, I am trying to think of a tests file name format to can allow this feature.

[npanuhin]

What do you mean by “two or more programs”? Other Sublime plugins like Easy CP?

Or user's executables? If you mean them: Is this feature implemented now and how does changing one character in path affect it?

[Jatana]

I mean something like if I edit the tests for file A.cpp and run A.py, then A.py is tested on the same tests as A.cpp.
It has not been implemented yet.

[qumusabel]

How about a command/shortcut to copy tests from another source file?

[Jatana]

How about a command/shortcut to copy tests from another source file?

Possible, but I am thinking more about sharing option than copying. In the example above, If I edit tests in file A.py, I want tests for A.cpp to be changed accordingly.

Well, one solution is to store a link to A.cpp:tests in A.py:tests file.

[npanuhin]

I mean something like if I edit the tests for file A.cpp and run A.py, then A.py is tested on the same tests as A.cpp.
It has not been implemented yet.

Wow, an interesting feature!

But in some cases, you do not need this linking. What if a person wants A.cpp and A.py to be different? Then he can't do it in any way. So an additional link in the form of a command or a shortcut would be better. So your last idea fits well:

Well, one solution is to store a link to A.cpp:tests in A.py:tests file.

[qumusabel]

Well, one solution is to store a link to A.cpp:tests in A.py:tests file.

IMO, this would be easier than a sophisticated naming scheme. (Also can provide some expandability)

[npanuhin]

Again speaking about colons in filenames in NTFS: files with them...

• are invisible for any explorer
• are invisible for python (os.listdir, os.walk, etc.) unless you directly specify fiename
• are not regular files at all

Please have a look at this answer - it explains how NTFS streams basically work. When I ran the program mentioned in the answer, I got this:

Which indicates that the test-case "files" were there. But they probably aren't there... I don't quite understand how NTFS streams work, but I'm sure you didn't mean that when you created the :tests suffix :)

PS:

It would be very cool if a user could specify a separate folder on the computer so that all test cases from each executable file on the computer would be saved there (e.g. "C:/cache"). To implement this it is needed to save the files with some ID-name so that the names are different for each file.

[qumusabel]

It would be very cool if a user could specify a separate folder on the computer so that all test cases from each executable file on the computer would be saved there (e.g. "C:/cache"). To implement this it is needed to save the files with some ID-name so that the names are different for each file.

I don't think this is a good idea, because that folder is sure to become cluttered, and if I would need to do something with test files, the unique-id scheme would make it even more difficult to find the file I need. I personally prefer having a folder with test files in each of my working directories, rather than some centralized but obscure location. That way I can just copy the entire working dir somewhere else and it'll still work.

[npanuhin]

Ok, I got the point. It would be great to implement both approaches (absolute and relative paths) so that the user has a choice.