-
Notifications
You must be signed in to change notification settings - Fork 149
/
Copy pathreadme.txt
259 lines (178 loc) · 7.03 KB
/
readme.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
FreeBASIC Tests
===============
This is the directory for FreeBASIC compiler and runtime tests.
Requirements - Windows / Linux / Dos
------------------------------------
- FreeBASIC Compiler 1.06.0 or above
- a build environment with common *nix commands available
on the path
- make, sh, find, xargs, grep, sed, cat, rm
Summary
-------
All of the tests are made and the results reported though a single
main makefile located in the top directory of the test tree.
use 'make' to get a list of the options available.
Making the Tests
----------------
The following two commands will make all tests:
$ make unit-tests
generates fbc-tests[.exe]
$ make log-tests
generates failed-test-fb.log
generates failed-test-fblite.log
generates failed-test-qb.log
generates failed-test-deprecated.log
if all tests passed, the log file reports "None found"
Other Examples:
Use 'make unit-tests' to build the fbcunit-compatible tests. This
will create fbc-tests[.exe] at the top of the tree.
If the fbcunit library is not already made, 'make unit-tests'
will automatically make it.
Use 'make log-tests' to build all non-fbcunit type tests. This will
create log files, which are summarized and saved in 'failed.log'
for any failed tests.
Use 'make failed-tests' to rebuild just those tests that failed in
the last testing session using 'make log-tests'.
Use 'make clean' to clean-up files, or to force the test package to
rescan directories for new or dropped test files.
Use 'make mostlyclean' to clean-up nearly all the files when it is
known that no tests have been added or dropped between test sessions.
Use 'make log-tests FB_LANG=fb | fblite | qb | deprecated' to make a specific
set of -lang tests.
Options
-------
OS=DOS
Indicate that test suite is being built under a djgpp environment
FBC=/path/fbc
Specify the location of the fbc compiler
FB_LANG=fb | fblite | qb | deprecated
Specify the compiler dialect
DEBUG=1
Adds '-g' option to add debugging information.
EXTRAERR=1
Adds '-exx' option to add extra error checking.
ARCH=486
Adds '-arch ARCH' option when compiling tests.
TARGET=arch-os-variant
Adds '-target TARGET' option when compiling tests. This is used
for cross-compiling; normally you pass a binutils target triplet.
FPU=FPU|SSE
Adds '-fpu FPU|SSE' option when compiling tests.
ENABLE_CHECK_BUGS=1
Adds '-d ENABLE_CHECK_BUGS=1' when compiling tests. This is used
enable additional tests for known bugs that are not yet fixed.
ENABLE_CONSOLE_OUTPUT=1
Add '-d ENABLE_CONSOLE_OUTPUT' when compiling tests. This is used
to turn on printing output to the console.
How the tests are collected
---------------------------
All of the directories listed in 'dirlist.mk' in the makefile
variable $(DIRLIST) are scanned for files with the extension
'.bmk' or '.bas'. When matching filenames are found, the contents
of the file is then scanned to determine the method of testing
needed for that specific file. For adding tests in new folders
manually add the new folder names in dirlist.mk.
This scan autogenerates "unit-tests.inc", which holds all files to be
included in unit-tests.
For forcing a rescan you should run "make clean", which will remove all
files resulting from previous runs and leave a cleaned tests environment.
By default '-lang fb' tests are collected unless 'FB_LANG=?' option
is given.
fbcunit tests (unit-tests)
------------------------------
fbcunit compatible tests are linked with the fbcunit
library and most follow a structure as shown in the following
example:
Sample FBCUNIT compatible test (using SUITE/TEST macros)
' TEST_MODE : FBCUNIT_COMPATIBLE
#include "fbcunit.bi"
SUITE( pretest )
TEST( test_true )
CU_ASSERT_TRUE( true )
END_TEST
END_SUITE
' EOF
Sample FBCUNIT compatible test (using the 'old' way)
' TEST_MODE : FBCUNIT_COMPATIBLE
#include "fbcunit.bi"
namespace fbc_tests.pretest
sub test_true cdecl ()
CU_ASSERT_TRUE( true )
end sub
private sub ctor () constructor
fbcu.add_suite("fbc_tests.pretest")
fbcu.add_test("test_true", @test_true)
end sub
end namespace
' EOF
Consult the fbcunit documentation for a listing of available
assertions and testing methods (tests\fbcunit\inc\fbcunit.bi).
non-unit compatible test (log-tests)
-------------------------------------
Each non-fbcunit compatible test should have one of the following
test-mode tags present in the source file:
' One of:
' TEST_MODE : COMPILE_ONLY_OK
' TEST_MODE : COMPILE_ONLY_FAIL
' TEST_MODE : COMPILE_AND_RUN_OK
' TEST_MODE : COMPILE_AND_RUN_FAIL
' TEST_MODE : MULTI_MODULE_TEST
Files are scanned for these tags to build of list of tests to perform,
the method to use when performing the test and the expected result.
Files that do not have one of the tags present may not get tested.
Mutli-module tests and encoded files
------------------------------------
Mult-module tests are composed of a '.bmk' file and several source
files. An example .bmk file:
# One of:
# TEST_MODE : MULTI_MODULE_OK
# TEST_MODE : MULTI_MODULE_FAIL
# TEST_MODE : FBCUNIT_COMPATIBLE
MAIN := my_test.bas
SRCS := my_test1.bas my_test2.bas my_test3.bas
# EOF
Use a '.bmk' file to link modules compiled by other compilers, e.g.
C/C++. To do that just define a list of .o files as EXTRA_OBJS, and
provide build rules for those files. See e.g. namespace/cpp/cpp.bmk.
A '.bmk' can also be used to indicate the test type for a source file
that is not normally readable with various GNU based streams. For,
example UTF-32. The '.bmk' is read only for the tag, and at testing
time, is substituted with '.bas'.
Files
-----
Makefile
the main makefile for building tests and reporting results. This
makefile makes use of other makefile (sub-makefiles) to get the
job done.
dirlist.mk
list of directories that to be testsed. Edit this file when new
directories are added to the testing tree.
unit-tests.mk
a sub-makefile for building fbcunit compatible tests
log-tests.mk
a sub-makefile for building all other tests that are not fbcunit
compatible.
bmk-make.mk
a sub-makefile for building a single executable.
common.mk
some common variables needed by most other makefiles
unit-tests.inc
generated automatically when 'make unit-tests' is invoked.
Lists all of the fbcunit-compatible tests.
log-tests.inc
generated automatically when 'make log-tests' is invoked.
Lists all of the tests.
failed.log
a summary of the test results for all failed tests. To get more
information on a specific failed test, view the '.log' saved in
the same directory as the test file.
const/generator
helper program and database to generate and check const type tests
optimizations\generator
helper program to create extensive 'math-torture-*.bas' tests
check-suite-names.sh
maintenance tool to test that SUITE() names in basic source files are
named in an expected way. Test suite names should match the
directory/filename.bas where they are written. See comments in the bash
script for more information on the method used.
## EOF