diff --git a/internal/docs/common/TDG.html b/internal/docs/common/TDG.html deleted file mode 100644 index 0bceddb4fa..0000000000 --- a/internal/docs/common/TDG.html +++ /dev/null @@ -1,1590 +0,0 @@ - - -
- - - - - - - - - - - -1.0 Creating an Assertion List for a Technology
-Specification
-
- -just point to Kevin's process
-doc?
-
-2.0 Creating a TestPlan
-
-3.0 Developing Tests for a Technology
-
- 3.1 End-to-End Tests
-
- 3.2 API (Service) Tests
-
- 3.3 Signature Tests
-
- 3.4 Using Existing Porting classes
-
- 3.5 Common Apps
-
- 3.6 Source File headers
-(copyrights
-and sccs keywords)
-
-4.0 Creating a
-Deliverable (TCK)
-
-5.0 Building Tests
-
-6.0 Creating a
-Porting Class
-7.0 Running Tests
-
1.0 Creating an Assertion List for a Technology -Specification -
-For each Technology specification, an -assertion -list must be created. For details, please see the Assertion -Process -document (link here). In general, two documents will exist for -each -technology specification, one from the written specification and -another -from the corresponding javadoc. Each one must be an xml document -and must follow the rules in the assertion list dtds (links to spec and -javadoc dtds here). These documents must be checked into the -workspace -under $TS_HOME/internal/docs/<tech-name>/<spec-version> -subtree (e.g. -internal/docs/ejb/2.1). The documents should be named -<tech-name>SpecAssertions.xml -and <tech-name>JavadocAssertions.xml. At build time, the -xml documents -should be run through a predefined style sheet to create external html -versions of the lists which will be included in any deliverables. -
-1.1 XML Tools -
-Describe available tools - both editing tools and -utilities from the tools ws. -
-2.0 Creating a Test Plan for a Technology -Specification -
-Test plans should -be -created according to the template (link to Steve C's doc) -
-3.0 Developing tests -for -a particular technology -
-All tests that are integrated into the spider -workspace -must follow this list of conventions... -
-Checklist of Conventions for All tests -
-1. All test names should be meaningful and should not include
-numbers. Instead of simply naming tests... test1, test2, test3,
-use
-names like, statefulCreateTest, resultSetTest, etc.
-
-1. Test Clients must include test description tags as well as
-setup.props tags.
-
-2. All test names must have the string "test" (case-insensitive)
-in it.
-
-3. All technology areas should have a top level directory under
-the "com.sun.ts.tests" package.
-
-4. All source java sources must live under the $TS_HOME/src
-directory
-and under src, the directory structure must correspond to the java
-package
-for sources.
-
-5. All web content used by tests must live under the
-$TS_HOME/src/web
-directory.
-
-4. No TCK or Deliverable specific strings should appear under
-the src tree (with the exception of Deliverable specific files under
-the
-"com.sun.ts.lib.deliverable.<deliv-name>" package. For example,
-do not
-name a source file CTSTests.java, or name a variable name sCTSString.
-
-5. All source files should be saved in unix file format.
-If you use an editor on the win32 platform, be sure to save the file in
-unix format prior to checkin. Some PC editors have an option to
-do
-this automatically.
-
-6. Follow the Java Software coding conventions http://???.
-
-7. All directory names should be lowercase.
-
Each test always begins in a test Client, either -an application client for a URL client. The Client's .java file -is -required to contain a test description for each test that begins in -that -class. Test Descriptions are written in Javadoc comments withthe -exception of the addtional '*' that is required by Javadoc -comments. -Just use standard multiline comment notation. Here are some -sample -setup and Test Description tags from a JAX-RPC client .java file... -
- /*
-
-
-* @class.setup_props: webServerHost;
-
-
-*
-webServerPort;
-
-
-*
-ts_home;
-
-
-*/
-
-/*
-
-
-* @testName: AttachmentsTest
-
-
-*
-
-
-* @assertion_ids: JAXRPC:SPEC:41; JAXRPC:SPEC:42;
-JAXRPC:SPEC:304;
-
-
-*
-JAXRPC:SPEC:310;
-
-
-*
-
-
-* @test_Strategy: Create a stub instance to our service
-definition
-
-
-* interface, set the target endpoint to the servlet,
-
-
-* and invoke the RPC methods for each primitive type.
-
-
-* For each primitive type pass its value as input to the
-
-
-* corresponding RPC method and receive it back as the
-
-
-* return value. Compare results of each value/type of
-
-
-* what was sent and what was returned. Verify they are
-
-
-* equal.
-
-
-*/
-
The class.setup_props tag is optional. If -your -tests require properties from the ts.jte file, then you must include -this -tag along with any properties your tests rely on. The values for -the properties specified will be passed to the setup method of your -client -class. If possible, limit the number of test descriptions per -client -source file to under 20. For each test description, the following -tags are required... -
- @testName: The name of this
-test.
-This value should be something meaningful and must contain the
-case-insensitive
-substring, "test". There must also be a method on this class
-which
-matches the testname.
-
- @assertion_ids: One or more assertion IDs
-which are tested by this test. NOTE: Only put those
-assertion
-IDs which are explicitely tested by this test. Do not put any and
-all assertion IDs which happen to be covered in this test, even though
-the purpose of the test is to verify something else. For example,
-most EJB tests create EJBs. There is no need to specify the
-assertion ID for creating an EJB in every EJB test description
-
- @test_Strategy: This is a brief descrioption
-of the approah that was taken to test the specified assertions.
-
- @class.testArgs: Additional arguments (if
-any) to be passed when execing the client class.
-
Each tag name must end with a colon and then at -least -one whitespace prior to the first tag value. For tags that can -possibly -have multiple values, like class.setup_props and assertion_ids, a -semi-colon -must separate each tag value. Tag values may or may not span -multiple -lines. The JavaTest harness will read each test desription and -count -it as a single test found. For API tests, for each test -description, -one to four tests may be counted based on the number of vehicles that -the -test is required to run in. -
-It is recommended that all classes associated -with -a directory of tests live in that test directory (e.g. If a test has a -client which talks to an ejb, then the ejb classes should also appear -in -this test directory). However, in some cases (where many -different -classes in different test directories extend a common base class) this -doesn't make sense. In the case where common utility classes (not -those in com.sun.ts.lib) are shared across multiple technology -directories, -these classes should go under the com.sun.ts.tests.common directory. -
-When developing a set of tests for a given -technology, -you will want to ask yourself the following questions... -
-1. Is this API going to be part of the J2EE Platform?
-
-2. If you answered "yes" to question 1, then is it possible that
-this API will be accessible and therefore testable from multiple J2EE
-containers?
-If so, then you will probably want some (if not all) of your test
-clients
-to extend ServiceEETest. This base class provides for automatic
-distribution
-of your test clients to any of 4 possible generic Vehicle components -
-ejb, servlet, jsp, and appclient. See section ??.
-
-
-
3.1 End-to-End Tests -
-End-to-End Tests (EE) -are tests that start in a client (application client or URL client), -accesses -some server-side component, and then possibly access some other -resource -such as a back-end database. When writing these types of tests, -the -test developer is responsible for writing and building each -component. -All EE test clients must extend the class, -com.sun.ts.lib.harness.EETest. -They must also provide at least one setup, cleanup, and -<testname> method. -Most client classes provide single setup and cleanup methods and many -<testname> -methods, since all tests defined in the client are similar enough to -share -thesetup and cleanup. A main method must also be provided, since -each test client is execed in a separate VM by the Javatest -harness. -A sample ejb test client is shown below... -
- -package -com.sun.ts.tests.samples.ejb.ee.simpleHello; -
- -import -java.io.*; -
-import -com.sun.ts.lib.util.*; -
-/** -
-public -class HelloClient extends EETest -
--//Harness requirements -
--public static void main(String[] args) -
--/* Test setup: */ -
- --/* -
--/* Run test */ -
--// invoke method on the EJB -
--/* cleanup -- none in this case */ -
-The test client's -main -method is called when it is execed in its own VM. From there, the -parent class' run method is called. This method will use -reflection -to call the setup method. If that method does not throw a Fault, -then the <testname> method is called. If setup did throw a -Fault, -then the <testname> method is not called. The cleanup -method is -always called. -
- -The ejb classes for -this -
- - -3.2 API (Service) -Tests -
-The name, "Service -Tests" -dates back to the first days of the J2EE-CTS where several APIS were -defined -to be available and testable within different J2EE containers. -These -tests differed from End-to-End tests which focused on component -lifecycles -and multi-tier scenarios. These APIs could be thought of as -"services" -available to each container, thus the name "Service Tests". With -the advent of "Web Services", we should probably just refer to these -types -of tests as API tests to avoid any confusion. When writing these -types of tests, the test developer is only resonsible for writing the -test -client and for passing this client (and any utility classes to the -Vehicles -task to be built. The Vehicles task will automatically build the -components that this API should be run in as defined by the -DeliverableInterface -implementation. -
-When writing your Test Client, focus on -testing -the API, irrespective of where the client is going to be run -from. -The Client class must do the following... -
--Extend -com.sun.ts.lib.harness.ServiceEEtest -
-
-A sample API test client is presented below ( from
-$TS_HOME)/src/com/sun/ts/tests/samples/jdbc/ee/testConn)
-
package -com.sun.ts.tests.samples.jdbc.ee.testConn; -
-import java.io.*;
-
-import java.util.*;
-
import java.sql.*;
-
-import javax.sql.*;
-
import
-com.sun.ts.lib.util.*;
-
-import
-com.sun.ts.lib.porting.*;
-
-import
-com.sun.ts.lib.harness.*;
-
import -java.rmi.RemoteException; -
-import -com.sun.javatest.Status; -
-public class
-testConnClient
-extends ServiceEETest implements Serializable
-
-{
-
-//
-private static final String testName = "samples.jdbc.ee.testConn";
-
-
-
- //
-Naming specific member variables
-
-
-private
-TSNamingContextInterface jc = null;
-
-//
-Harness requirements
-
-
-private
-transient Connection conn = null;
-
-
-private
-DataSource ds1 = null;
-
-
-private
-Properties props = null;
-
-
-
- //
-Tables to be created
-
-
-private
-String pTableName = null;
-
-
-private
-String fTableName = null;
-
-/*
-Run test in standalone mode */
-
-
-public
-static void main(String[] args)
-
- {
-
-
-testConnClient theTests = new testConnClient();
-
-
-Status s=theTests.run(args, System.out, System.err);
-
-
-s.exit();
-
- }
-
-/*
-
-
-* @class.setup_props: Driver;
-
-
-*
-db1;
-
-
-*
-user1;
-
-
-*
-password1;
-
-
-*
-db2;
-
-
-*
-user2;
-
-
-*
-password2;
-
-
-*
-DriverManager;
-
-
-*
-ptable;
-
-
-*
-ftable;
-
-
-*
-cofSize;
-
-
-*
-cofTypeSize;
-
-
-*
-
-
-* @class.testArgs: -ap tssql.stmt
-
-
-*
-
-
-*/
-
-public
-void setup(String[] args, Properties p) throws Fault
-
- {
-
-
-try
-
-
-{
-
-
-try
-
-
-{
-
-
-props = new Properties(p);
-
-
-pTableName = p.getProperty("ptable");
-
-
-fTableName = p.getProperty("ftable");
-
-TestUtil.logTrace("Getting the initial context...");
-
-
-jc = new TSNamingContext();
-
-
-TestUtil.logTrace("Initial context: " + jc.toString());
-
-TestUtil.logTrace("Looking up the JNDI DataSource names");
-
-
-ds1 = (DataSource)jc.lookup("java:comp/env/jdbc/DB1");
-
-
-TestUtil.logTrace("ds1: " + ds1.toString());
-
-getConnection(); -
-
-}
-
-
-catch(SQLException ex)
-
-
-{
-
-
-logErr("SQL Exception : " + ex.getMessage());
-
-
-}
-
-
-}
-
-
-catch( Exception e )
-
-
-{
-
-
-logErr("Setup Failed!");
-
-
-TestUtil.printStackTrace(e);
-
-
-}
-
- }
-
-/*
-
-
-* @testName:
-testCreateTable
-
-
-* @assertion: To verify this
-JDBC connection can create tables.
-
-
-*
-
-
-* @test_Strategy: Using the Connection object created
-in the setup()
-
-
-*
-method, create a table, insert rows into it, then
-
-
-*
-drop it. This sample will ensure that your environment is
-
-
-*
-suitable to run the JDBC tests.
-
-
-*
-
-
-*/
-
-
-public
-void testCreateTable() throws Fault
-
- {
-
-
-try
-
-
-{
-
-
-createTable(props, conn);
-
-
-}
-
-
-catch (RemoteException re)
-
-
-{
-
-
-TestUtil.logMsg("Remote exception on createTable: " + re.getMessage());
-
-
-throw new Fault("Call to testCreateTable Failed.");
-
-
-}
-
- }
-
-/*
-cleanup */
-
-
-public
-void cleanup() throws Fault
-
- {
-
-
-try
-
-
-{
-
-
-dropTables(conn);
-
-
-logMsg("Dropped tables OK");
-
-
-conn.close();
-
-
-logMsg("Closed connection OK");
-
-
-}
-
-
-catch (Exception e)
-
-
-{
-
-
-logErr("An error occurred while closing the database connection");
-
-
-logErr(e.getMessage());
-
-
-}
-
- }
-
}
-
-
-
-
-
3.3 Signature Tests -
- Signature tests
-should
-be written for each technolgy area (XXX Add more here)
-
-
-
-3.4 Using Existing Porting classes
-
-3.5 Common Apps
-
-3.6 Source File headers (copyrights
-and sccs keywords)
-
-4.0 Creating a Deliverable (TCK)
-
-
This section contains -instructions for creating a deliverable out of the Spider workspace. A -deliverable is generically defined as any bundle of tests that is -released -to some customer (internal or external). An example deliverable -would -be the CTS or the JAXRPC-TCK. I can imagine that we might want to -create -a deliverable for the CTS smoke tests. Every deliverable must -have -the following in the spider workspace... -
-1. An install directory -
-Your Deliverable's install directory -
-Under the -$TS_HOME/install -directory, create a subdirectory which describes your deliverable (e.g -j2ee, jaxrpc, smoke, etc.). This directory will hold -configuration -files for your deliverable. Within the workspace, when setup is run for -this deliverable, the files in this subdir will be copied to top level -directories in the workspace. Any tests that are built or run from that -point (until a different deliverable's setup is run) will do so -according -to this deliverable's configuration and build files. A sample -deliverable's -install directory might look like the following... -
-install -
-Creating Your Deliverable Bundle -
- A createzip ANT
-build.xml
-file should be created under the release directory with a default
-target
-of "createzip".
-
-
-
5.0 Building Tests -
-Creating leaf directory build.xml files -
-Once you've created your test sources, -you -must then create an ANT build.xml file in your test source directory. -Please -see the complete documentation on the custom tasks (link to custom -tasks -here) that exist for the spider workspace. From any point in the src -directory -(excluding under com/sun/ts/lib), you should be able to build using the -ant -command. Please see the sample leaf level build.xml file below... -
-<insert ejb sample here> -
-The build.xml files have become much -simpler -due to default values being set for many attributes. By default, all -components, -application archives, deployment descriptors, and RI runtime.xml files -will be generated or copied into the appropriate package structure -under -$TS_HOME/dist. Your test directory will not be modified. -
--build files (ANT). -
-All packaging is done through ANT custom
-tasks that are called from individual build.xml files. The
-<package>
-task will check the build.level property in the bin/build.properties
-file
-for the deliverable that you are building for. It will use this value
-to
-determine whether to build only components (wars, jars, rars) or
-applications
-(ears) as well. Support for further configuration values may be added
-in
-the future. This property allows for high level control of what gets
-built
-
-
-
-
-
Building tests -
- To build only a certain
-subdirectory of tests, cd into a given test directory under
-$TS_HOME/src and type "ant". This will build all tests under
-this directory. Most deliverables' top level build file
-($TS_HOME/bin/build.xml) contains a target to build all tests for that
-particular deliverable. In the case of the CTS, the target is
-called build.all.tests.
-
-
-
- 6.0 Creating a Porting Interface
-
TBD
-
7.0 -Running Tests -
- Almost everything in
-the
-spider workspace or bundle generated from the workspace is done through
-an ANT target. The only platform specific script and bat files
-are
-those belonging to ANT (ant/ant.bat).
-All ant targets should be accessed through the wrapper script and bat
-files.
-These wrappers provide added functionality through an ANT build
-listener
-to set default values for locations in the workspace.
-
-
-
cd into any test directory under
-$TS_HOME/src/com/sun/ts/tests and type "ant runclient" to run any
-tests in the current directory as well as any sub-directories. To run
-an individual test cd into the test leaf directory and invoke: "ant
--Dtest=<test-name> runclient" where <test-name> is the name
-of the test you wish to invoke.
-
-
-
-
Appendix A – Inside -the -Test Harness -
-We provide an implementation for the
-Javatest Script and TestFinder interfaces...
-
com.sun.ts.lib.harness.TSScript
-com.sun.ts.lib.harness.TSTestFinder
-
The
-Harness Packages
-
com.sun.ts.lib.harness
-
-
-This package includes the core harness classes responsible for...
-
-
--finding and returning test descriptions
-
--running test clients
-
--determining deployable archives
-
--customization of runtime information from configuration files
-
--providing common functionality for test clients
-
-
-com.sun.ts.lib.deliverable
-
-
This package provides the necessary
-interfaces for abstracting deliverable specific functionality from the
-core harness package. Each TCK may require customized
-implementations of these interfaces. Examples of deliverables include
-the CTS and other TCKs. Implementations for some of these are included
-in subpackages.
-
-
-com.sun.ts.lib.porting
-
-
This package contains the factories,
-proxies,and interfaces for all porting packages. It also contains
-a DeploymentInfo class which contains runtime information necessary to
-deploy an app on a J2EE server.
-
-
-com.sun.ts.lib.util
-
-
This package provides general utility
-classes for distributed logging, web test support, naming context
-creation, and standalone deployment
-
-
-com.sun.javatest.SimpleDefaultInterviewparameters
-
-
-This class provides a simple interview for the Javatest harness.
-Default values are determined by the deliverable.
-
-
-com.sun.ts.lib.implementation
-
-
-This package provides Sun reference implementation classes for each of
-the porting interfaces in the porting package.
-
-
-com.sun.ts.lib.tests
-
-
-This package contains some general technology specific utility classes
-such as those used to drive the web clients for jsp, servlet, jstl, etc.
-
Appendix B – Spider -Directory -structure -
-Appendix C – ANT -custom -Tasks -
-Appendix D – FAQ -
-Q. Where are all of the scripts and bat files (ts/ts.bat, -ts_env/ts_env.bat, -etc.)? -
-A. All platform script and bat files were removed and replaced -by the ANT equivalent. All properties from ts_env now are -configurable -in a given deliverable's build.properties file. All other scripts -are now accessible through ANT targets. This allows for -consistency -across different platforms and no reliance on unix toolkits on Win32 -Platforms. -In the case of CTS, see the conversion table below for reference... -
-CTS
-1.3.1
-CTS 1.4
-
-------------------------------------------------------------
-
-ts/ts.bat
-ant rungui
-
-ts_env/ts_env.bat
-$TS_HOME/build.properties
-
-
-where to put tests -
--creating deliv. specific ts.jar -
---what to include in this jar -
--what your TCK looks like in the ws and -in -a bundle -
---sample deliverable example with just -sample -tests?? -
--what goes under -install/<delivname>/ -
--structure under -install/<delivname> -
--subdirs should match top level dir
-where
-files are copied during setup
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-naming conventions for "_component" -jars -and wars to be passed to the deployment porting class -
--link to java coding conventions -
-Where to put your tests -
--directory organization -
--when/(when not) to use tests/common -
---when/(when not) to put code into -ts/lib -
--Developing a test or set of tests -
--needed files -
--rules -
---parent classes -
----EETest -
----ServiceEETest -
---logging -
----do tcks need to implement their own -loggerhandler -under TestUtil? -
---testtags -
----linking to assertion lists -
--web content (where does it go) -
-
-