This executable test suite (ETS) verifies the conformance of the secure client behavior as the implementation under test (IUT) with respect to the following specification(s):
Conformance testing is a kind of “black box” testing that examines the externally visible characteristics or behaviors of the IUT while disregarding any implementation details.
Several conformance classes are defined in the principal specification; the ones listed below are covered by this test suite:
The test suite definition file (testng.xml) is located in the root package, org.opengis.cite.securityclient10. A conformance class corresponds to a <test> element, each of which includes a set of test classes that contain the actual test methods. The general structure of the test suite is shown in Table 1.
Conformance class | Test classes |
---|---|
Conformance Level Common Security | org.opengis.cite.securityclient10.levelCommonSecurity.* |
Conformance Level OWS Common | org.opengis.cite.securityclient10.levelOwsCommon.* |
Conformance Level WMS 1.1.1 | org.opengis.cite.securityclient10.levelWms111.* |
Conformance Level WMS 1.3.0 | org.opengis.cite.securityclient10.levelWms13.* |
The Javadoc documentation provides more detailed information about the test methods that constitute the suite.
The test run arguments are summarized in Table 2. The Obligation descriptor can have the following values: M (mandatory), O (optional), or C (conditional).
Name | Value domain | Obligation | Description |
---|---|---|---|
service_type | String | M |
A string representing the service type to emulate for secure clients. Will affect the query parameters required to get capabilities. Valid values are:
|
address | String | M | Host interface on which to bind test server. When using IPv6, do not use square brackets, e.g. ::1 |
port | Integer | M | Port on which to bind test server. |
host | String | M | Host name that will be advertised to clients. May be an IP address or domain name. Clients must be able to resolve this IP or name to the machine running the test server. If you are using a certificate from a Certificate Authority, this parameter must match the common name on that certificate. When using IPv6, use square brackets around the address, e.g. [::1] |
path | String | O |
URL Path at which the Test Server will listen. For example, test-session would result in the Test Server creating a servlet for https://host:port/test-session. If left blank, a random string will be generated. When used with TEAM Engine, the CTL script will automatically fill in a path such that the test session web page can inform the tester of the URL before starting the test session. |
jks_path | String | M | A Java KeyStore containing the X.509 certificates for the host address must be located at jks_path in order for the embedded Jetty server to provide HTTPS. Self-signed certificates are permitted, although the test client will have to trust that certificate manually. |
jks_password | String | M | The password required to unlock the Java KeyStore. Using a CDATA section is recommended to wrap passwords that have character data that may interfere with XML character entities. |
authentication | String | O |
Specify the authentication method that will be presented to Secure Clients via the capabilities document. Only one authentication method may be specified. Values are not case sensitive. Enabling this will include an ows:Constraint for requiring an authentication from the secure client. For some authentication methods, additional test run properties may be mandatory. If this element is omitted or is empty, then authentication will not be enabled. For SAML2 authentication, specify saml2 and specify the idp_url test run property: <entry name="authentication">saml2</entry> |
idp_url | String | O |
As part of the annotated capabilities document presented to the secure client, include Requirements Class "SAML2" (https://www.opengis.net/def/security/1.0/rc/authentication/saml2). Enabling this will include an ows:Constraint for requiring SAML2 authentication from the secure client. The authentication test run property must be set to saml2, otherwise this test run property is ignored. The value must be a URL that resolves to a SAML2 SSO resource. This value will then be passed to the secure client in the ows:Constraint. Note that it is up to the test user to validate the URL and the service at that URL, the ETS will do no verification and pass the URL "as-is" to the secure client in the capabilities document. If this element is omitted or is empty, then SAML2 authentication will not be enabled. |
http_methods | String | O |
As part of the annotated capabilities document presented to the secure client, include Requirements Class "HTTP Methods" (https://www.opengis.net/def/security/1.0/rc/http-methods). Enabling this will include an ows:Constraint for listing all supported HTTP methods. Only a value of true will enable this property, any other value will be evaluated as false. If the w3c_cors property is set to true, then this property will be overridden to true as well. |
w3c_cors | String | O |
As part of the annotated capabilities document presented to the secure client, include Requirements Class "W3C CORS" (https://www.opengis.net/def/security/1.0/rc/cors). Enabling this will include an ows:Constraint for the W3C recommendation"Cross Origin Resource Sharing". Only a value of true will enable this property, any other value will be evaluated as false. If set to true, then the http_methods test run property will also be set to true regardless of your configuration; "HTTP Methods" is required for this Requirements Class. |
http_exception_handling | String | O |
As part of the annotated capabilities document presented to the secure client, include Requirements Class "HTTP Exception Handling" (https://www.opengis.net/def/security/1.0/rc/http-exception-handling). Enabling this will include an ows:Constraint for enabling HTTP error code mapping to OWS Common exception codes. Only a value of true will enable this property, any other value will be evaluated as false. |
http_post_content_type | String | O |
As part of the annotated capabilities document presented to the secure client, include Requirements Class "HTTP POST Content-Type" (https://www.opengis.net/def/security/1.0/rc/content-type). Enabling this will include an ows:Constraint for listing the mime-types permitted to be submitted by HTTP POST. At the minimum, the `application/x-www-form-urlencoded` MIME type must (and will) be supported and advertised by the service. Only a value of true will enable this property, any other value will be evaluated as false. |
Additional test run properties may be added in the future to support other authentication methods such as SAML 2.0 and OpenID Connect.
The test suite is built using Apache Maven v3. Maven is necessary for self-hosting the test suite in your own test environment.
The test suite can be ran in the following environments:
Use a Java IDE such as Eclipse, NetBeans, or IntelliJ. Clone the repository and build the project.
Set the main class to run: org.opengis.cite.security-client10.TestNGController
Arguments: The first argument must refer to an XML properties file containing the required test run arguments. If not specified, the default location at ${user.home}/test-run-props.xml will be used.
The TestNG results file (testng-results.xml) will be written to a subdirectory in ${user.home}/testng/ having a UUID value as its name.
One of the build artifacts is an “all-in-one” JAR file that includes the test suite and all of its dependencies; this makes it very easy to execute the test suite in a command shell:
java -jar ets-security-client10-0.1-SNAPSHOT-aio.jar [-o|--outputDir $TMPDIR] [test-run-props.xml]
This will require you to first compile and build the test suite as a Java application.
Use TEAM Engine, the official OGC test harness. The latest test suite release are usually available at the beta testing facility. You can also build and deploy the test harness yourself and use a local installation.
For more details, please see the guide on setting up TEAM Engine with this ETS.
If you need to debug a secure client connection and inspect the HTTP or HTTPS details, add these to your Java VM arguments:
-DDEBUG=true -Dorg.eclipse.jetty.LEVEL=DEBUG -Djavax.net.debug=ssl,handshake,data
This will print very detailed information about cipher suites and any TLS extensions available from the client.
This repository contains a sample Java KeyStore with a self-signed certificate for testing purposes. The keystore can be created with the following command.
$ keytool -keystore src/main/resources/security.jks -storepass "ets-security-client" -genkey -alias dummy-key -keyalg RSA -sigalg SHA256withRSA -dname "cn=ETS Test Operator, ou=None, o=None, c=us"
This creates src/main/resources/security.jks with a single key (dummy-key) and protects the file with a password (ets-security-client). As this is a self-signed certificate, secure clients must allow insecure server certificates or install the certificate to their keystore.
When running the test suite from an IDE, this KeyStore will be used as it is specified in src/main/config/test-run-props.xml.
To create a PEM version of the file for secure clients to use:
$ keytool -exportcert -alias dummy-key -keystore src/main/resources/security.jks -storepass "ets-security-client" -rfc -file src/main/resources/security.pem
If you would like to get involved, you can: