/search.css" rel="stylesheet" type="text/css"/> /search.js">
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Tests and Benchmarks

The tests/ directory contains tests and timing harnesses for the components of the Random123 library.

Compiling and Running the code

Installing and using Random123 requires only the use of the header files, and has no prerequisites other than a reasonable C99 or C++98 compiler.

With a modern GNU make (3.80 or newer), building and running the core tests and examples can be as easy as running gmake with no arguments. Note, though, that the provided tests/GNUmakefile intentionally avoids setting any of the standard make variables: CC, CXX, CPPFLAGS, CFLAGS, CXXFLAGS, TARGET_ARCH, LDFLAGS, LOADLIBES, LDLIBS. GNU make will inherit settings for these variables from the environment, or they may be set on the command line. If none are set, compilation will proceed using system-wide default flags, generally without advanced optimization, architectural tuning, warnings, or other common options.

Before putting the Random123 library to use in an application, it is important to test it using the same compiler flags and features that the application will use. In other words, the conventional make variables should be set the same way when testing the library as they will be set when the library is actually compiled into your application. Something like:

gmake CFLAGS="-std=c99" CXXFLAGS="-std=c++0x" CPPFLAGS="/alternate/location/include -O3 -Wall -Wstrict-aliasing=2" TARGET_ARCH="-march=native"

would confirm that all is well with optimization on, and output targeted at an architecture with the same capabilities as the machine running the compilation.

Very old versions of GNU make (pre-2002) or non-GNU make will not work with tests/GNUmakefile.. Lacking a suitably modern GNU make, our advice is to invoke the C or C++ compiler directly on the source files in the tests/ directory. The file tests/BUILD.LOG contains a list of sample build commands. They will almost certainly need to be adapted to the target system. For Windows users, BUILDVC.BAT invokes the Microsoft Visual Studio compiler. Edit it as needed for your platform.

Tests

It is recommended that Random123 be tested on the target system, with the target compiler, intended optimization levels, options, target architectures, etc. before relying it. The library uses architecture- and compiler-specific intrinsics, features and assembly language. We have seen cases where one compiler (open64 version 4.2.4) masquerades as another compiler (it defines GNUC) accepts extensions specific to the other compiler (__uint128_t) without error or warning, and then silently produces incorrect code. The only way to guard against this kind of misbehavior is to compile and run the tests with the compiler and options that you intend to use and the platform that you intend to run on.

Known Answer Tests

Testing that your compiled code computes the same "Known Answers" as the reference implementation which has been subjected to the Crush batteries of statistical tests is critically important.

The file tests/kat_vectors contains a few dozen "Known Answer Test" vectors, i.e., tuples of (method, counter, key, answer). The source file katc.c is incorporated into kat_c.c (C), kat_cpp.cpp (C++), kat_cuda.cu (CUDA) and kat_opencl.c (OpenCL), which are compiled into kat_c, kat_cpp, kat_cuda and kat_opencl, respectively. Each of these will read kat_vectors and verify that the compiled code obtains the same "known answers".

The kat vectors are not language-specific. Implementations of CBRNGs in other languages could also be validated against kat_vectors. The kat vectors are also byte-order independent. In other words, the CBRNGs in the library should produce the same numerical results on little-endian and big-endian hardware, but this behavior is largely untested.

Unit Tests

tests/ also contains tests of specific components of the library. While not exhaustive, these tests verify that a variety of invariants are satisfied by the public methods (e.g., that incr(N) is the same as incr() N times). They also serve to verify some of the compile-time feature-test logic which, if incorrect can lead to mysterious errors (e.g., is it necessary to #include <smmintrin.h>). Unit tests include:

Measuring performance

We include some timing harnesses that can be used to measure the performance of these CBRNGs on various platforms. These timing harnesses report a cycles-per-byte (cpB) metric, which should be independent of clock-rate or number of cores, but depends on compilers and the architecture of the processor being run on. They also report aggregate throughput in GB/sec: a more direct measure of performance, but one that depends on clock speed and number of cores being used. The timing harnesses are obscured by tricks required for portability across platforms and CBRNG type. As a result, they are not recommended as examples of the use of library and its APIs.

time_serial, time_thread, time_cuda, time_opencl all use a common kernel defined in time_random123.h. They all use various util_* header files for utility functions and platform-related boilerplate (also used by the pi_* examples).