gentest.txt

INTRODUCTION
============

GENTEST is a program to generate test vectors for the AES combined
encryption and authentication modes CCM, GCM and EAX. It is also
used to generate test vectors for XTS, EME-2, OMAC and CMAC. 

Historically Test Vectors for AES modes are hard to find and are
very limited in scope.  As a result developers needing test vectors
have had to use existing software implementations to generate them
with the risk that they would be wrong as a result of incorrect
implementation by the program's author. The author of this package
offers AES mode implementations that have been used for test vector
generation but which have subsequently been found to be incorrect.
Moreover, this is also true for other implementations so it is hard
to develop confidence in the vectors that are being used.

GENTEST is an attempt to overcome this problem.  GENTEST combines
two indpendent implementations of CCM, GCM and EAX, one set by the
author (Brian Gladman) and the other derived from the source code
for these modes in LibTomCrypt.  The latter has been changed in
form but care has been taken to preseve the logical structure of
then original code in order to ensure that this offers a truly
independent second implementation.  They both use the author's own
AES code but this is very widely used for many years without any
known instances in which incorrect results have been produced so
it is extremely unlikely to cause any test vector errors.

These two sets of mode implementations have been tested against each
other for a wide range of random inputs for many hours without any
disagreements about the resulting output values.  Also, in the test
vector generation to be described, each vector is generated by the
two different mode implementations (BRG and LTC) for the mode in
question and is only generated if the results are in complete
agreement.

ALthough this still doesn't guarantee correctness, it is far more
likely to produce accurate test vectors than the 'hit or miss' use
of existing modes software implementations used alone.

For XTS and EME-2 only the authors implementations are used so the
correctness is less assured. 

USING GENTEST
=============

All the supplied files need to be compiled into a single executable
file GENTEST.EXE.  THis is then used with the command line:

  GENTEST input_file output_file

where 'input_file' is the name of a driver text file (to be described
below) and 'output_file' is the name of the file to which the resulting
test vectors will be written.  In the examples provided the input file
names are of the form *.n.txt and provide output files named *.n (where
n is an integer). 

THE DRIVER FILE
===============

Here is an example of the text in a driver file (the '----' lines
are not part of the file):

---------------
MODE GCM
VEC 0001

KEY 00000000000000000000000000000000
NCE 000102030405060708090a0b
HDR 0010203040
PTX ffffffffffffffffffffff
GEN
---------------

The first non-empty line starts with the word MODE followed by a
space and a three or four character mode designator, for example, 
'CCM', 'GCM', 'EAX', 'XTS' or 'EME2'. Lines prior to this line 
are ignored so they can contain descriptions of the file.

All the line designators are four characters long and all except 'MODE'
are three characters plus a space, which must be present.

The line designators have the following meanings for CCM, GCM and EAX:

MODE - the mode to which the file refers
VEC  - the starting value for test vector numbering
KEY  - the key value
NCE  - the IV or nonce value
HDR  - the authenticated header data
PTX  - the plaintext
TGL  - the length of the authentication tag (if not 16 bytes)
CTX  - the ciphertext (in the output file only)
TAG  - the authentication tag (in the output file only)
REM  - a comment that will be embedded in the output file
GEN  - the instruction to proceed with test vector generation
END  - the end of a sequence of test vectors (in the output file only)

for XTS (if the bit length is not a multiple of eight, the extra
bits of the sector data are stored in the last byte of this data
with bit 0..7 in the most..least signiicant positions respectively:

LEN  - the length of the encrypted data in BITS
EKY  - the encryption key
TKY  - the tweak key
LBA  - the logical block address
PTX  - the sector data
CTX  - the encrypted sector data

and for EME2:

KEY  - the key
ADT  - the associated data
PTX  - the plaintext
CTX  - the ciphertext (in the output file only)

If a line is not present the resulting value is set to empty (zero length).
However the key value must be present. Each of the line designators KEY,
NCE, HDR and PTX is followed by a sequence of an even number of hexadecimal
digits, each pair setting the value of a byte in the array of bytes for the
item in question. The array index starts at zero and increases by 1 for 
each pair of hexadecimal digits on the line. The values for consecutive 
lines with the same designators are concatenated. Hence:

KEY 00000000000000000000000000000000

KEY 0000000000000000
KEY 0000000000000000

give the same value.

Arrays of bytes can also be generated from decimal numbers up to 64-bits in
length.  Decimal numbers must be preceeded by a '#' and are normally placed
into arrays of bytes in little endian order. However the inputs:

KEY #<1024
KEY #>1024

will assemble a decimal value in little and big endian order respectively.

Bracket Terms
=============

Bracket terms provide a way of inputing long items and random values easily.
For example the line

KEY (10)

sets the key to a 16 byte random number (16 because all input values are two
*hexadecimal* digits) .  In general the line

KEY (hh)

will set the value of the designated item to a random array of bytes of length
'hh in hexadecimal. Although all input is normally in hexadecimal, numbers within
bracketed terms can be expressed in decimnal by preceeding them with a '#' symbol
(but not with any '<' or '>' suffix in this case).

Hence:

KEY (#16)

has the same effect as above.  Here are all the possible bracket terms and their
actions (where DES stands for any line designator)

1. Setting single test vectors:

DES (nn)             -- set a random value of length 'nn' bytes.
DES (nn,hh)          -- set a value of length 'nn' bytes, each of value 'hh'.
DES (nn,hh,hi)       -- set a value of length 'nn' bytes, the first byte of
                        value 'hh' and incrementing each subsequent byte by
                        'hi'.

2. Setting sequences of test vectors of varying value

DES (no,v0,vi,1)     -- set a sequence of 'no' test vectors starting with the 
                        value v0 and incrementing by vi for each subsequent 
                        vector and mapping the values to bytes in little 
                        endian format.
                        
DES (no,v0,vi,2)     -- set a sequence of 'no' test vectors starting with the 
                        value v0 and incremnting by vi for each subsequent 
                        vector and mapping the values to bytes in big 
                        endian format.
                        
3. Setting sequences of test vectors of varying length:
                   
DES (no,lo,li,3)     -- set a sequence of 'no' test vectors starting with a 
                        length of 'lo' bytes and increasing the length by 'li'
                        bytes for each subsequent test vector. The content is
                        random.
                        
DES (no,lo,hi,4)     -- set a sequence of 'no' test vectors of random length
                        between lo and hi inclusive and with random content.

DES (no,lo,li,hh,hi) -- set a sequence of 'no' test vectors starting with a 
                        length of 'lo' bytes and increasing the length by 'li'
                        bytes for each subsequent test vector. The content is
                        byte values starting at hh and incrementing by hi for
                        each subsequent byte in the test vector
                        
For example:

    PTX (#8,#3,#1,00,10)

generates the sequence:

    PTX 001020
    PTX 00102030
    PTX 0010203040
    ...
    PTX 00102030405060708090

and

    PTX (#17,#0,#1,ff,ff)

generates
    
    PTX
    PTX ff
    PTX fffe
    PTX fffefd
    ...
    PTX fffefdfcfbfaf9f8f7f6f5f4f3f2f1f0

It is important to remember that if one field genrates M values and
another generates N values then M * N test vectors will be produced.
For example, the specification:

KEY 00000000000000000000000000000000
NCE (00,0c)
HDR (00),(01)
PTX (00,01,10,00,01)
GEN

will generate 34 test vectors, 17 with no header:

VEC 1
KEY 00000000000000000000000000000000
TAG 66e94bd4ef8a2c3b884cfa59ca342b2e

VEC 2
KEY 00000000000000000000000000000000
PTX 00
CTX 58
TAG 4c60cd1b5f259605d5acdfc3c6c1fa4e

VEC 3
KEY 00000000000000000000000000000000
PTX 0001
CTX 58e3
TAG e866e713c68111999c3e246952f2cca5

.......

VEC 16
KEY 00000000000000000000000000000000
PTX 000102030405060708090a0b0c0d0e
CTX 58e3fecdfe7b36663e76175ca8ea4b
TAG 98fe202699231d9eab6e07da4fc40bd6

VEC 17
KEY 00000000000000000000000000000000
PTX 000102030405060708090a0b0c0d0e0f
CTX 58e3fecdfe7b36663e76175ca8ea4b55
TAG 77a132c2a8967ece63818621e96dd5d5

and then 17 with a random 1 byte header:

VEC 18
KEY 00000000000000000000000000000000
HDR d6
TAG 394dca7fa7f7d8b95b839d986c043437

VEC 19
KEY 00000000000000000000000000000000
HDR d6
PTX 00
CTX 58
TAG 603f64064fb1763d2948457c40e7c829

.......

VEC 34
KEY 00000000000000000000000000000000
HDR d6
PTX 000102030405060708090a0b0c0d0e0f
CTX 58e3fecdfe7b36663e76175ca8ea4b55
TAG 5bfe9bdfb8029ef69f651c9e6f4be7b2

END

For plaintext only the designator can be followed by an '=' sign 
as follows:

PTX =

which uses the last ciphertext output as the new plaintext.  This 
allows Monte Carlo style tests.


Since each of the inputs is composed with all the others in this way, 
a very small specification can generate a huge number of test vectors. 
Hence a lot of careful thought is necessary to generate sensible test
vector sequences.

It is also important to note that the program does not know about
any contraints on inputs imposed by the various modes. It is hence
the user's responsibility to ensure that only legal input values are
generated.

ACKNOWLEDGEMENTS
================

I have used  code derived from Tom St Denis's LibTomCrypt library
to develop GENTEST. His contribution is acknowledged.

I also want to thank Colin Sinclair, Matt Ball, Danh Tran and
Michael Torla, all of whom spent time tracking down elusive
'edge case' faults in my AES modes implementations.  Without
their efforts this program would not have been possible.

    Brian Gladman, Worcester, UK,
    24th January 2014