sourCEntral - mobile manpages

pdf

STXXL_TOOL

NAME

stxxl_tool - STXXL scratch file administration, and performance testing tool

SYNOPSIS

stxxl_tool <subtool> ...

DESCRIPTION

The stxxl_tool is an umbrella program that houses several sub-tools for STXXL relating to (e.g.) scratch file manipulation and benchmarking. The behaviour of each sub-tool is given below.

Size arguments will accept a string like "343KB" or "44 GiB" as their input.

Subtools

info

Print out information about the build system and which optional modules where compiled into STXXL.

create_files <filesize> <filename ...>

Pre-create large files to keep file system allocation time out to measurements.

benchmark_disks [options] <size> [r|w] [alloc]

Benchmark the disks configured by the standard .stxxl disk configuration files mechanism. Blocks of 8 MiB are written and/or read in sequence using the block manager. The batch size describes how many blocks are written/read in one batch. The are taken from block_manager using given the specified allocation strategy. If size == 0, then writing/reading operation are done until an error occurs.

Parameters:
size Amount of data to write/read from disks (e.g. 10GiB)
r|w Only read or write blocks (default: both write and read)
alloc Block allocation strategy: RC, SR, FR, striping. (default: RC) Options:
-b, --batch Number of blocks written/read in one batch (default: D * 8MiB)

benchmark_files [options] <length> <filename ...>

Open a file using one of STXXL’s file abstractions and perform write/read/verify tests on the file. Block sizes and batch size can be adjusted via command line. If length == 0 , then operation will continue till end of space (please ignore the write error). Memory consumption: block_size * batch_size * num_files

Parameters:
length Length to write in file.
filename File path to run benchmark on. Options:
-o, --offset Starting offset to write in file.
--no-direct open files without O_DIRECT
--sync open files with O_SYNC|O_DSYNC|O_RSYNC
--resize resize the file size after opening, needed e.g. for
creating mmap files
--block_size block size for operations (default 8 MiB)
--batch_size increase (default 1) to submit several I/Os at once and
report average rate
-f, --file-type Method to open file (syscall|mmap|wincall|boostfd|...)
default: syscall
-p, --operations [w]rite pattern, [r]ead without verification, read and
[v]erify pattern (default: ’wv’)
--pattern 32-bit pattern to write (default: block index)

Benchmark different file access methods, e.g. syscall or mmap_files.

benchmark_sort [options] <size>

Run benchmark tests of different sorting methods in STXXL. size is the amount of data to sort, in GiB

Valid options are:
-M, --ram : amount of ram to use when sorting, default 256 MiB

benchmark_disks_random <span> [block_size] [size] [i|r|w] [alloc]

Benchmark _random_ block access on the disks configured by the standard .stxxl disk configuration files mechanism. Available block sizes are power of two from 4 KiB to 128 MiB. A set of three operations can be performed: sequential initialization, random reading and random writing.

Parameters:
span Span of external memory to write/read to (e.g. 10GiB).
block_size Size of blocks to randomly write/read (default: 8MiB).
size Amount of data to operate on (e.g. 2GiB), default: whole span.
i|r|w Operations: [i]nitialize, [r]ead, and/or [w]rite (default: all).
alloc Block allocation strategy: RC, SR, FR, striping (default: RC).

benchmark_pqueue [options] [size]

Benchmark the priority queue implementation using a sequence of operations. The PQ contains pairs of 32- or 64-bit integers, or a 24 byte struct. The operation sequence is either a simple fill/delete cycle or fill/intermixed inserts/deletes. Because the memory parameters of the PQ must be set a compile-time, the benchmark provides only three PQ sizes: for 256 MiB, 1 GiB and 8 GiB of RAM, with the maximum number of items set accordingly.

Parameters:
size Amount of data to insert (e.g. 1GiB) Options:
-t, --type Value type of tested priority queue:
1 = pair of uint32,
2 = pair of uint64 (default),
3 = 24 byte struct
0 = all of the above
-p, --pq Priority queue configuration to test:
1 = small (256 MiB RAM, 4 GiB elements)
2 = medium (1 GiB RAM, 16 GiB elements) (default)
3 = big (8 GiB RAM, 64 GiB elements)
0 = all of the above
-o, --opseq Operation sequence to perform:
1 = insert all, delete all (default)
2 = insert all, intermixed insert/delete
0 = all of the above

mlock <size>

Allocate some memory and mlock() it to consume physical memory. Needs to run as root to block more than 64 KiB in default settings.

mallinfo

Show mallinfo statistics.

AUTHOR

stxxl is written by many authors, and can be found in the AUTHORS section of the documentation package, or on the homepage

HOMEPAGE

libstxxl Project Home : http://stxxl.sourceforge.net

This manual page was written by D Haley <mycae gmx com>, for the Debian project (and may be used by others).

pdf