Merge pull request #76 from hpc/freeing-sync-lists

Freeing sync lists

doc/man/man1/dbcast.1

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 1.19.1
+.\" Automatically generated by Pandoc 1.19.2
 .\"
 .TH "DBCAST" "1" "" "" ""
 .hy
@@ -7,40 +7,52 @@
 dbcast \- distributed broadcast
 .SH SYNOPSIS
 .PP
-\f[B]dbcast CHUNK_SIZE SRC_FILE DEST_FILE\f[]
+\f[B]dbcast [OPTION] SRC DEST\f[]
 .SH DESCRIPTION
 .PP
-Parallel MPI application to recursively broadcast a file from a global
-file system to node\-local storage.
+Parallel MPI application to recursively broadcast a single file from a
+global file system to node\-local storage, like ramdisk or an SSD.
 .PP
-dbcast helps one copy a source file to node\-local storage on each
-compute node.
-.PP
-The file is logically sliced at chunk boundaries as specified by
-CHUNK_SIZE and collectively copied from a global file system to
+The file is logically sliced into chunks and collectively copied from a
+global file system to node\-local storage.
+The source file SRC must be readable by all MPI processes.
+The destination file DEST should be the full path of the file in
 node\-local storage.
-The CHUNK_SIZE parameter should be specified in bytes, and one may use
-units like "MB" or "GB".
-It is recommended to use the stripe size of a file if this is known.
-The path to the source file should be given in SRC_FILE.
-The source file should be in a path readable by all MPI processes.
-The destination path is given in DEST_FILE.
-The destination file should be a path to a node\-local storage location,
-like ramdisk or an SSD.
-.PP
-In its current implementation, the tool requires at least two MPI
-processes per compute node, and all compute nodes should have the same
+If needed, parent directories for the destination file will be created
+as part of the broadcast.
+.PP
+In the current implementation, dbcast requires at least two MPI
+processes per compute node, and all compute nodes must run an equal
 number of MPI processes.
 .SH OPTIONS
+.TP
+.B \-s, \-\-size SIZE
+The chunk size in bytes used to segment files during the broadcast.
+Units like "MB" and "GB" should be immediately follow the number without
+spaces (ex.
+2MB).
+The default size is 1MB.
+It is recommended to use the stripe size of a file if this is known.
+.RS
+.RE
+.TP
+.B \-h, \-\-help
+Print the command usage, and the list of options available.
+.RS
+.RE
 .SH EXAMPLES
 .IP "1." 3
-To broadcast a file to /ssd on each node slicing at 1MB chunks:
+To broadcast a file to /ssd on each node:
 .PP
-mpirun \-np 128 dbcast 1MB /global/path/to/filenane /ssd/filename
+mpirun \-np 128 dbcast /global/path/to/filenane /ssd/filename
 .IP "2." 3
+Same thing, but slicing at 10MB chunks:
+.PP
+mpirun \-np 128 dbcast \-s 10MB /global/path/to/filenane /ssd/filename
+.IP "3." 3
 To read the current striping parameters of a file on Lustre:
 .PP
-lfs getstripe /path/to/file
+lfs getstripe /global/path/to/filename
 .SH SEE ALSO
 .PP
 \f[C]dbcast\f[] (1).

doc/man/man1/dchmod.1

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 1.19.1
+.\" Automatically generated by Pandoc 1.19.2
 .\"
 .TH "DCHMOD" "1" "" "" ""
 .hy
@@ -10,7 +10,7 @@ dchmod \- distributed tool to set permissions and group
 \f[B]dchmod [OPTION] PATH...\f[]
 .SH DESCRIPTION
 .PP
-Parallel MPI application to recurseively change permissions and/or group
+Parallel MPI application to recursively change permissions and/or group
 from a top level directory.
 .PP
 dchmod provides functionality similar to \f[I]chmod(1)\f[] and
@@ -31,19 +31,23 @@ Change group to specified GROUP name.
 .RE
 .TP
 .B \-m, \-\-mode MODE
-The mode that you want the file or directory to be.
+The mode to apply to each item.
 MODE may be octal or symbolic syntax similar to chmod(1).
 In symbolic notation, "ugoa" are supported as are "rwxX".
+As with chmod, if no leading letter "ugoa" is provided, mode bits are
+combined with umask to determine the actual mode.
 .RS
 .RE
 .TP
 .B \-\-exclude REGEX
-Do not modify items whose full path matches REGEX (regexec).
+Do not modify items whose full path matches REGEX, processed by
+regexec(3).
 .RS
 .RE
 .TP
 .B \-\-match REGEX
-Only modify items whose full path matches REGEX (regexec).
+Only modify items whose full path matches REGEX, processed by
+regexec(3).
 .RS
 .RE
 .TP
@@ -55,10 +59,9 @@ full path.
 .TP
 .B \-v, \-\-verbose
 Run in verbose mode.
-Prints a list of statistics/timing data for the command.
-How many files walked, how many levels there are in the tree, and how
-many files the command operated on.
-This option also prints the files/sec for each of those.
+Prints a list of statistics including the number of files walked, the
+number of levels there are in the directory tree, and the number of
+files the command operated on, and the files/sec rate for each of those.
 .RS
 .RE
 .TP
@@ -72,13 +75,12 @@ Use octal mode to change permissions:
 .PP
 mpirun \-np 128 dchmod \-\-mode 755 /directory
 .IP "2." 3
-Use symbolic mode to change permissions (you can list modes for user,
-group, etc if you separate them with a comma):
+Set group and mode in a single command using symbolic mode:
 .PP
-mpirun \-np 129 dchmod \-\-group mygroup \-\-mode u+r,g+rw /directory
+mpirun \-np 128 dchmod \-\-group mygroup \-\-mode u+r,g+rw /directory
 .IP "3." 3
-Change permissions to u+rw on all files/directories EXCEPT those whose
-name match regex:
+Change permissions to u+rw on all items EXCEPT those whose name match
+regex:
 .PP
 mpirun \-np 128 dchmod \-\-name \-\-exclude \[aq]afilename\[aq] \-\-mode
 u+rw /directory

doc/man/man1/dcmp.1

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 1.19.1
+.\" Automatically generated by Pandoc 1.19.2
 .\"
 .TH "DCMP" "1" "" "" ""
 .hy
@@ -7,28 +7,22 @@
 dcmp \- distributed compare
 .SH SYNOPSIS
 .PP
-\f[B]dcmp [OPTION] SOURCE DEST \f[]
+\f[B]dcmp [OPTION] SRC DEST \f[]
 .SH DESCRIPTION
 .PP
-Parallel MPI application to compare two files or to recurseively compare
-files with same prefix path within two different directories.
-This tool is similar to \f[I]cmp(1)\f[].
-It reports how many files in two different directories are the same (or
-different).
+Parallel MPI application to compare two files or to recursively compare
+files with same relative paths within two different directories.
 .PP
 dcmp provides functionality similar to a recursive \f[I]cmp(1)\f[].
+It reports how many files in two different directories are the same (or
+different).
 .SH OPTIONS
 .TP
 .B \-b, \-\-base
 Do a base comparison.
 .RS
 .RE
 .TP
-.B \-o, \-\-output <FIELD0=STATE0@FIELD1=STATE1,FIELD2=STATE2:FILE>
-write output fields and states to a file.
-.RS
-.RE
-.TP
 .B \-v, \-\-verbose
 Run in verbose mode.
 Prints a list of statistics/timing data for the command.

doc/man/man1/dcp.1

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 1.19.1
+.\" Automatically generated by Pandoc 1.19.2
 .\"
 .TH "DCP" "1" "" "" ""
 .hy
@@ -7,7 +7,7 @@
 dcp \- distributed copy
 .SH SYNOPSIS
 .PP
-\f[B]dcp [OPTION] SOURCE DEST\f[]
+\f[B]dcp [OPTION] SRC DEST\f[]
 .SH DESCRIPTION
 .PP
 Parallel MPI application to recursively copy files and directories.
@@ -51,11 +51,11 @@ Print a brief message listing the \f[I]dcp(1)\f[] options and usage.
 .SH RESTRICTIONS
 .PP
 If a long\-running copy is interrupted, one should delete the partial
-copy and rerun dcp from the beginning.
+copy and run dcp again from the beginning.
 One may use drm to quickly remove a partial copy of a large directory
 tree.
 .PP
-To ensure the copy was successful, one should run dcmp after dcp
+To ensure the copy is successful, one should run dcmp after dcp
 completes to verify the copy, especially if dcp was not run with the \-s
 option.
 .SH EXAMPLES
@@ -76,9 +76,7 @@ mpirun \-np 128 dcp \-p /source/dir1/ /dest/dir2
 Using the \-S option for sparse files does not work yet at LLNL.
 If you try to use it then dcp will default to a normal copy.
 .PP
-The \-g option is still being developed.
-.PP
-The maximum supported filename length for any file transfered is
+The maximum supported file name length for any file transferred is
 approximately 4068 characters.
 This may be less than the number of characters that your operating
 system supports.

doc/man/man1/dfilemaker.1

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 1.19.1
+.\" Automatically generated by Pandoc 1.19.2
 .\"
 .TH "DFILEMAKER" "1" "" "" ""
 .hy
@@ -19,7 +19,7 @@ they actually exist? Unless there is another version of the code
 somewhere?\f[]
 .TP
 .B \-d, \-\-depth=\f[I]min\f[]\-\f[I]max\f[]
-Specify the depth of the filesystem tree to generate.
+Specify the depth of the file system tree to generate.
 The depth will be selected at random within the bounds of min and max.
 The default depth is set to 10 min, 20 max.
 .RS
@@ -60,7 +60,7 @@ The default file size is set from 1MB to 5MB.
 .RE
 .TP
 .B \-w, \-\-width=\f[I]min\f[]\-\f[I]max\f[]
-Specify the width of the filesystem tree to generate.
+Specify the width of the file system tree to generate.
 The width will be selected at random within the bounds of min and max.
 The width of the tree is determined by counting directories.
 The default width is set to 10 min, 20 max.
@@ -88,4 +88,4 @@ Print version information and exit.
 \f[C]dwalk\f[] (1).
 .PP
 The mpiFileUtils source code and all documentation may be downloaded
-from <http://fileutils.io>
+from <https://github.com/hpc/mpifileutils>

doc/man/man1/dfind.1

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 1.19.1
+.\" Automatically generated by Pandoc 1.19.2
 .\"
 .TH "DFIND" "1" "" "" ""
 .hy

doc/man/man1/dgrep.1

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 1.19.1
+.\" Automatically generated by Pandoc 1.19.2
 .\"
 .TH "DGREP" "1" "" "" ""
 .hy

doc/man/man1/dparallel.1

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 1.19.1
+.\" Automatically generated by Pandoc 1.19.2
 .\"
 .TH "DPARALLEL" "1" "" "" ""
 .hy
@@ -35,4 +35,4 @@ Print version information and exit.
 \f[C]dwalk\f[] (1).
 .PP
 The mpiFileUtils source code and all documentation may be downloaded
-from <http://fileutils.io>
+from <https://github.com/hpc/mpifileutils>

doc/man/man1/drm.1

@@ -1,4 +1,4 @@
-.\" Automatically generated by Pandoc 1.19.1
+.\" Automatically generated by Pandoc 1.19.2
 .\"
 .TH "DRM" "1" "" "" ""
 .hy
@@ -13,12 +13,8 @@ drm \- distributed remove
 Parallel MPI application to recursively delete a directory and its
 contents.
 .PP
-drm is a tool for removing files recurseivly in parallel.
-drm is similar to \f[I]rm(1)\f[], but unlike \f[I]rm(1)\f[] there is no
-need to pass a recursive option as files will be removed from the top
-level directory to the bottom of the file tree by default.
-.PP
-Be careful: it behaves like \f[I]rm \-rf\f[], but it is much faster.
+drm is a tool for removing files recursively in parallel.
+Be careful: drm behaves like \f[I]rm \-rf\f[], but it is much faster.
 .SH OPTIONS
 .TP
 .B \-i, \-\-input FILE
@@ -36,14 +32,15 @@ Walk file system without stat.
 .IP
 .nf
 \f[C]
-Do\ not\ remove\ items\ whose\ full\ path\ matches\ REGEX\ (regexec).
+Do\ not\ remove\ items\ whose\ full\ path\ matches\ REGEX,\ processed\ by\ regexec(3).
 \f[]
 .fi
 .RS
 .RE
 .TP
 .B \-\-match REGEX
-Only remove items whose full path matches REGEX (regexec).
+Only remove items whose full path matches REGEX, processed by
+regexec(3).
 .RS
 .RE
 .TP
@@ -54,9 +51,10 @@ full path.
 .RE
 .TP
 .B \-d, \-\-dryrun
-Print out a list of files that \f[B]would\f[] be deleted.
+Print a list of files that \f[B]would\f[] be deleted without deleting
+them.
 This is useful to check list of items satisfying \-\-exclude or
-\-\-match options without deleting them.
+\-\-match options before actually deleting anything.
 .RS
 .RE
 .TP
@@ -80,7 +78,7 @@ directory tree:
 .PP
 mpirun \-np 128 drm \-\-match \[aq].core$\[aq] /dir/to/delete/from
 .IP "3." 3
-List items that would be deleted, without removing them:
+List items that would be deleted without removing them:
 .PP
 mpirun \-np 128 drm \-\-dryrun \-\-match \[aq].core$\[aq]
 /dir/to/delete/from