Skip to content

Commit

Permalink
Add line breaks in description of log.sources setting
Browse files Browse the repository at this point in the history
  • Loading branch information
adutra committed Sep 9, 2020
1 parent 561ec2a commit 23ad297
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 0 deletions.
4 changes: 4 additions & 0 deletions manual/application.template.conf
Original file line number Diff line number Diff line change
Expand Up @@ -1417,17 +1417,21 @@ dsbulk {
# Whether to print record sources in debug files. When set to true (the default), debug files
# will contain, for each record that failed to be processed, its original source, such as the
# text line that the record was parsed from.
#
# Furthermore, when loading, enabling this option also enables the creation of so-called "bad
# files", that is, files containing the original lines that could not be inserted; these files
# could then be used as the data source of a subsequent load operation that would load only the
# failed records.
#
# This feature is useful to locate failed records more easily and diagnose processing failures –
# especially if the original data source is a remote one, such as an FTP or HTTP URL.
#
# But for this feature to be possible, record sources must be kept in memory until the record is
# fully processed. For large record sizes (over 1 megabyte per record), retaining record sources
# in memory could put a high pressure on the JVM heap, thus exposing the operation to
# out-of-memory errors. This phenomenon is exacerbated when batching is enabled. If you are
# experiencing such errors, consider disabling this option.
#
# Note that, regardless of the value of this option, DSBulk will always print the record's
# *resource* – that is, the file name or the database table where it came from – and the
# record's *position* – that is, the ordinal position of the record inside the resource, when
Expand Down
4 changes: 4 additions & 0 deletions manual/settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -1272,9 +1272,13 @@ Default: **50**.
#### --log.sources<br />--dsbulk.log.sources _&lt;boolean&gt;_

Whether to print record sources in debug files. When set to true (the default), debug files will contain, for each record that failed to be processed, its original source, such as the text line that the record was parsed from.

Furthermore, when loading, enabling this option also enables the creation of so-called "bad files", that is, files containing the original lines that could not be inserted; these files could then be used as the data source of a subsequent load operation that would load only the failed records.

This feature is useful to locate failed records more easily and diagnose processing failures – especially if the original data source is a remote one, such as an FTP or HTTP URL.

But for this feature to be possible, record sources must be kept in memory until the record is fully processed. For large record sizes (over 1 megabyte per record), retaining record sources in memory could put a high pressure on the JVM heap, thus exposing the operation to out-of-memory errors. This phenomenon is exacerbated when batching is enabled. If you are experiencing such errors, consider disabling this option.

Note that, regardless of the value of this option, DSBulk will always print the record's *resource* – that is, the file name or the database table where it came from – and the record's *position* – that is, the ordinal position of the record inside the resource, when available (for example, this could be the line number in a CSV file).

Default: **true**.
Expand Down
4 changes: 4 additions & 0 deletions workflow/commons/src/main/resources/dsbulk-reference.conf
Original file line number Diff line number Diff line change
Expand Up @@ -412,9 +412,13 @@ dsbulk {
verbosity = 1

# Whether to print record sources in debug files. When set to true (the default), debug files will contain, for each record that failed to be processed, its original source, such as the text line that the record was parsed from.
#
# Furthermore, when loading, enabling this option also enables the creation of so-called "bad files", that is, files containing the original lines that could not be inserted; these files could then be used as the data source of a subsequent load operation that would load only the failed records.
#
# This feature is useful to locate failed records more easily and diagnose processing failures – especially if the original data source is a remote one, such as an FTP or HTTP URL.
#
# But for this feature to be possible, record sources must be kept in memory until the record is fully processed. For large record sizes (over 1 megabyte per record), retaining record sources in memory could put a high pressure on the JVM heap, thus exposing the operation to out-of-memory errors. This phenomenon is exacerbated when batching is enabled. If you are experiencing such errors, consider disabling this option.
#
# Note that, regardless of the value of this option, DSBulk will always print the record's *resource* – that is, the file name or the database table where it came from – and the record's *position* – that is, the ordinal position of the record inside the resource, when available (for example, this could be the line number in a CSV file).
sources = true

Expand Down

0 comments on commit 23ad297

Please sign in to comment.