FileCollector

FileCollector

Being a Collector means that it can receive data from a single source and then store it in some form.

This particular collector:

• Writes data received to a file

Purpose

Store data from a source in a file.

Main features

• Flushing of data from memory is based on time since last data received or amount of messages received
• By default system newline is used, but can be altered
• Multiple header lines allowed that use the set newline
• Allow rollover based on a fixed schedule and zipping of the old file
• Allow renaming based on max file size reached and optional zipping
• Execute commands on idle,max size and rollover with {path} as macro for the resulting file path

Example

  <collectors>
    <!-- Maximum options used --> 
    <file id="sensor" src="raw:test"> <!-- the id and the source of data -->
      <flush batchsize="10" age="15s"/> <!-- Write if 10 messages are received or the data is 15s old -->
      <rollover count="15" unit="min" zip="yes">HHmm</rollover> <!-- New file every 15minutes, {rollover} is replaced with calculated HHmm -->
      <maxsize zip="true">100mb</maxsize>  <!-- When an append increases the size over 100mb the file will be renamed to file.1.extension and zipped -->
      <path>csv/data_{rollover}.csv</path> <!-- file to write to with rollover -->
      <header>columnnames</header> <!-- header line -->
      <header>units</header> <!-- another header line -->
      <cmd trigger="rollover">email:admin,Rollover done,Get old file?</cmd> <!-- After the rollover was executed do this -->
      <cmd trigger="idle">email:admin,Not getting data,sensor issue?</cmd>  <!-- If a timeout was reached and the databuffer is empty, do this -->
      <cmd trigger="maxsize">email:admin,File got big!, Check {path}</cmd>  <!-- If max size limit reached, do this -->
    </file>
  </collectors>