Release 2.5.1

Various compatibility issues and a possible runtime bug, related to pcap-thread, fixed.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dsc/dsc-2.5.1.tar.gz
sha256: bae45e2d4722c6064902dc802149a6f4f3531e7145792d2827ce58f6cc463ff3

Packages are available at: https://dev.dns-oarc.net/packages/

Commits:

5ed03e3 Compat for OS X
8605759 Fix compiler warnings
5fbad26 Update pcap-thread to v2.1.2
47ed110 Update pcap-thread to v2.1.1

Release 2.5.0

Resolved memory leaks within the IP fragment reassembly code that was reported by Klaus Darilion (NIC.AT) and added config option to control some parts of the fragment handling.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dsc/dsc-2.5.0.tar.gz
sha256: c736a4ea14f72a967db806797d199344119238817b0903a5079792ec541d494d

Packages are available at: https://dev.dns-oarc.net/packages/

Fixes:

• Add pcap_layers_clear_fragments() to remove old fragments after MAX_FRAG_IDLE (60 seconds)
• Use correct alloc/free functions for dataset hash
• Fix spacing in dsc.conf(5) man-page

New config option:

• drop_ip_fragments will disable IP fragmentation reassembling and drop any IP packet that is a fragment (even the first)

Commits:

eaee6c0 Drop IP fragments
3ebb687 Issue #146: Fix leak in fragment handling
9a5e377 Use correct alloc/free
35f663c Fix #107: add const

Release 2.4.0

Since there have been a few major issues with the threaded capturing code it is now default disabled and have to be enabled with a configure option to use: ./configure --enable-threads ...

A lot of work has been done to ensure stability and correct capturing, as of now dsc is continuously running on the testing platforms with simulated traffic and tests are performance every 5-15 minutes:

https://dev.dns-oarc.net/jenkins/view/dsctest/

With the rewrite of the config parser to C it was missed that Hapy allowed CR/LF within the values of the options. Changing the C parser to allow it is a bit of work and having CR/LF within the value may lead to other issues so it is now documented that CR/LF are not allowed in config option values.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dsc/dsc-2.4.0.tar.gz
sha256: dacbe67ed724be2d6b96f3f1f2a4eb5509ee7f6541b3adf7188afeb708a6179f

Packages are available at: https://dev.dns-oarc.net/packages/

Fixes:

• The -T flag was just controlling pcap-thread usage of threads, it now controls all usage of threads including how signals are caught.
• Fix program name, was incorrectly set so it would be reported as /dsc.
• Use thread safe functions (_r).
• Handle very long config lines by not having a static buffer, instead let getline() allocate as needed.
• Use new activation in pcap-thread to activate the capturing of pcaps after the initial interval sync have been done during start-up.
• Use factions of second for start-up interval sync and interval wait.
• Fix memory leaks if config options was specified more then once.
• Use new absolute timed run in pcap-thread to more exactly end capturing at the interval.
• Fix config parsing, was checking for tab when should look for line feed.
• Exit correctly during pcap-thread run to honor dump_reports_on_exit.
• Use 100ms as default pcap-thread timeout, was 1s before but the old code used 250ms.
• Various enhancements to logging of errors.

New config options/features:

• pcap_buffer_size can be used to increase the capture buffer within pcap-thread/libpcap, this can help mitigate dropped packets by the kernel during interval breaks.
• no_wait_interval will skip the interval sync that happens during start-up and start capturing directly, the end of the interval will still be the modulus of the interval.
• pcap_thread_timeout can be used to change the internal timeout use in pcap-thread to wait for packets (default 100ms).
• Log non-fatal errors from pcap-thread w.r.t. setting the filter which can indicate that the filter is running in userland because lack of support or that it is too large for the kernel.

Special thanks to:

• @aabdnn Anand Buddhdev, RIPE NCC
• @klaus3000 Klaus Darilion, NIC.AT
• @McStork Vincent Charrade, Nameshield

Commits:

ee59572 Fix #111, fix #116: Update pcap-thread to v2.0.0, remove debug code
64befef Update copyright year
40a1fb4 Fix #139: Use 100ms as default pcap-thread timeout
2a07185 Fix #137: Graceful exit on signal during run
f1b3ec3 Issue #116: Try and make select issue more clear
950ea96 Fix #133: Return from Pcap_run() on signal/errors
667cc91 Issue #116: Add config option pcap_thread_timeout
3c9e073 Notice if non-fatal errors was detected during activation
4ea8f54 Fix #108: Document that CR/LF are not allowed within configuration line
9fda332 Check for LF and not tab
15a1dc0 Use pcap-thread timed run to interface
1e98f8b Fix potential memory leaks if config options specified more then once
a9b38e9 Add missing LF and indicate what config option was wrong if possible
f8a2821 Use fractions of seconds for both start up interval sync and timed run, always adjust for inter-run processing delay
f47069a Fix #121: Update to pcap-thread latest develop
fc13d73 Issue #116: Feature for not waiting on the interval sync
c832337 Fix #122: Update pcap-thread to v1.2.3 for fix in timed run
4739111 Add pcap_buffer_size config option
7d9bf90 Update pcap-thread to v1.2.2
ef43335 Make threads optional and default disabled
c2399cf getline() returns error on eof, don't report error if we are
5c671e6 Clarify config error message and report getline() error
8bd6a67 Fix #114: Handle very long lines
47b1e1a Use _r thread safe functions when possible
0f5d883 Update daemon.c
f18e3ea Update doc, -T now disables all usage of threads
57aacbe Honor the -T flag when installing signal handlers

Release 2.3.0

Rare lockup has been fixed that could happen if a signal was received in the wrong thread at the wrong time due to pcap_thread_stop() canceling and waiting on threads to join again. The handling of signals have been improved for threaded and non-threaded operations.

A couple of bugfixes, one to fix loading of GeoIP ASN database and another to use the lowest 32 bits of an IP address (being v4 or v6) in the IP hash making it a bit more efficient for v6 addresses.

New functionality for the configure option local_address, you can now specify a network mask (see man 5 dsc.conf for syntax).

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dsc/dsc-2.3.0.tar.gz
sha256: 4bc0b578d4486dcd45f61c5aa351cfd6788873ae91bb97aff81a56ddafefa68f

Packages are available at: https://dev.dns-oarc.net/packages/

Commits:

e286298 Fix CID 158968 Bad bit shift operation
c15db43 Update to pcap-thread v1.2.1
1ac06ac Move stopping process to not require a packet
597dd34 Handle signals better with and without pthreads
bcf99e8 Add RPM spec and ACLOCAL_AMFLAGS to build on CentOS 6
667fe69 fixed load geoIP ASN database from config-file
e1304d4 Fix #97: Add optional mask to local_address so you can specify networks
5dae7dd Fix #96: Hash the lowest 32 bits of IP addresses

Release 2.2.1

Two bugfixes with one being critical (caused segfault, sorry for that)

• pcap-thread had an issue that threads where not closed on exit of pcap_thread_run(), this only created many threads in my testing environment but reports came in that it segfaulted.
• When started dsc (not in debug mode) it will wait to align with the time, that did not get update with the configurable interval change.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dsc/dsc-2.2.1.tar.gz
sha256: d7b27533c56a3b599ffee82eb503fd0160a9ca1e9461c9d4d75f8bbe9aa1bf12

Packages are available at: https://dev.dns-oarc.net/packages/

Commits:

6e3654b Fix #90: Update pcap-thread to v1.1.2 to fix a segfault issue
c9350a3 Fix #92: Honor configured interval when aligning
acaf617 minor typo

Release 2.2.0

Some big changes in this release are the removal of the C++ configure parser library Hapy and the addition of pcap-thread to (hopefully) handle capturing packet in a correct and efficient way.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dsc/dsc-2.2.0.tar.gz
sha256: 94e90fc30e638db3052a821432838637e6fd377f488a66296692bffa68101924

Packages are available at: https://dev.dns-oarc.net/packages/

With the addition of pcap-thread comes new runtime options:

• -m sets monitor mode on interfaces
• -i sets immediate mode on interfaces
• -T disable the usage of threads in pcap thread

Bugfixes:

• d95190a fixes a small memory leak in Pcap_init() and a possible bug where -p might never been used because of not being declared external.
• 55e1056 added check for netinet/ip_compat.h for use in src/pcap_layers/pcap_layers.c.

Commits:

0a1ce91 Fix coverity issues
5a1d410 Delete useless line (related to mayasd#84)
443db3e Check if the file was previously linked but not yet unlinked (Tim CLERC.IM)
02a7621 Fix #82: Oops, pcap_thread_set_filter() had changed during development and missed this
2a8aa29 Move definition of token struct inside and changed name to not conflict on FreeBSD
43da964 Fix #9: Implement conf parser in C and remove dependency of Hapy and C++
9f46f0d Update pcap thread to version 1.1.1
d95190a Use pcap thread, new options -miT and possible bugfix for -p and a small memory leak fix
55e1056 Fix #77: Check for netinet/ip_compat
4e120f9 Fix travis script, only expand dir.

Release 2.1.1

Bugfixes:

• 22688c1 Fix pcap/select last_ts
  In some cases select will return the fd set as if there are packets
  to read but there aren't. That would case the last timestamp to not
  advance and the Pcap_run loop to never finish.
  This fix add a check on packets captured and sets last timestamp if
  none where caught.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dsc/dsc-2.1.1.tar.gz
sha256: f5952a01032741b7bfc2fa932d131e69c82b3f05fbea91a5f2106a8f2bd8e763

Release 2.1.0

This release brings a couple of new features, the ability to change the interval (#64) for which DSC writes out the statistics files and a new indexer for AS numbers (#66).
There was also an issue (#62) detected in Ubuntu 16.04 when reading pcap files would result in doubling the statistics if running in daemon mode. Looking at the strace it might be a issue with the kernel, libc, pcap or a combination that results in the open file handle to reset and essentially read all the data twice.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dsc/dsc-2.1.0.tar.gz
sha256: 47ba402b0bace79cfb95a4231ca53dca6ccd6decb32cec685be748dc561c8c49

Credits:

• Klaus Darilion @klaus3000 NIC.AT for interval changes and AS number indexer.

Release 2.0.0

This release brings a major update to the DSC software with the separation
of the Collector and the Presenter, this repository will only include
the Collector from now on.

Please read UPGRADE.md for information on upgrading from previous version.

Due to submodules in the repository please download this tarball:
https://www.dns-oarc.net/files/dsc/dsc-2.0.0.tar.gz
sha256: 56d1c58cc912aeb6db244ea71be5ee3ae53f78c70bfbd258c4d9ccd63988068b

Major changes / additions are:

• Use of Automake and rework of the Makefiles
• Conform to FHS 3.0
• Man-pages (man dsc, man dsc.conf)
• Continuous Integration testing using Travis-CI
• Compatibility testing on Debian, CentOS, FreeBSD and OpenBSD
• Use of Coverity Scan to find defects
• JSON output format, see output_format in dsc.conf(5).
• IPv6 support in country indexer and libgeoip is now runtime
  configurable, see geoip_v4_dat / geoip_v6_dat in dsc.conf(5).
• Signal handling and optional write reports on exit, see
  dump_reports_on_exit in dsc.conf(5).
• Upload scripts are deprecated

Credits:

• Klaus Darilion @klaus3000 NIC.AT for GeoIP IPv6 patch.
• Michael Braunoeder NIC.AT for NXDOMAIN filter patch.
• L-root for overflow bugfix IP fragments.
• @McStork for JSON output patch.

Release 2.0.0-rc.1

Release candidate 1 for version 2.0.0.