diff --git a/README.md b/README.md index 5745fe8..5b2683b 100644 --- a/README.md +++ b/README.md @@ -122,25 +122,19 @@ Both model-level directives accept additional options to be passed into ActiveJo # For change journaling: journal_changes_to :email, as: :identity_change, enqueue_with: { priority: 10 } +# For audit logging: +has_audit_log enqueue_with: { priority: 30 } + # Or for custom journaling: journal_attributes :email, enqueue_with: { priority: 20, queue: 'journaled' } ``` -### Change Journaling - -Out of the box, `Journaled` provides an event type and ActiveRecord -mix-in for durably journaling changes to your model, implemented via -ActiveRecord hooks. Use it like so: - -```ruby -class User < ApplicationRecord - include Journaled::Changes - - journal_changes_to :email, :first_name, :last_name, as: :identity_change -end -``` +### Attribution -Add the following to your controller base class for attribution: +Before using `Journaled::Changes` or `Journaled::AuditLog`, you will want to +set up automatic "actor" attribution (i.e. tracking the current user session). +To enable this feature, add the following to your controller base class for +attribution: ```ruby class ApplicationController < ActionController::Base @@ -153,6 +147,20 @@ end Your authenticated entity must respond to `#to_global_id`, which ActiveRecords do by default. This feature relies on `ActiveSupport::CurrentAttributes` under the hood. +### Change Journaling with `Journaled::Changes` + +Out of the box, `Journaled` provides an event type and ActiveRecord +mix-in for durably journaling changes to your model, implemented via +ActiveRecord hooks. Use it like so: + +```ruby +class User < ApplicationRecord + include Journaled::Changes + + journal_changes_to :email, :first_name, :last_name, as: :identity_change +end +``` + Every time any of the specified attributes is modified, or a `User` record is created or destroyed, an event will be sent to Kinesis with the following attributes: @@ -179,6 +187,213 @@ journaling. Note that the less-frequently-used methods `toggle`, `increment*`, `decrement*`, and `update_counters` are not intercepted at this time. + +### Audit Logging with `Journaled::AuditLog` + +Journaled includes a feature for producing audit logs of changes to your model. +Unlike `Journaled::Changes`, which will emit individual sets of changes as +"logical" events, `Journaled::AuditLog` will log all changes in their entirety, +unless otherwise told to ignore changes to specific columns. + +This behavior is similar to +[papertrail](https://github.com/paper-trail-gem/paper_trail), +[audited](https://github.com/collectiveidea/audited), and +[logidze](https://github.com/palkan/logidze), except instead of storing +changes/versions locally (in your application's database), it emits them to +Kinesis (as Journaled events). + +#### Audit Log Configuration + +To enable audit logging for a given record, use the `has_audit_log` directive: + +```ruby +class MyModel < ApplicationRecord + has_audit_log + + # This class will now be audited, + # but will ignore changes to `created_at` and `updated_at`. +end +``` + +To ignore changes to additional columns, use the `ignore` option: + +```ruby +class MyModel < ApplicationRecord + has_audit_log ignore: :last_synced_at + + # This class will be audited, + # and will ignore changes to `created_at`, `updated_at`, and `last_synced_at`. +end +``` + +By default, changes to `updated_at` and `created_at` will be ignored (since +these generally change on every update), but this behavior can be reconfigured: + +```ruby +# change the defaults: +Journaled::AuditLog.default_ignored_columns = %i(createdAt updatedAt) + +# or append new defaults: +Journaled::AuditLog.default_ignored_columns += %i(modified_at) + +# or disable defaults entirely: +Journaled::AuditLog.default_ignored_columns = [] +``` + +Subclasses will inherit audit log configs: + +```ruby +class MyModel < ApplicationRecord + has_audit_log ignore: :last_synced_at +end + +class MySubclass < MyModel + # this class will be audited, + # and will ignore `created_at`, `updated_at`, and `last_synced_at`. +end +``` + +To disable audit logs on subclasses, use `skip_audit_log`: + +```ruby +class MySubclass < MyModel + skip_audit_log +end +``` + +Subclasses may specify additional columns to ignore (which will be merged into +the inherited list): + +```ruby +class MySubclass < MyModel + has_audit_log ignore: :another_field + + # this class will ignore `another_field`, IN ADDITION TO `created_at`, `updated_at`, + # and any other fields specified by the parent class. +end +``` + +To temporarily disable audit logging globally, use the `without_audit_logging` directive: + +```ruby +Journaled::AuditLog.without_audit_logging do + # Any operation in here will skip audit logging +end +``` + +#### Audit Log Events + +Whenever an audited record is created, updated, or destroyed, a +`journaled_audit_log` event is emitted. For example, calling +`user.update!(name: 'Bart')` would result in an event that looks something like +this: + +```json +{ + "id": "bc7cb6a6-88cf-4849-a4f0-a31b0b199c47", + "event_type": "journaled_audit_log", + "created_at": "2022-01-28T11:06:54.928-05:00", + "class_name": "User", + "table_name": "users", + "record_id": "123", + "database_operation": "update", + "changes": { "name": ["Homer", "Bart"] }, + "snapshot": null, + "actor": "gid://app_name/AdminUser/456", + "tags": {} +} +``` + +The field breakdown is as follows: + +- `id`: a randomly-generated ID for the event itself +- `event_type`: the type of event (always `journaled_audit_log`) +- `created_at`: the time that the action occurred (should match `updated_at` on + the ActiveRecord) +- `class_name`: the name of the ActiveRecord class +- `table_name`: the underlying table that the class interfaces with +- `record_id`: the primary key of the ActiveRecord +- `database_operation`: the type of operation (`insert`, `update`, or `delete`) +- `changes`: the changes to the record, in the form of `"field_name": + ["from_value", "to_value"]` +- `snapshot`: an (optional) snapshot of all of the record's columns and their + values (see below). +- `actor`: the current `Journaled.actor` +- `tags`: the current `Journaled.tags` + +#### Snapshots + +When records are created, updated, and deleted, the `changes` field is populated +with only the columns that changed. While this keeps event payload size down, it +may make it harder to reconstruct the state of the record at a given point in +time. + +This is where the `snapshot` field comes in! To produce a full snapshot of a +record as part of an update, set use the virtual `_log_snapshot` attribute, like +so: + +```ruby +my_user.update!(name: 'Bart', _log_snapshot: true) +``` + +Or to produce snapshots for all records that change for a given operation, +wrap it a `with_snapshots` block, like so: + +```ruby +Journaled::AuditLog.with_snapshots do + ComplicatedOperation.run! +end +``` + +Events with snapshots will continue to populate the `changes` field, but will +additionally contain a snapshot with the full state of the user: + +```json +{ + "...": "...", + "changes": { "name": ["Homer", "Bart"] }, + "snapshot": { "name": "Bart", "email": "simpson@example.com", "favorite_food": "pizza" }, + "...": "..." +} +``` + +#### Handling Sensitive Data + +Both `changes` and `snapshot` will filter out sensitive fields, as defined by +your `Rails.application.config.filter_parameters` list: + +```json +{ + "...": "...", + "changes": { "ssn": ["[FILTERED]", "[FILTERED]"] }, + "snapshot": { "ssn": "[FILTERED]" }, + "...": "..." +} +``` + +They will also filter out any fields whose name ends in `_crypt` or `_hmac`, as +well as fields that rely on Active Record Encryption / `encrypts` ([introduced +in Rails 7](https://edgeguides.rubyonrails.org/active_record_encryption.html)). + +This is done to avoid emitting values to locations where it is difficult or +impossible to rotate encryption keys (or otherwise scrub values after the +fact), and currently there is no built-in configuration to bypass this +behavior. If you need to track changes to sensitive/encrypted fields, it is +recommended that you store the values in a local history table (still +encrypted, of course!). + +#### Caveats + +Because Journaled events are not guaranteed to arrive in order, events emitted +by `Journaled::AuditLog` must be sorted by their `created_at` value, which +should correspond roughly to the time that the SQL statement was issued. +**There is currently no other means of globally ordering audit log events**, +making them susceptible to clock drift and race conditions. + +These issues may be mitigated on a per-model basis via +`ActiveRecord::Locking::Optimistic` (and its auto-incrementing `lock_version` +column), and/or by careful use of other locking mechanisms. + ### Custom Journaling For every custom implementation of journaling in your application, define the JSON schema for the attributes in your event. @@ -338,7 +553,7 @@ Returns one of the following in order of preference: * a string of the form `gid://[app_name]` as a fallback In order for this to be most useful, you must configure your controller -as described in [Change Journaling](#change-journaling) above. +as described in [Attribution](#attribution) above. ### Testing diff --git a/app/models/journaled/audit_log/event.rb b/app/models/journaled/audit_log/event.rb index abfd759..165ab6a 100644 --- a/app/models/journaled/audit_log/event.rb +++ b/app/models/journaled/audit_log/event.rb @@ -3,7 +3,7 @@ # make sense to move it to lib/. module Journaled module AuditLog - Event = Struct.new(:record, :database_operation, :unfiltered_changes) do + Event = Struct.new(:record, :database_operation, :unfiltered_changes, :enqueue_opts) do include Journaled::Event journal_attributes :class_name, :table_name, :record_id, @@ -13,6 +13,10 @@ def journaled_stream_name AuditLog.default_stream_name || super end + def journaled_enqueue_opts + record.class.audit_log_config.enqueue_opts + end + def created_at case database_operation when 'insert' diff --git a/lib/journaled/audit_log.rb b/lib/journaled/audit_log.rb index 5071c29..78ca40a 100644 --- a/lib/journaled/audit_log.rb +++ b/lib/journaled/audit_log.rb @@ -15,6 +15,7 @@ module AuditLog mattr_accessor(:default_ignored_columns) { %i(created_at updated_at) } mattr_accessor(:default_stream_name) { Journaled.default_stream_name } + mattr_accessor(:default_enqueue_opts) { {} } mattr_accessor(:excluded_classes) { DEFAULT_EXCLUDED_CLASSES.dup } thread_mattr_accessor(:snapshots_enabled) { false } thread_mattr_accessor(:_disabled) { false } @@ -64,18 +65,37 @@ def classic_exclude!(name) end end - Config = Struct.new(:enabled, :ignored_columns) do - private :enabled + Config = Struct.new(:enabled, :ignored_columns, :enqueue_opts) do + def self.default + new(false, AuditLog.default_ignored_columns.dup, AuditLog.default_enqueue_opts.dup) + end + + def initialize(*) + super + self.ignored_columns ||= [] + self.enqueue_opts ||= {} + end + def enabled? !AuditLog._disabled && self[:enabled].present? end + + def dup + super.tap do |config| + config.ignored_columns = ignored_columns.dup + config.enqueue_opts = enqueue_opts.dup + end + end + + private :enabled end included do prepend BlockedMethods singleton_class.prepend BlockedClassMethods - class_attribute :audit_log_config, default: Config.new(false, AuditLog.default_ignored_columns) + class_attribute :audit_log_config, default: Config.default + attr_accessor :_log_snapshot after_create { _emit_audit_log!('insert') } @@ -84,19 +104,16 @@ def enabled? end class_methods do - def has_audit_log(ignore: []) - ignored_columns = _audit_log_inherited_ignored_columns + [ignore].flatten(1) - self.audit_log_config = Config.new(true, ignored_columns.uniq) + def has_audit_log(ignore: [], enqueue_with: {}) + self.audit_log_config = audit_log_config.dup + audit_log_config.enabled = true + audit_log_config.ignored_columns |= [ignore].flatten(1) + audit_log_config.enqueue_opts.merge!(enqueue_with) end def skip_audit_log - self.audit_log_config = Config.new(false, _audit_log_inherited_ignored_columns.uniq) - end - - private - - def _audit_log_inherited_ignored_columns - (superclass.try(:audit_log_config)&.ignored_columns || []) + audit_log_config.ignored_columns + self.audit_log_config = audit_log_config.dup + audit_log_config.enabled = false end end @@ -177,7 +194,7 @@ module BlockedClassMethods def _emit_audit_log!(database_operation) if audit_log_config.enabled? - event = Journaled::AuditLog::Event.new(self, database_operation, _audit_log_changes) + event = Journaled::AuditLog::Event.new(self, database_operation, _audit_log_changes, audit_log_config.enqueue_opts) ActiveSupport::Notifications.instrument('journaled.audit_log.journal', event: event) do event.journal! end diff --git a/lib/journaled/version.rb b/lib/journaled/version.rb index bc37eb6..9f26ec8 100644 --- a/lib/journaled/version.rb +++ b/lib/journaled/version.rb @@ -1,3 +1,3 @@ module Journaled - VERSION = "5.1.0".freeze + VERSION = "5.1.1".freeze end diff --git a/spec/lib/journaled/audit_log_spec.rb b/spec/lib/journaled/audit_log_spec.rb index 12b7a14..d49cd8c 100644 --- a/spec/lib/journaled/audit_log_spec.rb +++ b/spec/lib/journaled/audit_log_spec.rb @@ -66,6 +66,16 @@ def encrypted_attribute?(_key) end end + describe '.default_enqueue_opts' do + it 'defaults to timestamps, but is configurable' do + expect(described_class.default_enqueue_opts).to eq({}) + described_class.default_enqueue_opts = { priority: 99 } + expect(described_class.default_enqueue_opts).to eq(priority: 99) + ensure + described_class.default_enqueue_opts = {} + end + end + describe '.excluded_classes' do let(:defaults) do %w( @@ -100,15 +110,19 @@ def encrypted_attribute?(_key) it 'enables/disables audit logging' do expect(subject.audit_log_config.enabled?).to be(false) expect(subject.audit_log_config.ignored_columns).to eq(%i(DEFAULTS)) + expect(subject.audit_log_config.enqueue_opts).to eq({}) subject.has_audit_log expect(subject.audit_log_config.enabled?).to be(true) expect(subject.audit_log_config.ignored_columns).to eq(%i(DEFAULTS)) - subject.has_audit_log ignore: %i(foo bar baz) + expect(subject.audit_log_config.enqueue_opts).to eq({}) + subject.has_audit_log ignore: %i(foo bar baz), enqueue_with: { priority: 30 } expect(subject.audit_log_config.enabled?).to be(true) expect(subject.audit_log_config.ignored_columns).to eq(%i(DEFAULTS foo bar baz)) + expect(subject.audit_log_config.enqueue_opts).to eq(priority: 30) subject.skip_audit_log expect(subject.audit_log_config.enabled?).to be(false) expect(subject.audit_log_config.ignored_columns).to eq(%i(DEFAULTS foo bar baz)) + expect(subject.audit_log_config.enqueue_opts).to eq(priority: 30) end it 'can be composed with multiple calls' do @@ -140,13 +154,17 @@ def encrypted_attribute?(_key) it 'inherits the config by default, and merges ignored columns' do expect(MySubclass.audit_log_config.enabled?).to be(false) expect(MySubclass.audit_log_config.ignored_columns).to eq(%i(DEFAULTS)) - subject.has_audit_log ignore: %i(foo) + expect(MySubclass.audit_log_config.enqueue_opts).to eq({}) + subject.has_audit_log ignore: %i(foo), enqueue_with: { priority: 10 } expect(MySubclass.audit_log_config.enabled?).to be(true) expect(MySubclass.audit_log_config.ignored_columns).to eq(%i(DEFAULTS foo)) - MySubclass.has_audit_log ignore: :bar + expect(MySubclass.audit_log_config.enqueue_opts).to eq(priority: 10) + MySubclass.has_audit_log ignore: :bar, enqueue_with: { priority: 30 } expect(MySubclass.audit_log_config.enabled?).to be(true) expect(subject.audit_log_config.ignored_columns).to eq(%i(DEFAULTS foo)) + expect(subject.audit_log_config.enqueue_opts).to eq(priority: 10) expect(MySubclass.audit_log_config.ignored_columns).to eq(%i(DEFAULTS foo bar)) + expect(MySubclass.audit_log_config.enqueue_opts).to eq(priority: 30) end it 'allows the subclass to skip audit logging, and vice versa' do