Project

General

Profile

Actions

Task #2717

closed

Enhance audit logging in the text log of CzechIdM

Added by Petr Fišer about 3 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
High
Assignee:
Ondrej Husník
Category:
Logging
Target version:
Start date:
03/12/2021
Due date:
% Done:

100%

Estimated time:
40.00 h
Owner:

Description

On many projects, we have to integrate CzechIdM with SIEM software. Such software usually consumes textual log messages via syslog protocol. Consuming them via application REST API may not be ideal, because:
  • Customer may not be able (SIEM product features / lack of funding / ...) to modify their SIEM to work with CzechIdM REST API.
  • Real-time queries into audit trail highten load on our application.

It would be awesome to have audit trail also written into text log of the application. That way, event will be logged at application's discretion and also will make it to syslog and then to SIEM.

Customers want to see those audit messages in their SIEMs (compiled from actual customer requests):
  • Log-on and log-off attempts; both successful and failed ones.
  • Manipulation with user account (=identity) and its privileges; both successful and failed ones.
    • Change of attributes. (Simply logging "somebody changed this identity" should be enough. I think we should not log changed attributes.)
    • Create/delete of identity.
    • Enable/disable of identity.
    • Password change.
    • Password reset.
    • Role assignment/unassignment.
  • Approval or disapproval of a role request.
  • Events on roles.
    • Create/delete/update.
    • Adding/removing privilege on role.
  • Actions performed by administrators. (I guess this is covered by other items in this list, maybe we should explicitly log that "this user is an administrator"?)
  • Failed actions due to missing privileges.
  • Attempts to modify the audit trail. (And maybe also audit trail reads?)
  • LRT start/end and also its success/error state.
  • LRT errors.
  • Application configuration changes.
    • When I define property through configuration agenda, application should log that create/update/delete of property happenned and by whom. Do not log actual value of the property.

The list above may need a bit of thought. It probably will cause heavier logging into application log... so we probably need to create application properties that will make possible to turn parts of it on/off.
Also, readability is a concern here - it is not possible to use internal UUIDs in this log because those data is useless outside of IdM (e.g. in the SIEM).
Hypothetical example of how the message could look like (maybe we need some other fileds there?):

$timestamp $thread $class: AUDIT/IDENTITY_PASSWORD_RESET status:SUCCESS target:uživatel123456 target_uuid:... performed_by:helpdeskový_administrátor performed_by_uuid:... reason:null
$timestamp $thread $class: AUDIT/IDENTITY_PASSWORD_RESET status:FAILURE target:uživatel123456 target_uuid:... performed_by:helpdeskový_administrátor2 performed_by_uuid:... reason:INSUFFICIENT_PRIVILEGES
$timestamp $thread $class: AUDIT/ROLE_CREATE status:SUCCESS target:ADčková_role target_uuid:... performed_by:admin performed_by_uuid:... reason:null

This ticket will probably consist of three main parts:
  • Adding audit logging to various parts of the CzechIdM.
  • Refining existing logging - for example, LRT start/end is already logged, but there are UUIDs. We need to put LRT names there to make the log usable in the SIEM.
  • Making audit matrix and documenting what is audited.
Actions #1

Updated by Radek Tomiška about 3 years ago

  • Category changed from Audit to Logging
  • Assignee deleted (Radek Tomiška)
Actions #2

Updated by Vít Švanda almost 3 years ago

  • Assignee set to Ondrej Husník
  • Priority changed from Normal to High
  • Target version set to 11.2.0
Actions #4

Updated by Vít Švanda almost 3 years ago

  • Estimated time set to 40.00 h
Actions #5

Updated by Ondrej Husník over 2 years ago

  • Status changed from New to In Progress
Actions #6

Updated by Ondrej Husník over 2 years ago

  • % Done changed from 0 to 90

There is the implementation of the audit logging dedicated to using with SIEM tools. To see currently logged events take a look to the documentation. The logger allows hierarchical enabling of individual parts of the logged entities. The logger provides optimization which avoids all data preparation if entity logging is disabled. Logs are written into the stdout.

https://github.com/bcvsolutions/CzechIdMng/commit/ea39adf6fbd245237aed40c94a2513bb19716c77
https://github.com/bcvsolutions/CzechIdMng/commit/0cccd8dceaae133da355b79f7ef40bb57ffd7890
Doc
https://wiki.czechidm.com/devel/documentation/audit/dev/audit_logging_for_siem_tools?s[]=%2Asiem%2A

Please, could you provide me the feedback.

Actions #7

Updated by Ondrej Husník over 2 years ago

  • Status changed from In Progress to Needs feedback
  • Assignee changed from Ondrej Husník to Vít Švanda
Actions #8

Updated by Ondrej Husník over 2 years ago

I refactored the previous logger implementation and moved entity dependent code into appropriate services. AbstractReadDtoService and derived services are able to log siem logs directly. If the log is performed using entity event, the entity dependent code is implemented in the overloaded log method of given service the predecessor of which is put into AbstractEventableDtoService. It contains an empty implementation by default. I preserved the SiemLoggerManager because it contains a lot of dedicated code so it doesn't make sense to me to merge it with the LogbackLoggerManager.

Please provide me a feedback of this code.
https://github.com/bcvsolutions/CzechIdMng/commit/53ecb66ed9bff1b8dc0685c57c81d42e40b2f07c

Actions #9

Updated by Vít Švanda over 2 years ago

  • Status changed from Needs feedback to Resolved
  • Assignee changed from Vít Švanda to Ondrej Husník
  • % Done changed from 90 to 100

I did review and tested it. Works fine. I am gald this feature is module independent now. I modified only loging in role-request-service, where concept with roles were logged. I changed it on logging directly role-request.

I am not sure about name of the method 'AbstractEventableDtoService.siemLog'. May be only 'log' will be more abstract.

I found and fixed some warnings:

Commit: https://github.com/bcvsolutions/CzechIdMng/commit/d0cfec9e3a999838094fe6e91f31961387489f10

Actions #10

Updated by Ondrej Husník over 2 years ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF