Project

General

Profile

Task #2717

Enhance audit logging in the text log of CzechIdM

Added by Petr Fišer 7 months ago. Updated 30 days ago.

Status:
Closed
Priority:
High
Assignee:
Ondrej Husník
Category:
Logging
Target version:
Start date:
03/12/2021
Due date:
% Done:

100%

Estimated time:
Milestones:

Description

On many projects, we have to integrate CzechIdM with SIEM software. Such software usually consumes textual log messages via syslog protocol. Consuming them via application REST API may not be ideal, because:
  • Customer may not be able (SIEM product features / lack of funding / ...) to modify their SIEM to work with CzechIdM REST API.
  • Real-time queries into audit trail highten load on our application.

It would be awesome to have audit trail also written into text log of the application. That way, event will be logged at application's discretion and also will make it to syslog and then to SIEM.

Customers want to see those audit messages in their SIEMs (compiled from actual customer requests):
  • Log-on and log-off attempts; both successful and failed ones.
  • Manipulation with user account (=identity) and its privileges; both successful and failed ones.
    • Change of attributes. (Simply logging "somebody changed this identity" should be enough. I think we should not log changed attributes.)
    • Create/delete of identity.
    • Enable/disable of identity.
    • Password change.
    • Password reset.
    • Role assignment/unassignment.
  • Approval or disapproval of a role request.
  • Events on roles.
    • Create/delete/update.
    • Adding/removing privilege on role.
  • Actions performed by administrators. (I guess this is covered by other items in this list, maybe we should explicitly log that "this user is an administrator"?)
  • Failed actions due to missing privileges.
  • Attempts to modify the audit trail. (And maybe also audit trail reads?)
  • LRT start/end and also its success/error state.
  • LRT errors.
  • Application configuration changes.
    • When I define property through configuration agenda, application should log that create/update/delete of property happenned and by whom. Do not log actual value of the property.

The list above may need a bit of thought. It probably will cause heavier logging into application log... so we probably need to create application properties that will make possible to turn parts of it on/off.
Also, readability is a concern here - it is not possible to use internal UUIDs in this log because those data is useless outside of IdM (e.g. in the SIEM).
Hypothetical example of how the message could look like (maybe we need some other fileds there?):

$timestamp $thread $class: AUDIT/IDENTITY_PASSWORD_RESET status:SUCCESS target:uživatel123456 target_uuid:... performed_by:helpdeskový_administrátor performed_by_uuid:... reason:null
$timestamp $thread $class: AUDIT/IDENTITY_PASSWORD_RESET status:FAILURE target:uživatel123456 target_uuid:... performed_by:helpdeskový_administrátor2 performed_by_uuid:... reason:INSUFFICIENT_PRIVILEGES
$timestamp $thread $class: AUDIT/ROLE_CREATE status:SUCCESS target:ADčková_role target_uuid:... performed_by:admin performed_by_uuid:... reason:null

This ticket will probably consist of three main parts:
  • Adding audit logging to various parts of the CzechIdM.
  • Refining existing logging - for example, LRT start/end is already logged, but there are UUIDs. We need to put LRT names there to make the log usable in the SIEM.
  • Making audit matrix and documenting what is audited.

History

#1 Updated by Radek Tomiška 7 months ago

  • Category changed from Audit to Logging
  • Assignee deleted (Radek Tomiška)

#2 Updated by Vít Švanda 5 months ago

  • Assignee set to Ondrej Husník
  • Priority changed from Normal to High
  • Target version set to 11.2.0

#5 Updated by Ondrej Husník 3 months ago

  • Status changed from New to In Progress

#6 Updated by Ondrej Husník about 2 months ago

  • % Done changed from 0 to 90

There is the implementation of the audit logging dedicated to using with SIEM tools. To see currently logged events take a look to the documentation. The logger allows hierarchical enabling of individual parts of the logged entities. The logger provides optimization which avoids all data preparation if entity logging is disabled. Logs are written into the stdout.

https://github.com/bcvsolutions/CzechIdMng/commit/ea39adf6fbd245237aed40c94a2513bb19716c77
https://github.com/bcvsolutions/CzechIdMng/commit/0cccd8dceaae133da355b79f7ef40bb57ffd7890
Doc
https://wiki.czechidm.com/devel/documentation/audit/dev/audit_logging_for_siem_tools?s[]=%2Asiem%2A

Please, could you provide me the feedback.

#7 Updated by Ondrej Husník about 2 months ago

  • Status changed from In Progress to Needs feedback
  • Assignee changed from Ondrej Husník to Vít Švanda

#8 Updated by Ondrej Husník about 2 months ago

I refactored the previous logger implementation and moved entity dependent code into appropriate services. AbstractReadDtoService and derived services are able to log siem logs directly. If the log is performed using entity event, the entity dependent code is implemented in the overloaded log method of given service the predecessor of which is put into AbstractEventableDtoService. It contains an empty implementation by default. I preserved the SiemLoggerManager because it contains a lot of dedicated code so it doesn't make sense to me to merge it with the LogbackLoggerManager.

Please provide me a feedback of this code.
https://github.com/bcvsolutions/CzechIdMng/commit/53ecb66ed9bff1b8dc0685c57c81d42e40b2f07c

#9 Updated by Vít Švanda about 1 month ago

  • Status changed from Needs feedback to Resolved
  • Assignee changed from Vít Švanda to Ondrej Husník
  • % Done changed from 90 to 100

I did review and tested it. Works fine. I am gald this feature is module independent now. I modified only loging in role-request-service, where concept with roles were logged. I changed it on logging directly role-request.

I am not sure about name of the method 'AbstractEventableDtoService.siemLog'. May be only 'log' will be more abstract.

I found and fixed some warnings:

Commit: https://github.com/bcvsolutions/CzechIdMng/commit/d0cfec9e3a999838094fe6e91f31961387489f10

#10 Updated by Ondrej Husník 30 days ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF

Go to top