Project

General

Profile

Actions
Copy link

Defect #2684

closed

Some messages are missing in the Logging events

Added by Alena Peterová about 3 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Vít Švanda
Category:
Logging
Target version:
11.0.0
Start date:
02/12/2021
Due date:
% Done:

100%

Estimated time:
8.00 h
Affected versions:
Citrine (7.3.0)
Owner:

Description

Tested on 10.7.2 when logger logs also in the DB. See the logback configuration in the attachment.

Some messages aren't saved in the logging events, e.g. when getting an invalid certificate with SSL connection.
(Connector test failed javax.naming.CommunicationException: simple bind failed: 172.31.255.180:636 [Root exception is javax.net.ssl.SSLHandshakeException: No subject alternative names matching IP address 172.31.255.180 found])

Probably the message is too long.

Here are some errors from the PostgreSQL log (the first is caused by the wrong certificate, the others unknown).

2021-02-12 14:34:05.487 UTC [783] ERROR:  value too long for type character varying(254)
2021-02-12 14:34:05.487 UTC [783] STATEMENT:  INSERT INTO logging_event_exception (event_id, i, trace_line) VALUES ($1, $2, $3)

2021-02-03 12:34:02.812 UTC [758] ERROR:  invalid byte sequence for encoding "UTF8": 0x00
2021-02-03 12:34:02.812 UTC [758] STATEMENT:  INSERT INTO logging_event_exception (event_id, i, trace_line) VALUES ($1, $2, $3)

2021-02-03 14:21:00.326 UTC [1116] ERROR:  invalid byte sequence for encoding "UTF8": 0x00
2021-02-03 14:21:00.326 UTC [1116] STATEMENT:  INSERT INTO logging_event (timestmp, formatted_message, logger_name, level_string, thread_name, reference_flag, arg0, arg1, arg2, arg3, caller_filename, caller_class, caller_method, caller_line) VALUES ($1, $2, $3 ,$4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)

Files

logback-spring.xml (1.7 KB) logback-spring.xml Alena Peterová, 02/12/2021 02:32 PM

Related issues

Related to IdStory Identity Manager - Defect #717: Cannot save 0x00 to Postgres columnClosedVít Švanda09/25/2017

Actions
Related to IdStory Identity Manager - Task #574: New agenda for LOGsClosedOndřej Kopr07/13/2017

Actions
Related to bcv-czechidm - Task #2737: Adjust DB logging when CzechIdM 11.0.0 is releasedNew03/29/2021

Actions
Actions
Copy link
 #1

Updated by Petr Fišer about 3 years ago

This one is AD connection with LDAP connector (plaintext LDAP) with bad credentials.

Feb 12 14:07:37 localhost czechidm-db[1190]: 2021-02-12 14:07:37.215 UTC [67] ERROR:  invalid byte sequence for encoding "UTF8": 0x00
Feb 12 14:07:37 localhost czechidm-db[1190]: 2021-02-12 14:07:37.215 UTC [67] STATEMENT:  INSERT INTO logging_event (timestmp, formatted_message, logger_name, level_string, thread_name, reference_flag, arg0, arg1, arg2, arg3, caller_filename, caller_class, caller_method, caller_line) VALUES ($1, $2, $3 ,$4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)
Feb 12 14:07:37 localhost czechidm-db[1190]: #011RETURNING "event_id"

Actions
Copy link
 #2

Updated by Radek Tomiška about 3 years ago

  • Status changed from New to In Progress
  • Target version set to 10.8.0
Actions
Copy link
 #3

Updated by Radek Tomiška about 3 years ago

  • Related to Defect #717: Cannot save 0x00 to Postgres column added
Actions
Copy link
 #4

Updated by Radek Tomiška about 3 years ago

  • Related to Task #574: New agenda for LOGs added
Actions
Copy link
 #5

Updated by Radek Tomiška about 3 years ago

  • Target version changed from 10.8.0 to 11.0.0
  • Affected versions Citrine (7.3.0) added
  • Affected versions deleted (10.7.2)
Actions
Copy link
 #6

Updated by Radek Tomiška about 3 years ago

  • Estimated time set to 8.00 h
Actions
Copy link
 #7

Updated by Vít Švanda about 3 years ago

  • Assignee changed from Radek Tomiška to Vít Švanda
Actions
Copy link
 #8

Updated by Vít Švanda about 3 years ago

I was able simulated this. Bug is in the DBAppender https://jira.qos.ch/browse/LOGBACK-493.

ALTER TABLE public.logging_event_exception ALTER COLUMN trace_line TYPE text USING trace_line::text;

Actions
Copy link
 #9

Updated by Vít Švanda about 3 years ago

The solution in column extension in DB is functional, but not ideal (it would not work, for example, for H2 databases ...). Solving the second problem, replacing characters that cannot be stored in Postgresql, was not easy to solve because we do not have one place to solve this in IdM.

For these reasons, I implemented my own IdmDBAppender, which solves the above problems (max size and replacement of illegal characters u0000/x00).

Commit: https://github.com/bcvsolutions/CzechIdMng/commit/059f82c1b90ee3a2da4bb78351d117b1227960e9

Tests and documentation remains.

Actions
Copy link
 #11

Updated by Radek Tomiška about 3 years ago

  • Assignee changed from Radek Tomiška to Petr Fišer

I did test and code review, it works and code looks nice, our own appender is cool, thx!

Petr, could you please check documentation (mainly installation guides, appliance) from your point of view and add info about better db appender if needed?

Actions
Copy link
 #12

Updated by Petr Fišer about 3 years ago

  • Assignee changed from Petr Fišer to Vít Švanda

I checked the install guide and changelog and it looks ok. Thanks for updating it.
For future change in the IAM appliance, I created #2737 which we will implement once CzechIdM 11.0.0 is released.

I guess that's all, considering the documentation. Thanks for the fix. :)

Actions
Copy link
 #13

Updated by Petr Fišer about 3 years ago

  • Related to Task #2737: Adjust DB logging when CzechIdM 11.0.0 is released added
Actions
Copy link
 #14

Updated by Radek Tomiška about 3 years ago

  • Status changed from Needs feedback to Closed
  • % Done changed from 90 to 100

Awesome, thx!

Actions
Copy link

Also available in: Atom PDF