Project

General

Profile

Actions
Copy link

Task #1806

open

ClearDirtyStateForContractSliceTaskExecutor runs in single transaction when started after Synchronization

Added by Alena Peterová over 5 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Long running task
Target version:
-
Start date:
08/20/2019
Due date:
% Done:

0%

Estimated time:
20.00 h
Owner:

Description

Version 9.7.3

ClearDirtyStateForContractSliceTaskExecutor runs in one single transaction when started after Synchronization. This is problematic when there are many time slices of contracts or many EAV's of contracts. It fails to finish. And when it fails, the processed dirty flags are not deleted from the DB => we are at the beginning.
When ClearDirtyStateForContractSliceTaskExecutor starts as a scheduled task, it processes every dirty flag separately. So when it fails, we can continue where we left.

Please change it so ClearDirtyStateForContractSliceTaskExecutor doesn't run in single transaction. And also please solve #1413, which slows it down very much.

Details:
We have 64919 slices after reconcilation, every contract (slice) has 10 EAVs. The ClearDirtyStateForContractSliceTaskExecutor after synchronization processed 27669 in 6 hours, but no change was commited. No entity event was visible in IdM and no provisioning was done (I don't know if they started or not...).
When I cancelled the task manually, it started to commit the big transaction. After 2 hours, it still wasn't commited. When I restarted IdM, all was lost.
During the 2 hours, the database queries almost always inserted something in idm_audit or selected next value from hibernate_sequence. The hibernate_sequence used cca 20 000 IDs.
Now I run ClearDirtyStateForContractSliceTaskExecutor from scheduled task. It has processed 24230 in 6,5 hours and everything is commited. The hibernate_sequence used cca 300 000 IDs.
There is evidently some problem with the big transaction for all slices - it's much slower than single transactions for every slice.

Some screenshots are attached - how the task was canceled and what happened in PostgreSQL during the commit.

Files

Download all files
hibernate_sequence_value.png (11.4 KB) hibernate_sequence_value.png Alena Peterová, 08/20/2019 03:21 PM
postgres_transaction.png (56.3 KB) postgres_transaction.png Alena Peterová, 08/20/2019 03:21 PM
cancel_task.png (16.9 KB) cancel_task.png Alena Peterová, 08/20/2019 03:21 PM
dirty_states_DB.png (6.38 KB) dirty_states_DB.png Alena Peterová, 08/20/2019 03:21 PM
scheduled_tasks.png (54.9 KB) scheduled_tasks.png Alena Peterová, 08/20/2019 03:21 PM
lock.xlsx (7.79 KB) lock.xlsx Ondřej Kopr, 05/20/2021 05:20 PM
query.xlsx (6.28 KB) query.xlsx Ondřej Kopr, 05/20/2021 05:20 PM
003.png (66.7 KB) 003.png Ondřej Kopr, 05/20/2021 05:22 PM
002.png (68.8 KB) 002.png Ondřej Kopr, 05/20/2021 05:22 PM
001.png (40.9 KB) 001.png Ondřej Kopr, 05/20/2021 05:22 PM
004.png (128 KB) 004.png Ondřej Kopr, 05/20/2021 05:25 PM
takes.png (74 KB) takes.png Ondřej Kopr, 05/21/2021 05:28 AM
Actions
Copy link
 #1

Updated by Radek Tomiška over 5 years ago

  • Assignee changed from Radek Tomiška to Ondřej Kopr
Actions
Copy link
 #2

Updated by Alena Peterová over 5 years ago

More numbers:
4181 dirty flags after synchronization - ClearDirtyStateForContractSliceTaskExecutor run 57 minutes - commit of the big transaction took another 32 minutes.

Actions
Copy link
 #3

Updated by Ondřej Kopr over 3 years ago

The issues still remain even in new version 10.7.2.

After first initialization project was synchronize about 25k time slices. All behavior with creating new slices eavs was done about 40 minutes and then 4 hours running clear dirty state without progress information, or some informations.

From postgres activity can be obtain information about query that run from 2021-05-20 15:13:25 to 2021-05-20 19:13:54 and still running :(

More information in attached files.

Synchronization was started in [2021-05-20T14:46:25.838997+02:00[Europe/Prague]].
-------------------------
Synchronization is running as differential (entities will be updated only if least one attribute was changed).
-------------------------
Synchronization will use inner connector synchronization implementation.
-------------------------
Synchronization was correctly ended in [2021-05-20T15:28:20.933353+02:00[Europe/Prague]].

I will let the task run during night and we will see in the morning.

Actions
Copy link
 #4

Updated by Ondřej Kopr over 3 years ago

  • File deleted (004.png)
Actions
Copy link
 #5

Updated by Ondřej Kopr over 3 years ago

Actions
Copy link
 #6

Updated by Ondřej Kopr over 3 years ago

All tasks connected with time slices was executed.

Synchronization was started in [2021-05-20T14:46:25.838997+02:00[Europe/Prague]].
-------------------------
Synchronization is running as differential (entities will be updated only if least one attribute was changed).
-------------------------
Synchronization will use inner connector synchronization implementation.
-------------------------
Synchronization was correctly ended in [2021-05-20T15:28:20.933353+02:00[Europe/Prague]].
-------------------------
After success sync have to be run clear dirty state for contract slices. We start him (synchronously) now [2021-05-20T15:28:20.940337+02:00[Europe/Prague]].
-------------------------
Warning - select corrent contract slice is not executed correctly, Returned operation result is null. Ended in [2021-05-21T01:58:16.517695+02:00[Europe/Prague]].
-------------------------
After success sync have to be run HR task [HrEnableContractProcess]. We start him (synchronously) now [2021-05-21T01:58:16.790967+02:00[Europe/Prague]]. LRT ID: [aa825e47-ef6c-4b72-ba0e-9f675f284c08]
-------------------------
HR task [HrEnableContractProcess] ended in [2021-05-21T02:01:27.439763+02:00[Europe/Prague]].
-------------------------
After success sync have to be run HR task [HrEndContractProcess]. We start him (synchronously) now [2021-05-21T02:01:27.529425+02:00[Europe/Prague]]. LRT ID: [2ae6a92a-c86a-4dd5-ae10-a793f49fadfd]
-------------------------
HR task [HrEndContractProcess] ended in [2021-05-21T02:02:42.103070+02:00[Europe/Prague]].
-------------------------
After success sync have to be run HR task [HrContractExclusionProcess]. We start him (synchronously) now [2021-05-21T02:02:42.157825+02:00[Europe/Prague]]. LRT ID: [35e37235-3b89-4693-b392-edfd5989038a]
-------------------------
HR task [HrContractExclusionProcess] ended in [2021-05-21T02:02:42.212361+02:00[Europe/Prague]].
-------------------------
After success sync have to recount automatic roles (by attribute and tree structure). We start recount automatic roles by attribute (synchronously) now [2021-05-21T02:02:42.236533+02:00[Europe/Prague]].
-------------------------
Recalculation of automatic roles by attributes ended in [2021-05-21T03:12:01.421944+02:00[Europe/Prague]], role requests will be processed asynchronously.
-------------------------
We start recount automatic roles by tree structure (synchronously) now [2021-05-21T03:12:01.422175+02:00[Europe/Prague]].
-------------------------
Recalculation of automatic roles by tree structure ended in [2021-05-21T03:29:24.798241+02:00[Europe/Prague]], role requests will be processed asynchronously.

Executing ClearDirtyStateForContractSliceTaskExecutor takes 12:30 hours.

Actions
Copy link
 #7

Updated by Vít Švanda over 3 years ago

  • Priority changed from Normal to High
  • Target version set to 12.0.0
Actions
Copy link
 #8

Updated by Vít Švanda over 3 years ago

  • Estimated time set to 20.00 h
Actions
Copy link
 #9

Updated by Radek Tomiška about 3 years ago

  • Target version changed from 12.0.0 to 12.1.0
Actions
Copy link
 #10

Updated by Roman Kučera about 3 years ago

  • Target version changed from 12.1.0 to 12.2.0
Actions
Copy link
 #11

Updated by Peter Štrunc almost 3 years ago

  • Target version changed from 12.2.0 to 13.0.0
Actions
Copy link
 #12

Updated by Tomáš Doischer over 2 years ago

  • Assignee deleted (Vít Švanda)
  • Priority changed from High to Normal
  • Target version deleted (13.0.0)
Actions
Copy link

Also available in: Atom PDF