Project

General

Profile

Actions

Defect #2404

closed

Provisioning operations from event and sync. created at the same time can be executed in wrong order

Added by Vladimír Kotýnek almost 4 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Radek Tomiška
Category:
Provisioning
Target version:
Start date:
07/27/2020
Due date:
% Done:

100%

Estimated time:
Affected versions:
Owner:

Description

Provisioning created by synchronization preceded the one created by a regular event.
On my project I have a source system for identities which is synchronized every 2 minutes for changes. I also have an external application which updates identities via SCIM module.
SCIM module created an event (contract EAV had been changed) and at almost the same time the synchronization processed the same identity.
Both the event and the synchronization performed provisioning.
The provisioning operation from the event was created at 07:28:35 and executed (archived) at 7:28:46
The provisioning operation from the synchronization was created at 07:28:36 and executed (archived) at 07:28:38

The provisioning operation from the synchronization was created 1 second later but was executed earlier than the one from the event.

But the result of this situation is that the provisioning operation from the event worked with older data and overwrote the account that was previously updated by the synchronization (effectively removed a role).

Expected behavior: I would expect the provisioning operation from the synchronization to be put into the provisioning queue instead of being executed.


Related issues

Related to IdStory Identity Manager - Defect #2418: Automatic roles by organizations are sometimes not assigned/removed after synchronization of contractsClosedRadek Tomiška07/31/2020

Actions
Related to IdStory Identity Manager - Defect #2350: Cannot view active operations in provisioning queue (error in communication with server)ClosedRadek Tomiška06/29/2020

Actions
Related to IdStory Identity Manager - Defect #2495: Duplicit automatic roles given by two events and asynchronous role request in same timeClosedRadek Tomiška09/17/2020

Actions
Related to IdStory Identity Manager - Defect #2497: Automatic roles: Automatic role by tree node removes properly assigned rolesClosedRadek Tomiška09/21/2020

Actions
Related to IdStory Identity Manager - Task #2444: Implement waiting for the completion of the LRT after all asynchronous eventsClosedRadek Tomiška08/18/2020

Actions
Related to IdStory Identity Manager - Defect #2605: Automatic role by attribute generates duplicate role requests and assignes subroles directlyClosedRadek Tomiška12/09/2020

Actions
Actions #2

Updated by Vít Švanda almost 4 years ago

  • Target version set to 10.5.0
Actions #3

Updated by Vít Švanda almost 4 years ago

  • Status changed from New to In Progress
Actions #4

Updated by Vít Švanda almost 4 years ago

I tested many scenarios, but I am not able simulated this on the product version. I will continue with this (note: maybe immediately priority is using in the project somewhere).

Actions #5

Updated by Vít Švanda almost 4 years ago

Conclusion:

I was not able to simulate it on the default product because all provisioning operations are performed asynchronously by default. The event queue determines that the order of provisioning operations is correct (according to the creation date).
I was only able to simulate this problem if I use synchronous provisioning (an event with imidiately priority). In this case, the synchronous event does not persist in the event queue and causes the execution in different order.

For prevent this we need to ensure some async queue. The simplest solution is enable async provisioning on the system. Problem here is this mechanism is not use anymore on projects (replaced by event queue) and maybe will cause perfomance problem, because executing of provisioning queue is processed in one thread.

You can try enable async privisioning on your system a tested performance. Async provisioning can be improved in the product, but this is big change (maybe to the version 10.6.0).

Actions #6

Updated by Vít Švanda almost 4 years ago

  • Status changed from In Progress to Needs feedback
  • Assignee changed from Vít Švanda to Vladimír Kotýnek
Actions #7

Updated by Alena Peterová almost 4 years ago

I think I met the same behaviour in our project which has minimum project specific implementation. It was probably caused by ProcessSkippedAutomaticRoleByTreeForContractTaskExecutor which run just after the synchronization of contracts and so they operated on the same identity.
All details are in #2418 (the second part of the ticket description).

Actions #8

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2418: Automatic roles by organizations are sometimes not assigned/removed after synchronization of contracts added
Actions #9

Updated by Radek Tomiška over 3 years ago

  • Status changed from Needs feedback to In Progress
  • Assignee changed from Vladimír Kotýnek to Radek Tomiška
Actions #10

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2350: Cannot view active operations in provisioning queue (error in communication with server) added
Actions #11

Updated by Radek Tomiška over 3 years ago

  • Status changed from In Progress to Needs feedback
  • Assignee changed from Radek Tomiška to Vít Švanda
  • Affected versions 10.3.0, 10.3.1, 10.3.3, 10.4.0 added

Mixin synchronous and asynchronous processing should be avioded - role requests created from product was switched to asynchronous => event queue will preserve correct operation order.
If you are creating role requests or assign roles directly (without provisioning is skipped), then use asynchronous request too in your modules.

Commit:
https://github.com/bcvsolutions/CzechIdMng/commit/7f15dd0fd145fc626f508e1d6745755b0541523f

Could you provide me a feedback, please?

Note: All product versions are affected propably, but this issue is more visible after automatic roles by tree structure was redesigned to synchronous role request processing.

Actions #12

Updated by Alena Peterová over 3 years ago

A different situation would be probably solved by the previous change: Adding 2 automatic roles to the same organization collided and one of the roles wasn't assigned to the user sitting on the organization.

Please consider putting this fix in 10.4.4, is it possible?


In my project, we create the automatic roles programatically and the collision happened in cca 10 % of the automatic roles. We didn't notice it at first, because the error is visible only in the result of the LRT ProcessAutomaticRoleByTreeTaskExecutor. The LRT has the result Executed, only one of the item has Failed state:
AUTOMATIC_ROLE_ASSIGN_TASK_NOT_COMPLETE
org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [null];
Cannot insert duplicate key row in object 'bcv_idm_storage.acc_account' with unique index 'ux_account_uid'. The duplicate key value is (1237554366, 0xe19d97e4d4fe4079b43f517393a59280).

The inconsistent state could be repaired by running ProcessAllAutomaticRoleByTreeTaskExecutor - it processed 1031 automatic roles in 7 minutes and assigned 111 roles, which were missing due to the previous collisions.
If the fix wasn't present in 10.4.4, we would have to run this task periodically. It seems to be quick, but we will have 10k automatic roles in the end, so it isn't ideal.

Actions #13

Updated by Radek Tomiška over 3 years ago

  • Target version changed from 10.5.0 to 10.4.4
  • % Done changed from 0 to 90

Moved into 10.4.4

Actions #14

Updated by Vít Švanda over 3 years ago

  • Status changed from Needs feedback to Resolved
  • Assignee changed from Vít Švanda to Radek Tomiška
  • % Done changed from 90 to 100

I did review and tested it.

I think the execute a role requests (provisioning) asynchronosly will fix our current problem. This solution can potencialy cause problem with a dependecy LRT task starting after LRT recalculating automatic roles. In this case will be LRT ended befor a role-request will be executed (identity-role will be not assigned). After consolutions with others, no of our projects depending on this behavior. So this solution is correct now.

This potencial problem can solve new feature #2444 (implement waiting for the completion of the LRT after all asynchronous events).

Thanks for this fix.

Actions #15

Updated by Radek Tomiška over 3 years ago

  • Status changed from Resolved to Closed
Actions #16

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2495: Duplicit automatic roles given by two events and asynchronous role request in same time added
Actions #17

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2497: Automatic roles: Automatic role by tree node removes properly assigned roles added
Actions #18

Updated by Radek Tomiška over 3 years ago

  • Related to Task #2444: Implement waiting for the completion of the LRT after all asynchronous events added
Actions #19

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2605: Automatic role by attribute generates duplicate role requests and assignes subroles directly added
Actions

Also available in: Atom PDF