Project

General

Profile

Actions

Defect #2605

closed

Automatic role by attribute generates duplicate role requests and assignes subroles directly

Added by Vladimír Kotýnek over 3 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
High
Assignee:
Radek Tomiška
Category:
Automatic roles
Target version:
Start date:
12/09/2020
Due date:
% Done:

100%

Estimated time:
Affected versions:
Owner:

Description

I have role "FileSystemGroup_auto" which has a subrole "FileSystemGroup". The role "FileSystemGroup" has another super roles (200). The "FileSystemGroup_auto" role has no other super or sub roles. The "FileSystemGroup" has no subrole at all.

I have created an automatic role by attribute that assignes "FileSystemGroup_auto" with rules:
  • EAV of contract Attribute1 - value equals "ABC"
  • EAV of contract Attribute2 - value equals "Exampleíř 123" (contains these chars of Czech alphabet, a space and three numbers - it should represent a name and number)

Attribute1 and Attribute2 are single-value shortText attributes from the main definition of IdmIdentityContract.

Over 700 identities fulfill the conditions for the automatic role role.
LRT assigning the role processed more then 1.5 million items and created about 1.5 million role requests and events before being canceled.

Identities for whom the requests were executed has the role "FileSystemGroup_auto" assigned once as an automatic role. But they also have the "FileSystemGroup" assigned directly in some cases several hundred times (500x, 200x, 150x...).

@sourek @kopro Please add more details from debug

CzechIdM 10.6.3 runs on PostgreSQL database, version: PostgreSQL 9.6.19 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit


Files

3.png (98.2 KB) 3.png Ondřej Kopr, 12/10/2020 07:31 AM
4.png (138 KB) 4.png Ondřej Kopr, 12/10/2020 07:31 AM
2.png (164 KB) 2.png Ondřej Kopr, 12/10/2020 07:31 AM
1.png (173 KB) 1.png Ondřej Kopr, 12/10/2020 07:31 AM

Related issues

Related to IdStory Identity Manager - Task #1636: Redesign business roles assignmentClosedRadek Tomiška05/06/2019

Actions
Related to IdStory Identity Manager - Defect #2404: Provisioning operations from event and sync. created at the same time can be executed in wrong orderClosedRadek Tomiška07/27/2020

Actions
Related to IdStory Identity Manager - Defect #2495: Duplicit automatic roles given by two events and asynchronous role request in same timeClosedRadek Tomiška09/17/2020

Actions
Related to IdStory Identity Manager - Defect #2637: LRT: IdentityRoleExpirationTaskExecutor process duplicate assigned roles (duplicate role requests are created in the cycle)ClosedRadek Tomiška01/11/2021

Actions
Actions #1

Updated by Alena Peterová over 3 years ago

Tested also on "clean" product 10.6.3:
  • 863 identities with the same contract EAV value. All identities have 2 contracts, only one of them has the EAV filled with value.
  • 1 superrole + 1 subrole (the subrole has no other superior roles)
  • added automatic role for the superrole, rules: 1 contract EAV equals "testvalue"
  • result: all identities have the superrole assigned once and the subrole once as subrole (OK). But most of them have the subrole also directly assigned, some once, some more than once:
    select pocet as pocetprideleni, count(*) as pocetidentit from (SELECT count(*) pocet, i.username  FROM idm_identity i
    JOIN idm_identity_contract ic ON ic.identity_id=i.id 
    JOIN idm_identity_role ir ON ir.identity_contract_id=ic.id 
    JOIN idm_role r ON ir.role_id=r.id where r.code='subrole' and ir.direct_role_id is null and ir.created < '2020-12-09 18:39:00.000' group by i.username ) as t group by pocet order by pocet; 
     pocetprideleni | pocetidentit 
    ----------------+--------------
                  1 |          214
                  2 |          196
                  3 |          133
                  4 |          176
                  5 |           82
    

    I tested also situations where the automatic role was not a business role and there were no duplicities (tested for 1 or 2 EAVs of identity or contract).
    In all cases, the "Processed" field of the LRT ProcessAutomaticRoleByAttributeTaskExecutor shows numbers like 2982 / 863, but only business roles created duplicities.

Also when I added second superrole to the subrole and created automatic role for this second superrole, the number of direct assignments of "subrole" increased very much:

# select pocet as pocetprideleni, count(*) as pocetidentit from (SELECT count(*) pocet, i.username  FROM idm_identity i
JOIN idm_identity_contract ic ON ic.identity_id=i.id 
JOIN idm_identity_role ir ON ir.identity_contract_id=ic.id 
JOIN idm_role r ON ir.role_id=r.id where r.code='subrole' and ir.direct_role_id is null  group by i.username ) as t group by pocet order by pocet; 
pocetprideleni | pocetidentit 
----------------+--------------
              1 |            1
              2 |          167
              3 |           54
              4 |          140
              5 |           59
              6 |          107
              7 |           23
              8 |           51
              9 |           98
             10 |           20
             11 |           81

Actions #2

Updated by Alena Peterová over 3 years ago

It looks like the LRT ProcessAllAutomaticRoleByAttributeTaskExecutor has the same issue. With the settings from above:
  • synchronized new 228 identities and contracts
  • started ProcessAllAutomaticRoleByAttributeTaskExecutor
  • it processed 7/7 automatic roles by attributes
  • 8/228 ended with only subrole, other 220 identities had the subrole assigned directly and some more than once
    pocetprideleni | pocetidentit
    ----------------+--------------
    1 | 20
    4 | 58
    5 | 10
    6 | 52
    7 | 71
    8 | 9
Actions #3

Updated by Radek Tomiška over 3 years ago

  • Status changed from New to In Progress
Actions #4

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2567: Automatic role by EAV is not assigned when setting the EAV together with entity (e.g. through form projection) added
Actions #5

Updated by Ondřej Kopr over 3 years ago

After some small debugging and analyze the issue we had suspicion with ProcessAllAutomaticRoleByAttributeTaskExecutor with code

automaticRoleAttributeService.getContractsForAutomaticRole(automaticRoleId, true, defaultPageRequest);

The pagination is there reused:

defaultPageRequest

But in ticket #1710 is described why we do this. So reusing same first pagination is correct.

On current another project older (LTS 9.7.x) we doesn't have issues with duplication, but business role has different structure than the project from Vlada`s project.

In attachment log.txt is log from yesterday when was duplicated created - in RM must be added only 5MB :(.

  • Every created duplicit business role has same transactionId,
  • duplicit role was assigned even the original LRT was canceled - event was created,
  • after IdM restart was continued with events processing.

There are some screen from environment, if you will need some more glue we can consult all directly on project. Sorry for czech language in screens

Actions #6

Updated by Ondřej Kopr over 3 years ago

  • File deleted (log.txt)
Actions #7

Updated by Radek Tomiška over 3 years ago

  • Related to Task #1636: Redesign business roles assignment added
Actions #8

Updated by Radek Tomiška over 3 years ago

  • Related to deleted (Defect #2567: Automatic role by EAV is not assigned when setting the EAV together with entity (e.g. through form projection))
Actions #9

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2404: Provisioning operations from event and sync. created at the same time can be executed in wrong order added
Actions #10

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2495: Duplicit automatic roles given by two events and asynchronous role request in same time added
Actions #12

Updated by Radek Tomiška over 3 years ago

  • Status changed from In Progress to Needs feedback
  • Assignee changed from Radek Tomiška to Vít Švanda
  • Target version set to 10.6.4
  • % Done changed from 0 to 90
  • Affected versions 10.4.4, 10.6.0 added
  • Affected versions deleted (10.6.3)

Issue is related to asynchronous role requests added in version 10.4.4 (#2404) - duplicated role request is created, when automatic role by attribute will be newly assigned to more than 100 identities => pagination is used wrongly in LRTs since asynchronous role requests, because roles are not assigned immediately, which is required internally for evaluate automatic roles by attributes.
This issue was "hidden" by #2495, when duplicatelly created request (~concept) for automatic roles is canceled (~executed) automatically (which is in log above "Request change in concept [{0}], was not executed, because requested automatic role was already assigned (not from this role request)!"). But directly assigned automatic role is canceled automatically only, so when business roles are assigned by the same role request (since 10.6.0 (#1636)), then sub role is assigned duplicatelly.

I removed pagination used in both LRTs, commit:
https://github.com/bcvsolutions/CzechIdMng/commit/b4f925d20aad741cbcba27440f35b9aeaec73fd3

Now duplicate role requests are NOT created.

Could you provide me a feedback, please?

Actions #13

Updated by Vít Švanda over 3 years ago

  • Status changed from Needs feedback to Resolved
  • Assignee changed from Vít Švanda to Radek Tomiška
  • % Done changed from 90 to 100

I did review and tested it for 310 identities. No redundant role-requests were created now. Thanks for this fix, I like a load all UUIDs by one select instead using a iteration by page (this is a hell).

Actions #14

Updated by Radek Tomiška over 3 years ago

  • Subject changed from Automatic role by Contract EAVs generates too many roleRequests and assignes subroles directly to Automatic role by attribute generates duplicate role requests and assignes subroles directly
  • Status changed from Resolved to Closed

Version 10.6.4 released:
https://github.com/bcvsolutions/CzechIdMng/releases/tag/10.6.4

Fix merged into develop.

Actions #15

Updated by Radek Tomiška over 3 years ago

  • Affected versions 10.6.1, 10.6.2, 10.6.3 added
Actions #16

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2637: LRT: IdentityRoleExpirationTaskExecutor process duplicate assigned roles (duplicate role requests are created in the cycle) added
Actions

Also available in: Atom PDF