Project

General

Profile

Actions

Defect #2350

closed

Cannot view active operations in provisioning queue (error in communication with server)

Added by Marcel Poul almost 4 years ago. Updated almost 3 years ago.


Description

Provisioning operations in queue are broken. Related with #2404 - delete operation is executed before update.


Files

failed_role_request.png (94.2 KB) failed_role_request.png Alena Peterová, 07/31/2020 05:29 PM
roles_removing_met_update.png (119 KB) roles_removing_met_update.png Alena Peterová, 07/31/2020 05:33 PM
provisioning_queue.png (35.6 KB) provisioning_queue.png Alena Peterová, 07/31/2020 05:53 PM

Related issues

Related to IdStory Identity Manager - Defect #2418: Automatic roles by organizations are sometimes not assigned/removed after synchronization of contractsClosedRadek Tomiška07/31/2020

Actions
Related to IdStory Identity Manager - Defect #2404: Provisioning operations from event and sync. created at the same time can be executed in wrong orderClosedRadek Tomiška07/27/2020

Actions
Related to IdStory Identity Manager - Task #468: Account management is synchronous with Role add operationClosedRadek Tomiška05/31/2017

Actions
Related to IdStory Identity Manager - Task #1555: Force delete for rolesClosedRadek Tomiška03/15/2019

Actions
Related to IdStory Identity Manager - Task #2444: Implement waiting for the completion of the LRT after all asynchronous eventsClosedRadek Tomiška08/18/2020

Actions
Related to IdStory Identity Manager - Task #2810: Bulk action: Delete active provisioning operationsClosedRadek Tomiška05/13/2021

Actions
Actions #2

Updated by Marcel Poul almost 4 years ago

  • Description updated (diff)
Actions #5

Updated by Vít Švanda almost 4 years ago

  • Target version set to 10.4.0

Do you have some UC (different from described in the #19801) for simulate this issue?

Actions #6

Updated by Radek Tomiška almost 4 years ago

This operation fails with exception?

Add please description to reproduce issue into this ticket.

Actions #7

Updated by Marcel Poul almost 4 years ago

Radek Tomiška wrote:

This operation fails with exception?

Add please description to reproduce issue into this ticket.

https://redmine.bcvsolutions.eu/issues/20426

2020-06-29 15:14:22.427 ERROR 343963015 --- [ajp-nio-127.0.0.1-8009-exec-5] e.b.i.c.e.ExceptionControllerAdvice.handle : [9559270d-1bdb-495f-bca1-2603654aa980]
org.springframework.http.converter.HttpMessageNotWritableException: Could not write JSON: Null key for a Map not allowed in JSON (use a converting NullKeySerializer?); nested exception is com.fasterxml.jac
kson.databind.JsonMappingException: Null key for a Map not allowed in JSON (use a converting NullKeySerializer?) (through reference chain: org.springframework.hateoas.PagedResources["_embedded"]->java.util
.Collections$UnmodifiableMap["provisioningOperations"]->java.util.ArrayList3->org.springframework.hateoas.Resource["content"]->eu.bcvsolutions.idm.acc.dto.SysProvisioningOperationDto["provisioningContext
"]->eu.bcvsolutions.idm.acc.domain.ProvisioningContext["accountObject"]->java.util.HashMap["roles (SET)"]->java.util.HashMap["Uživatel CzechIdM 8"]->java.util.HashMap["null"])
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:296)
at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:103)

Actions #8

Updated by Radek Tomiška almost 4 years ago

Comment above is not related to this issue. We need to know, why system entity is missing (=> referential integrity is broken).

Actions #9

Updated by Marcel Poul almost 4 years ago

then tell us, where to find the right one. The problem is, that we just do not see provisioning queue in GUI and generic error occurs "chyba komunikace se serverem". @kotynekv and @apeterova may have more information.

Actions #10

Updated by Radek Tomiška almost 4 years ago

I hope some information is in:
- application log
- in database - broken operations and their state

Actions #11

Updated by Vít Švanda almost 4 years ago

  • Target version deleted (10.4.0)
Actions #12

Updated by Alena Peterová over 3 years ago

This happened to me in another project running on 10.4.1, which has minimum of project specific implementation.

The steps to reproduce are quite complicated - see #2418. But it's in our local server - https://redmine.czechidm.com/issues/2418#note-2

I was able to reproduce it on 2 identities. They had 6 resp. 5 accounts. The role request (roles_removing_met_update.png) should delete all of the accounts, but:
  • some of the DELETE provisionings failed (this is probably more related to #2404 so I will write it there)
  • some of the DELETE provisionings were successful
  • the broken record in the provisioning queue is an UPDATE to one of the accounts which were successfully deleted
  • the role which assigns this account has two system mappings, there were errors for both of the accounts. 1st is the broken UPDATE, 2nd is failed DELETE. This could be important, or it couldn't. But in my second project where this problem occurred, the roles have also two system mappings.

The log corresponding to viewing the provisioning queue:

2020-07-31 19:23:24.657 ERROR 16260607 --- [ajp-nio-127.0.0.1-8009-exec-6] e.b.i.c.e.ExceptionControllerAdvice.handle : [3c08eaf0-fca7-44e0-ac6c-9f5cf5758bb1]
org.springframework.orm.jpa.JpaObjectRetrievalFailureException: Unable to find eu.bcvsolutions.idm.acc.entity.SysProvisioningBatch with id 737c6a3b-e7fd-4d5a-81de-82f75c16b98a; nested exception is javax.persistence.EntityNotFoundException: Unable to find eu.bcvsolutions.idm.acc.entity.SysProvisioningBatch with id 737c6a3b-e7fd-4d5a-81de-82f75c16b98a
        at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:378)
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:255)
        at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:527)
        at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:61)
        at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:242)
        at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:153)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:144)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$ExposeRepositoryInvocationInterceptor.invoke(CrudMethodMetadataPostProcessor.java:364)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:61)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
        at com.sun.proxy.$Proxy360.findAll(Unknown Source)
        at eu.bcvsolutions.idm.core.api.service.AbstractReadDtoService.findEntities(AbstractReadDtoService.java:392)
.....
        at eu.bcvsolutions.idm.acc.service.impl.DefaultSysProvisioningOperationService$$EnhancerBySpringCGLIB$$f27091ba.find(<generated>)
        at eu.bcvsolutions.idm.core.api.rest.AbstractReadDtoController.find(AbstractReadDtoController.java:288)
        at eu.bcvsolutions.idm.acc.rest.impl.SysProvisioningOperationController.find(SysProvisioningOperationController.java:118)
.....
Caused by: javax.persistence.EntityNotFoundException: Unable to find eu.bcvsolutions.idm.acc.entity.SysProvisioningBatch with id 737c6a3b-e7fd-4d5a-81de-82f75c16b98a

Actions #13

Updated by Alena Peterová over 3 years ago

  • File deleted (provisioning_queue.png)
Actions #14

Updated by Alena Peterová over 3 years ago

Actions #15

Updated by Radek Tomiška over 3 years ago

  • Status changed from New to In Progress
  • Assignee changed from Vít Švanda to Radek Tomiška
  • Target version set to 10.5.0
Actions #16

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2418: Automatic roles by organizations are sometimes not assigned/removed after synchronization of contracts added
Actions #17

Updated by Radek Tomiška over 3 years ago

  • Related to Defect #2404: Provisioning operations from event and sync. created at the same time can be executed in wrong order added
Actions #18

Updated by Radek Tomiška over 3 years ago

  • Target version deleted (10.5.0)
Actions #19

Updated by Radek Tomiška over 3 years ago

  • Tracker changed from Task to Defect
  • Description updated (diff)
  • Status changed from In Progress to New
  • Affected versions Rhyolite (9.7.0), Rhyolite (9.7.2), Rhyolite (9.7.3), Rhyolite (9.7.4), Rhyolite (9.7.5), Rhyolite (9.7.6), Rhyolite (9.7.7), Rhyolite (9.7.8), Rhyolite (9.7.9), Rhyolite (9.7.10), Rhyolite (9.7.11), Rhyolite (9.7.12), Rhyolite (9.7.13), 10.0.0, Rhyolite (9.7.14), 10.0.1, 10.1.0, Rhyolite (9.7.15), Rhyolite (9.7.16), 10.2.0, 10.0.2, 10.3.0, 10.3.1, 10.3.2, Rhyolite (9.7.17), 10.3.3, 10.4.0, 10.4.1, 10.4.2, 10.4.3, 10.4.4 added
Actions #20

Updated by Radek Tomiška over 3 years ago

This issue affects all versions since asynchronous event processing was introduced. The fix implemented in #2404 improve this behavior too, but the issue remains for delete contract (or identity) operation, when synchronous role request is executed => delete operation can be executed before asynchronous update operation - operation for update can be broken (batch or system entity is already deleted).

Actions #21

Updated by Radek Tomiška over 3 years ago

  • Affected versions Hematite (8.0.0), Jade (8.1.0), Jade (8.1.2), Jade (8.1.3), Lapis (8.2.0), Jade (8.1.4), Malachite (9.0.0), Jade (8.1.6), Jade (8.1.7), Moonstone (9.1.0), Moonstone (9.1.1), Morganite (9.2.0), Morganite (9.2.1), Morganite (9.2.2), Onyx (9.3.0), Onyx (9.3.1), Opal (9.4.0-rc.1), Onyx (9.3.2), Onyx (9.3.3), Opal (9.4.0-rc.2), Onyx (9.3.4), Opal (9.4.0), Pyrite (9.5.0), Pyrite (9.5.1), Pyrite (9.5.2), Pyrite (9.5.3), Pyrite (9.5.4), Quartz (9.6.0), Quartz (9.6.1), Quartz (9.6.2), Quartz (9.6.3), Quartz (9.6.4), Quartz (9.6.5), Pyrite (9.5.1.2), Quartz (9.6.6) added
Actions #22

Updated by Radek Tomiška over 3 years ago

  • Related to Task #468: Account management is synchronous with Role add operation added
Actions #23

Updated by Radek Tomiška almost 3 years ago

  • Related to Task #1555: Force delete for roles added
Actions #24

Updated by Radek Tomiška almost 3 years ago

  • Status changed from New to In Progress
  • Target version set to 11.1.0
Actions #25

Updated by Radek Tomiška almost 3 years ago

  • Related to Task #2444: Implement waiting for the completion of the LRT after all asynchronous events added
Actions #26

Updated by Radek Tomiška almost 3 years ago

  • Related to Task #2810: Bulk action: Delete active provisioning operations added
Actions #27

Updated by Radek Tomiška almost 3 years ago

  • Status changed from In Progress to Needs feedback
  • Assignee changed from Radek Tomiška to Ondrej Husník
  • % Done changed from 0 to 90

Identity and contract is deleted asynchronously from bulk actions => asynchronous request are executed before identity and contract is deleted => queue is not broken now.
I improved provisioning queue too - when delete operation by synchronous role request is executed by a custom module or from synchronization (it's still possible, but prohibited), then referential integrity in provisioning queue is not broken now. Invalid provisioning operation can be deleted from database only - delete from FE will be solved in ticket #2810.

Commit:
https://github.com/bcvsolutions/CzechIdMng/commit/a341dcd3a2ecebf3e62825ed684aac689a5b9038

Could you provide me a feedback, please

Actions #28

Updated by Ondrej Husník almost 3 years ago

  • Status changed from Needs feedback to Resolved
  • Assignee changed from Ondrej Husník to Radek Tomiška
  • % Done changed from 90 to 100

I tried to simulate this issue but it didn't occur at all to me. So I walked the code. I believe it will be ok now.

Actions #29

Updated by Radek Tomiška almost 3 years ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF