Project

General

Profile

Actions

Feature #2696

closed

HA - move scheduled tasks and events to the current instance

Added by Alena Peterová about 3 years ago. Updated almost 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Radek Tomiška
Category:
Architecture
Target version:
Start date:
03/02/2021
Due date:
% Done:

100%

Estimated time:
32.00 h
Owner:

Description

This feature is proposed for the basic high availability - mode active/passive.

When an active instance (say "idm-primary") shuts down, the second node (say "idm-secondary") is started. After that, the admin needs some way to move all system jobs in IdM to the currently active instance, so it's fully functional.

System jobs: scheduled tasks, Created entity events/states, or anything else which would make sense for the use case.
It could be some button, some LRT, whatever.
In the first version, admin will do this manually. It could be more automatized in the future.


Related issues

Related to IdStory Identity Manager - Task #1367: Entity event queue - add base permissionsClosedRadek Tomiška11/08/2018

Actions
Actions #2

Updated by Radek Tomiška about 3 years ago

  • Estimated time set to 32.00 h
Actions #3

Updated by Vít Švanda about 3 years ago

  • Target version changed from 11.0.0 to 11.1.0
Actions #5

Updated by Vít Švanda about 3 years ago

  • Priority changed from High to Normal
  • Target version changed from 11.1.0 to 11.0.0
Actions #6

Updated by Vít Švanda about 3 years ago

  • Target version changed from 11.0.0 to 11.1.0
Actions #7

Updated by Radek Tomiška about 3 years ago

  • Status changed from New to In Progress
Actions #8

Updated by Radek Tomiška almost 3 years ago

  • Related to Task #1367: Entity event queue - add base permissions added
Actions #9

Updated by Radek Tomiška almost 3 years ago

  • % Done changed from 0 to 80

Feature is implemented in branch:
https://github.com/bcvsolutions/CzechIdMng/tree/rtomiska/2696-switch-instance-for-asynchronous-processing

Documentation remains - will be added after version 11.0.0 will be released.

Actions #12

Updated by Ondrej Husník almost 3 years ago

I tested your task and basically works well. I found only some details that could be improved.

1) The value of idm.pub.app.instanceId property set in ....app\src\main\resources\application.properties cannot be seen in IdM Configuration
  • There is set the Idm instance name in the property file to e.g. idm-XXX
  • If we run IdM and perform provisioning (for example ) we can see that corresponding entity event was executed on current Idm instance ie idm-XXX - setting from file works well
  • But in the Configuration page there is still displayed idm-primary.
    This can be very confusing for common users.
2) Entity event which is supposed to be executed on the instance given by overloaded value idm.sec.core.event.asynchronous.instanceId in the Idm is run on current instance
Steps to reproduce:
  • Current instace id is eg. idm.pub.app.instanceId=idm-XXX
  • Stop async event processing with idm.sec.core.event.asynchronous.stopProcessing=true
  • Set new instance for async operation execution idm.sec.core.event.asynchronous.instanceId=idm-YYY
  • Perform a kind of async operation eg provisioning. In entity event table there is a new event creted NOT run. It is planned for idm-YYY Idm instance.
  • Start async event processing with idm.sec.core.event.asynchronous.stopProcessing=false
  • Check entity event table. The newly created event is EXECUTED on current instance idm-XXX although planned for idm-YYY.
3) When idm.sec.core.event.asynchronous.instanceId set from property file.
  • It acts like situation in the point 2)
  • The value is empty in Idm Configuration setting. Value form property file is not propagated to Configuration table. Similar to the point 1)
Actions #13

Updated by Ondrej Husník almost 3 years ago

4) If Idm instance switching fails there is not obvious, what task it caused.
To reproduce:
  1. Create simple script (just return constant).
  2. Create a LRT executing this script (ExecuteScriptTaskExecutor)
  3. Delete this script.
  4. Try to reschedule Idm instance for async.

The instance change fails on the following exception. The problem is that it speaks about missing entity but you will be hardly able to find the LRT it pertains to especially in as of many scheduled tasks. I think there could be a specific exception for the task where rescheduling failed with its name. I guess there may be more situations when this may happen.

2021-06-15 12:11:45.702 ERROR 1237513 --- [base-task-executor-8] e.b.i.c.s.t.i.ExecuteScriptTaskExecutor.getScript : Script with code: [e420c71d-7103-4cf6-b24d-b9a4073e6081], not found.
2021-06-15 12:11:45.712 ERROR 1237523 --- [base-task-executor-8] e.b.i.c.a.b.a.AbstractBulkAction.processEntity : Processing of entity [d7f708b2-96f9-412b-bba7-26336ad2e166] failed.
eu.bcvsolutions.idm.core.api.exception.ResultCodeException: e420c71d-7103-4cf6-b24d-b9a4073e6081 not found.
    at eu.bcvsolutions.idm.core.scheduler.task.impl.ExecuteScriptTaskExecutor.getScript(ExecuteScriptTaskExecutor.java:75)
    at eu.bcvsolutions.idm.core.scheduler.task.impl.ExecuteScriptTaskExecutor.init(ExecuteScriptTaskExecutor.java:53)
    at eu.bcvsolutions.idm.core.scheduler.task.impl.ExecuteScriptTaskExecutor$$FastClassBySpringCGLIB$$5c3aa2b8.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:687)
    at eu.bcvsolutions.idm.core.scheduler.task.impl.ExecuteScriptTaskExecutor$$EnhancerBySpringCGLIB$$93bfee6a.init(<generated>)
    at eu.bcvsolutions.idm.core.scheduler.service.impl.DefaultSchedulerManager.updateTask(DefaultSchedulerManager.java:286)
    at eu.bcvsolutions.idm.core.scheduler.service.impl.DefaultSchedulerManager.switchInstanceId(DefaultSchedulerManager.java:461)
    at eu.bcvsolutions.idm.core.scheduler.service.impl.DefaultSchedulerManager$$FastClassBySpringCGLIB$$c2ee60b9.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:752)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
    at eu.bcvsolutions.idm.core.scheduler.service.impl.DefaultSchedulerManager$$EnhancerBySpringCGLIB$$92bd710d.switchInstanceId(<generated>)
    at eu.bcvsolutions.idm.core.bulk.action.impl.configuration.ConfigurationSwitchInstanceBulkAction.processDto(ConfigurationSwitchInstanceBulkAction.java:195)
    at eu.bcvsolutions.idm.core.bulk.action.impl.configuration.ConfigurationSwitchInstanceBulkAction.processDto(ConfigurationSwitchInstanceBulkAction.java:1)
    at eu.bcvsolutions.idm.core.api.bulk.action.AbstractBulkAction.processEntity(AbstractBulkAction.java:429)
    at eu.bcvsolutions.idm.core.api.bulk.action.AbstractBulkAction.processEntities(AbstractBulkAction.java:408)
    at eu.bcvsolutions.idm.core.api.bulk.action.AbstractBulkAction.internalProcess(AbstractBulkAction.java:316)
    at eu.bcvsolutions.idm.core.api.bulk.action.AbstractBulkAction.process(AbstractBulkAction.java:237)
    at eu.bcvsolutions.idm.core.api.bulk.action.AbstractBulkAction.process(AbstractBulkAction.java:1)
    at eu.bcvsolutions.idm.core.scheduler.api.service.AbstractLongRunningTaskExecutor.call(AbstractLongRunningTaskExecutor.java:269)
    at eu.bcvsolutions.idm.core.scheduler.api.service.AbstractLongRunningTaskExecutor$$FastClassBySpringCGLIB$$f9eae371.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:687)
    at eu.bcvsolutions.idm.core.bulk.action.impl.configuration.ConfigurationSwitchInstanceBulkAction$$EnhancerBySpringCGLIB$$184e4351.call(<generated>)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at eu.bcvsolutions.idm.core.config.DelegatingTransactionContextRunnable.run(DelegatingTransactionContextRunnable.java:39)
    at org.springframework.security.concurrent.DelegatingSecurityContextRunnable.run(DelegatingSecurityContextRunnable.java:84)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)


Added also small cleanup.
https://github.com/bcvsolutions/CzechIdMng/commit/7f73c40aec7c2488f0948fe4c5af00266c314a8a

Actions #14

Updated by Ondrej Husník almost 3 years ago

  • Status changed from Needs feedback to In Progress
  • Assignee changed from Ondrej Husník to Radek Tomiška
Actions #15

Updated by Radek Tomiška almost 3 years ago

  • Status changed from In Progress to Needs feedback
  • Assignee changed from Radek Tomiška to Ondrej Husník

Thx for feedback!
ad2) Fixed wrong configuration property, used for asynchronous processing.
ad4) Validations are ignored, when scheduled tasks are moved into different instance (it's not related to switch operation => task configuration should be fixed, when task is scheduled / edited).

Commit:
https://github.com/bcvsolutions/CzechIdMng/commit/86f313127fe04071ca305a3cc32939d7dea59b83

Other notes are not reproducible. Could you provide me a feedback again, please?

Actions #16

Updated by Ondrej Husník almost 3 years ago

  • Status changed from Needs feedback to Resolved
  • Assignee changed from Ondrej Husník to Radek Tomiška
  • % Done changed from 90 to 100

Thanks for fixes. Both fixed points works correctly now. The irreproducible ones were probably caused by wrong file refresh on my side.

Actions #17

Updated by Radek Tomiška almost 3 years ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF