Project

General

Profile

Task #1386

Entity event queue - support remove running events

Added by Radek Tomiška 5 months ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Category:
Event
Target version:
Start date:
11/22/2018
Due date:
% Done:

100%


Description

Clear running event cache, when running event is deleted.
Removing event is used as standard operation.

History

#1 Updated by Radek Tomiška 5 months ago

  • Category set to Event
  • Status changed from New to In Progress

#2 Updated by Radek Tomiška 5 months ago

  • Status changed from In Progress to Needs feedback
  • Assignee changed from Radek Tomiška to Ondřej Kopr
  • % Done changed from 0 to 90

Support remove running events without server restart is implemented. Running events can be removed and event's owner is not blocked from another processing.
Note: The only way, how can be reproduced is to create some processor, which never ends - e.g. provisioning to target system without timeout (running event was removed correctly other ways).

I've improved default event configuration for the slightly speeding up event processing (~ 2x) and prevent to exhaust system resources:
  • Event executor core pool size uses CPU count + 1 as default. Was configured CPU count * 2 => too overload
    scheduler.event.executor.corePoolSize=<CPU count + 1> 
  • Set higher queue capacity - prevent to prepare events repetitively and use additional threads till maxPoolSize is exhausted.
    scheduler.event.executor.queueCapacity=1000 
  • prepared (created) events are not loaded, when the previously loaded events still running

The first two point can be configured without release is needed.

I've added:
  • new bulk action for executing account management for the selected identity - usable for testing or e.g. if some accounts are deleted manually and has to be created again. Available for system admins (SYSTEM_ADMIN permission)
  • re-save identity bulk action was hidden (will be removed in 10.x version) - new action for executing provisioning only was created.

Doc:
https://wiki.czechidm.com/devel/documentation/application_configuration/dev/backend#scheduler

Commit:
https://github.com/bcvsolutions/CzechIdMng/commit/c12d6c6dc14015f49a7840fab9fe7ae1b67a475c

Could you do a review please?

#3 Updated by Radek Tomiška 5 months ago

I've improved loading prepared events from queue for different owners. Running event owners are excluded from search => prevent to one owner with 100+ events in queue blocks events for other owner.

Commit:
https://github.com/bcvsolutions/CzechIdMng/commit/6c22bfccef776e939329b532a54f53adb065c2d4

#4 Updated by Ondřej Kopr 4 months ago

  • Status changed from Needs feedback to Resolved
  • Assignee changed from Ondřej Kopr to Radek Tomiška
  • % Done changed from 90 to 100

For testing I use same behavior as you use in test. I try remove parents events, children events and from both sides it is working as I except. Simulate this problem is really hard (event simulate this in tests).

Also I tests some another magic with events, eq: generate hundrets of events and then remove all and etc.

Behavior with clean cache works correctly now.

#5 Updated by Radek Tomiška 4 months ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF