Project

General

Profile

Actions

Feature #353

closed

HR processes - functional specification and design

Added by Marcel Poul about 7 years ago. Updated almost 7 years ago.

Status:
Closed
Priority:
High
Assignee:
Jan Helbich
Category:
Scheduler
Target version:
Start date:
04/03/2017
Due date:
% Done:

100%

Estimated time:
Owner:

Description

The aim of the ticket is to write down functional specification and design of HR processes. More over, the design must be reviewed by senior developer

Actions #1

Updated by Marcel Poul about 7 years ago

  • Description updated (diff)
Actions #2

Updated by Jan Helbich about 7 years ago

HR processes

Generally, the processes react to the following events:
  • a change in identity's attributes (synchronization / manual change), including contracts etc.
  • time event, i.e. role expiration
We can divide the processes as:
  • stateless - processes all input items
  • stateful - remembers processed input items and treats these accordingly
The changes alone can be described as:
  • contextless - to execute the task, only the actual value of the item is required, typically something accessible through SQL query
  • contextful - depends on the original value of the item, Idm cannot intercept the change with SQL-like query (without comparison to audit)

Contextless and stateless process is i.e. role expiration. This process does not have an internal state, because all processed items are immediately removed. Description:
1. simple db query finds all expired roles: validTill < :today
2. unassign the roles

An example of contextless and stateful process may be email notification about missing manager. The desired functionality is that the admnistrator is notified only once if identity's manager is missing:
1. find all contracts without manager
2. if the contract has not been processed yet (throughout process history), send a notification somewhere
3. save the contract as processed

An example of contextful process - home organization change.
  • impossible to intercept through SQL query
  • the information about original organization is lost after tx commit

## Design proposal

### Contextless & stateless tasks

Realized as scheduler job / event processor. If more flexibility is needed, the processor or job only starts a predefined Activiti workflow, which realizes the process.

### Contextless & stateful tasks

Typically simple HR workflows. The set of items to process is otained via SQL query. The item is processed and if it still matches the defined criteria (the SQL query), it is persisted into process history.
The process has its own queue for processed items (the history). The queue should be modifiable, ideally even through GUI, and would allow the admin to remove items from it (maily testing purposes - rerunning the process).
After each run, the queue (history) performs some kind of self cleaning - all the items the are in history but were not found in SQL criteria are removed from queue.

### Contextful tasks

Contextful tasks depend on the original value of the item (attribute). We have to define a place in the application where the change will be intercepted and passed to the HR process (fire an event).
The requirement is that adding / removing of tracked attribute must not involve changes in core BE module and must be configurable at runtime. Also it shall not be limited to the identity entity only (IdmIdentity), but generally applicable to the domain model (implementation is not need, but must be supported, at least for contracts, organizations ets.).

I thought about following places where to intercept the attribute change - I'm open to all other ideas :)

  • Dto conversion to entity level
    • support would have to be inserted into all Dto services?
  • entity setter level
    • using AOP would be nice
    • ORM can become a problem - may invoke setters

The processing itself could either run immediately after the attribute change event, or could be added to queue and processed as scheduled job (personally I think the latter one is better).

The attribute change event has to be transactional. For example if the change is intercepted on the Dto transformation level, the transaction may rollback somewhere on the lower application levels. In that case the change must not be sumbitted to the HR process.
The queues of contextful tasks will hold the items that shall be processed, therefore the exact opposite purpose then contextless queues, holding the already processed items.

Actions #4

Updated by Jan Helbich about 7 years ago

Design proposal for specific processes of standard identity's lifecycle

New working position (contract)
Implemented as contextless & stateful process, meaning we do not necessarily need to intercept the change, but can use SQL query. Statefulness means the process needs inner history of processed items.
The SQL query is something like: SELECT e FROM IdmIdentityContract e WHERE e is valid # e is valid replace with date interval on valid[from|till] fields
The process adds default roles onto given (new) contract and enables identity, in case it has been disabled.
The process starts its own Activiti workflow realizing the given task.

Working position end (end of contract)
Implemented as contextless & stateful process, meaning we do not necessarily need to intercept the change, but can use SQL query. Statefulness means the process needs inner history of processed items.
The SQL query is something like: SELECT e FROM IdmIdentityContract e WHERE e is not valid # 'e is not valid' replace with date interval on valid[from|till] fields
The process removes all roles from given (expired) contract and disables identity, in case this contract has been identity's last valid contract.
The process starts its own Activiti workflow realizing the given task.

Exclusion from active employees (contract exlusion)
Implemented as contextless & stateful process, meaning we do not necessarily need to intercept the change, but can use SQL query. Statefulness means the process needs inner history of processed items.
SQL query select all disabled / enabled contracts and processes them accordingly. It may be desirable to remove / add roles from given (exluded) contract. If there are no more valid contracts for identity, the identity is blocked.
The process starts its own Activiti workflow realizing the given task.

Actions #5

Updated by Marcel Poul about 7 years ago

There are 2 more processes that are already implemented by IdM features.

New identity - create identity and generate login - done by Synchronization feature
Employee position change - change automatic roles - done by Automatic roles feature

Actions #6

Updated by Jan Helbich almost 7 years ago

  • Assignee changed from Jan Helbich to Radek Tomiška

Basic support for stateful tasks on BE should be done, including tests. This is the bare minimum we need before implementing the processes.
On FE I've just replaced the content key in LongRunningTaskManager to new Dto embedded property name and everything seems to work just fine.
Flyway script is included.

The code is in branch jhelbich/stateful-tasks. Can you make a review, pls? Thanks :)

Actions #7

Updated by Jan Helbich almost 7 years ago

I've rebased the source branch onto current develop.

Actions #8

Updated by Jan Helbich almost 7 years ago

  • Assignee changed from Radek Tomiška to Jan Helbich

I've finished the implementation of first HR process - contract exclusion task.
The process logic is implemented mainly in workflow script, started from HrContractExclusionProcess stateful executor.

I've struggled with running workflows outside of core-impl module and in the end gave up on resolving the issue. The error I've encountered prevents Spring from starting tests and reports missing beans for ActivitiRule. Therefore so far I keep all the implementation in the core-impl module.

Actions #10

Updated by Jan Helbich almost 7 years ago

Exclusion fixed and end of exclusion implemented.
I've done some minor refactoring and added a few tests.

Actions #11

Updated by Jan Helbich almost 7 years ago

  • Assignee changed from Jan Helbich to Peter Štrunc

I've finished implementing all speficied processes and rebased my changes as one commit in jhelbich/stateful-tasks.
Documentation in progress :)

Can you please test the implementation and provide feedback? Thx

Actions #13

Updated by Jan Helbich almost 7 years ago

Last issue I'm facing right now is getting the stateful task executors out to REST. Initially I did not have them annotated as @Service or @Component, which did not make them bean candidates and therefore SchedulerManager#getSupportedTasks did not return these.
BUT after I placed the annotation in place, all my beans lack concrete type and are just "JDK Proxies", i.e. some like "com.sun.proxy.$Proxy303".
I've tried a few things to run with Spring but cannot seem to get it working - I'd appreciate any kind of advice here.

Actions #14

Updated by Jan Helbich almost 7 years ago

The JDK proxy was created because AbstractSchedulableStatefulExecutor, which is a base class for all stateful tasks, had its #process method marked as @Transactional, which I've missed.
Many thanks to Peter for pointing this out.
I've commited the changes, so can you please make code review and test the feature now, Peter?

Also after further discussion with Ondra (sorry for false accusations :)) we have come to conclusion that there is no reliable way of recognizing original default contract, which is assigned to all identities during creation. The original process implementation relied on this being the "main" contract.
I'll discuss how to handle this default contract with others and update the implementation of processes accordingly.

Doc still in progress :)

Actions #16

Updated by Peter Štrunc almost 7 years ago

I got nothing regarding the behavior of the processes. I tested it on all scenarios i could have think of and it behaves as defined in documentation. I have got some issues that are not directly associated to behavior of HR processes.
  1. Count of processed records of each process does not reflect reality. For example when running end of contract, number of all found contracts is displayed and not the actual number of processed records.
  2. When process has no records to process this information is not displayed and only "Processing" is shown in progress bar instead.
  3. Running tasks are never shown as finished even though they are already done. Clinking on "Interrupt task" or "End task" then results in message saying that the task was already finished while leaving the state as it is. Also when interrupting task, returned message misses some arguments
Actions #17

Updated by Peter Štrunc almost 7 years ago

  • Assignee changed from Peter Štrunc to Radek Tomiška

Except for previous issues I have nothing more to say regarding functionality.I also went through the code and found nothing unusual, but Radek please do proper code review since I only did it to understand the implementation.

Actions #18

Updated by Radek Tomiška almost 7 years ago

  • Category set to Scheduler
  • Status changed from New to In Progress
  • Assignee changed from Radek Tomiška to Jan Helbich
  • % Done changed from 0 to 90
I did review and some changes:
  • ad2) Yes - if no count is provided, then "Processing" is shown. We don't have any other information.
  • ad3) Repository hql query was moved to criteria api - filter for running tasks works fine now.
  • newly added entities were added to ERD.
  • i merged jhelbich/stateful-tasks to develop - i will use this nice mechanism to IdentityContractExpirationTaskExecutor refactoring :)
Remaining issues:
* referential integrity for IdmProcessedTaskItem was added to IdmLongRunningTaskService and IdmScheduledTaskService too, this is nice. Maybe references in IdmProcessedTaskItem should be required too? If yes - add please new change script and change ERD appropriately .) I see a point now - only one of refenence could be filled, so its ok.
  • add1) Add please new ticket for this.
  • branch jhelbich/stateful-tasks could be removed, if nothing else is needed here?
  • what remains for ticket #335? Could i make a review and could be merged to develop as well ?
Actions #19

Updated by Jan Helbich almost 7 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100

Thanks for feedback!

I've added more precise description to executed workflows after consultations with L.
I've deleted the branch. #335 should have BE completely done, but didnt even started with FE, busybusybusy...
Counters were fixed in separate ticket -> I'm marking this one as resolved.

Actions #20

Updated by Jan Helbich almost 7 years ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF