Defect #1104
closed
Synchronization with MISSING ACCOUNT is running after stopping it
100%
Description
Affected version: 8.0.0.
When I stop the synchronization during resolving MISSING_ACCOUNT, it continues processing all missing accounts. It logs them duplicate in the "Unknown state" and "Missing account" state, please see the screenshot. The task is still visible in the Task Scheduler and even if I terminate the task, it is still processing the accounts. Only the field "Finished" and "Running" is changed.
If I stop synchronization during other states, the stopping works fine.
This is very uncomfortable and potentially dangerous. If the action was "delete entity", it would continue deleting the entities and the only solution would be to kill Tomcat.
This is probably a new bug, stopping in the version 7.8.5 worked well (there was only 1 record in the Unknown state after stopping sync.
Files
Updated by Alena Peterová almost 6 years ago
Current develop is also affected.
Also, please check the starting of the Autorole recalculation in the synchronization of identities. If recalculation is enabled and the synchronization is stopped manually, the Autorole recalculation is started. In the synchronization of contracts, it isn't started in such situation.
Updated by Alena Peterová almost 6 years ago
Sorry, please ignore the part about Autorole recalculation in the last comment (my mistake with custom filter).
Updated by Alena Peterová almost 6 years ago
Some strange magic here.
Version 7.8.5., I canceled a very slow synchronization (without caching) of tree nodes during resolving Linked operation. It continued processing 6 accounts in 3 minutes, then I lost my patience and restarted the Tomcat.
Then I started the synchronization with attribute caching. After some time I canceled it. It processed some more 25 linked accounts in 10 seconds and then stopped correctly.
Updated by Vít Švanda almost 6 years ago
- Status changed from New to Needs feedback
- Assignee changed from Vít Švanda to Alena Peterová
- % Done changed from 0 to 90
- I fixed it.
- I think this problem was here from the beginning.
https://github.com/bcvsolutions/CzechIdMng/commit/61f4bb0a5f8b739605b3c0c467990c36c4b7874a
https://github.com/bcvsolutions/CzechIdMng/commit/05fb6afcdf99bb573b8cac430cfc966a09533b52
----
From version 8 isn't autorole/HR recalculations started when user cancel sync.
Could you make the review please?
Updated by Alena Peterová almost 6 years ago
- Status changed from Needs feedback to In Progress
- Assignee changed from Alena Peterová to Vít Švanda
Canceling from "Systems -> Synchronization -> Cancel" or terminating from "Task Scheduler -> Running tasks -> Terminate" works perfect, thanks.
Canceling from "Task Scheduler -> Running tasks -> End task" does nothing with the synchronization, only sets the task as Canceled (so it's not even possible to terminate it). Even if I started the task from the Task Scheduler, it couldn't be stopped this way. Please could you make it work :), or at least disable the button?
Small typo: Realization/frontend/czechidm-acc/src/locales/en.json - "cannost"
Updated by Vít Švanda almost 6 years ago
- Status changed from In Progress to Needs feedback
- Assignee changed from Vít Švanda to Alena Peterová
- This second problem is completely different, but I agree the sync cannot be terminated via LRT.
- Next problem here is, the LRT is not updated (counter).
- I fixed this. https://github.com/bcvsolutions/CzechIdMng/commit/2c6ccd2f9af785febcbc23757ba623a5bc973ed0
Updated by Alena Peterová almost 6 years ago
- Assignee changed from Alena Peterová to Vít Švanda
Thanks, stopping the LRT now stops the sync.
For 1 project with 1 mil. users we test running multiple synchronizations on the same resource. On the version 8.0 it was possible (only once it fell with NullPointerException, but now it runs), but on current develop it isn't - the first synchronization is stopped immediately when I start the second synchronization.
Please could you look into it once more?
Updated by Vít Švanda almost 6 years ago
Running multiple synchronizations (generally any LRTs) was never tested and is not supported now. This sounds as greate new task, for next version ;) .
Updated by Marcel Poul almost 6 years ago
Vít Švanda wrote:
Running multiple synchronizations (generally any LRTs) was never tested and is not supported now. This sounds as greate new task, for next version ;) .
What Alena tried to say was that we must have this working in 8.1 the same way as in 8.0 (cannot stop a project because of this change ;-) ). Unless you improve the sync task like 5 times.
Updated by Vít Švanda almost 6 years ago
- File Selection_133.png Selection_133.png added
- File sync.png sync.png added
- As I said before, loading such large amounts of data requires a different approach (bulk update / create ...). I do not think the using the brutal force is ideal way.
- Time for sync one itme is now cca 70ms. That is acceptable to me.
- So the problem will be rather the duration of identity save (HR processes, searching of subordinates, managers ...). And generally change the approach to big data.
- I tried run two sync for same system parallel and both works corectlly. I had only issue with update LRT counter (works only for one sync) and with stoping via LRT. After stoped first sync, the secound is also stopped.
- Reason for this is simple .. the sync executor are 'singleton', but LRT executor is setts as statefull variable. But this should not be obstacle.
- I think my changes in this task does not has impact on parallel syncs.
Updated by Alena Peterová almost 6 years ago
OK, today I was able to start 2 parallel synchronizations. In few cases, the first one failed with the NullPointerException during increaseCounter. But in most cases they could be started and running (only one of them stops increasing LRT counter as you said).
What is interesting, when one of them is Reconciliation, the NullPointerException is thrown almost always. That's why I thought it doesn't work at all yesterday (I had one of them as Reconciliation locally) - sorry about misinformation.
Updated by Marcel Poul almost 6 years ago
Vít Švanda wrote:
- As I said before, loading such large amounts of data requires a different approach (bulk update / create ...). I do not think the using the brutal force is ideal way.
Agreed, but only possible now.
Updated by Vít Švanda almost 6 years ago
- Status changed from Needs feedback to Resolved
- % Done changed from 90 to 100
I created task for resolve async run more syncs for same system (#1121).
Updated by Vít Švanda almost 6 years ago
- Status changed from Resolved to Closed