Task #2649
open
Error caused by more than 10000 groups or users in AD
Added by David Štekl over 3 years ago. Updated over 2 years ago.
40%
Description
If there are more than 10000 groups in AD and "Base contexts for group entry searches" is set for OU=COMPANY,DC=ad,DC=COMPANY,DC=cz(root OU). Synchronization from system fails on error - [LDAP: error code 12 - 000020EF: SvcErr: DSID-03140552, problem 5010 (UNAVAIL_EXTENSION), data 0]; remaining name 'OU=COMPANY,DC=ad,DC=COMPANY,DC=cz'
The same error occurs also after synchronization for more than 10,000 users [LDAP: error code 12 - 000020EF: SvcErr: DSID-03140594, problem 5010 (UNAVAIL_EXTENSION), data 0]; remaining name 'OU=COMPANY,DC=ad,DC=COMPANY,DC=cz'
For now, the error is solved as follows:Separate ldap search with "Base context for group entry searches" and divide it into smaller searches(each line with one OU):
- OU=001OU,OU=COMPANY,DC=ad,DC=COMPANY,DC=cz
- OU=002OU,OU=COMPANY,DC=ad,DC=COMPANY,DC=cz
- OU=003OU,OU=COMPANY,DC=ad,DC=COMPANY,DC=cz
Another way to solve this problem is by using "Custom group search filter" in the system configuration.
Updated by Ondřej Kopr over 3 years ago
After some consultation with AD administrator from customer we try change internal behavior with AD configuration by windows register. The changes works well for some time, but after several synchronization the error occurs again.
For us also helps AD restart. After restart AD on test the synchronization synchronize again all items. We tested this on test environment, for production environment is this useless.
When the restart helps we decided that this can be some strange behavior on AD, but who knows?
Updated by David Štekl over 3 years ago
- % Done changed from 0 to 40
I simulated the error in our test environment #2459. In the AD BCV piskoviste, I created exactly 10001 users for which the synchronization ended immediately with the following error.
For 10000 users synchronization works.
The next step is to find the reason for this error in ad connector.
Error during synchronization
-------------------------
org.identityconnectors.framework.common.exceptions.ConnectorException: javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 000020EF: SvcErr: DSID-03140552, problem 5010 (UNAVAIL_EXTENSION), data 0
]; remaining name 'OU=testUsers,DC=piskoviste,DC=bcv'
at net.tirasa.connid.bundles.ldap.search.LdapInternalSearch.execute(LdapInternalSearch.java:76)
at net.tirasa.connid.bundles.ad.search.ADSearch.executeADQuery(ADSearch.java:110)
at net.tirasa.connid.bundles.ad.ADConnector.executeQuery(ADConnector.java:137)
at net.tirasa.connid.bundles.ad.ADConnector.executeQuery(ADConnector.java:57)
at org.identityconnectors.framework.impl.api.local.operations.SearchImpl.rawSearch(SearchImpl.java:171)
at org.identityconnectors.framework.impl.api.local.operations.SearchImpl.search(SearchImpl.java:130)
at sun.reflect.GeneratedMethodAccessor1653.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.identityconnectors.framework.impl.api.local.operations.ConnectorAPIOperationRunnerProxy.invoke(ConnectorAPIOperationRunnerProxy.java:98)
at com.sun.proxy.$Proxy364.search(Unknown Source)
at sun.reflect.GeneratedMethodAccessor1653.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.identityconnectors.framework.impl.api.local.operations.ThreadClassLoaderManagerProxy.invoke(ThreadClassLoaderManagerProxy.java:96)
at com.sun.proxy.$Proxy364.search(Unknown Source)
at sun.reflect.GeneratedMethodAccessor1653.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.identityconnectors.framework.impl.api.BufferedResultsProxy$BufferedResultsHandler.run(BufferedResultsProxy.java:165)
Caused by: javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 000020EF: SvcErr: DSID-03140552, problem 5010 (UNAVAIL_EXTENSION), data 0
]; remaining name 'OU=testUsers,DC=piskoviste,DC=bcv'
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3214)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3100)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2891)
at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1846)
at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1769)
at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341)
at net.tirasa.connid.bundles.ad.search.ADVlvIndexSearchStrategy.searchBaseDN(ADVlvIndexSearchStrategy.java:152)
at net.tirasa.connid.bundles.ad.search.ADVlvIndexSearchStrategy.doSearch(ADVlvIndexSearchStrategy.java:105)
at net.tirasa.connid.bundles.ldap.search.LdapInternalSearch.execute(LdapInternalSearch.java:67)
... 19 more
ldap policy:
Policy Current(New)
MaxPoolThreads 4
MaxPercentDirSyncRequests 0
MaxDatagramRecv 4096
MaxReceiveBuffer 10485760
InitRecvTimeout 120
MaxConnections 5000
MaxConnIdleTime 900
MaxPageSize 11000
MaxBatchReturnMessages 0
MaxQueryDuration 120
MaxTempTableSize 10000
MaxResultSetSize 262144
MinResultSets 0
MaxResultSetsPerConn 0
MaxNotificationPerConn 5
MaxValRange 1500
MaxValRangeTransitive 0
ThreadMemoryLimit 0
SystemMemoryLimitPercent 0
Updated by Roman Kubica over 3 years ago
Hi @stekld I noticed you received different error codes. You have type error SvcErr and I can see we received LdapErr. To give you some extra information about this group synchronization. It runs in kind of random numbers every day. Sometimes it processes few hundred (rounded) groups and stops. Sometimes few thousands rounded to 100s and sometimes specific number like 5783 processed items. We will try to switch Domain Controller and I shall provide more information then.
LDAP: error code 12 - 000020EF: SvcErr: DSID-03140552, problem 5010 (UNAVAIL_EXTENSION), data 0 LDAP: error code 12 - 00000057: LdapErr: DSID-0C090858, comment: Error processing control, data 0, v2580]
org.identityconnectors.framework.common.exceptions.ConnectorException: javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 00000057: LdapErr: DSID-0C090858, comment: Error processing control, data 0, v2580]; remaining name 'OU=001,OU=COMPANY,DC=DOMAIN,DC=cz'
at net.tirasa.connid.bundles.ldap.search.LdapInternalSearch.execute(LdapInternalSearch.java:76)
at net.tirasa.connid.bundles.ad.search.ADSearch.executeADQuery(ADSearch.java:110)
at net.tirasa.connid.bundles.ad.ADConnector.executeQuery(ADConnector.java:137)
at net.tirasa.connid.bundles.ad.ADConnector.executeQuery(ADConnector.java:57)
at org.identityconnectors.framework.impl.api.local.operations.SearchImpl.rawSearch(SearchImpl.java:171)
at org.identityconnectors.framework.impl.api.local.operations.SearchImpl.search(SearchImpl.java:130)
at sun.reflect.GeneratedMethodAccessor2359.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.identityconnectors.framework.impl.api.local.operations.ConnectorAPIOperationRunnerProxy.invoke(ConnectorAPIOperationRunnerProxy.java:98)
at com.sun.proxy.$Proxy516.search(Unknown Source)
at sun.reflect.GeneratedMethodAccessor2359.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.identityconnectors.framework.impl.api.local.operations.ThreadClassLoaderManagerProxy.invoke(ThreadClassLoaderManagerProxy.java:96)
at com.sun.proxy.$Proxy516.search(Unknown Source)
at sun.reflect.GeneratedMethodAccessor2359.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.identityconnectors.framework.impl.api.BufferedResultsProxy$BufferedResultsHandler.run(BufferedResultsProxy.java:165)
Caused by: javax.naming.OperationNotSupportedException: [LDAP: error code 12 - 00000057: LdapErr: DSID-0C090858, comment: Error processing control, data 0, v2580]; remaining name 'OU=001,OU=COMPANY,DC=DOMAIN,DC=cz'
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3214)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3100)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2891)
at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1846)
at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1769)
at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341)
at net.tirasa.connid.bundles.ad.search.ADVlvIndexSearchStrategy.searchBaseDN(ADVlvIndexSearchStrategy.java:152)
at net.tirasa.connid.bundles.ad.search.ADVlvIndexSearchStrategy.doSearch(ADVlvIndexSearchStrategy.java:105)
at net.tirasa.connid.bundles.ldap.search.LdapInternalSearch.execute(LdapInternalSearch.java:67)
... 19 more
Updated by Roman Kučera over 2 years ago
unfortunately out AD test env with 10 000 is dead so we need to create users again in a new test AD.
However I tried to look into code of AD connector. But I was not able to see where is the issue for now.
PageSize which is configured in connector is used, but then probably is not used.
AD connector has some search strategies.
We are using ADVlvIndexSearchStrategy, but other strategy is ADPagedSearchStrategy but this paged strategy is used without vlvsort.
- Try to turn of vlv sort, because connector will use ADPagedSearchStrategy then.
- Validate if the issue is still there
- If ADPagedSearchStrategy will work, there is probably chance to implement this paging from ADPagedSearchStrategy into ADVlvIndexSearchStrategy
Updated by Roman Kučera over 2 years ago
- With vlvsort enabled = Same error
- Without vlvsort enabled = It only return the amount of items which is set in pageSize. If you use pagesize bigger then 1000 I got only 1000 So it looks even without vlvsort the paging strategy is not working.
Next try to debug behavior without vlvsort, why paging strategy is not working. When we figure it out we can maybe implement similar paging into vlvsort