Actions
Defect #2312
closedImport of several virtual systems at once causes "Too many open files" errors and unavailability of IdM
Start date:
06/09/2020
Due date:
% Done:
100%
Estimated time:
Affected versions:
Owner:
Description
@10.3.2
I have an export .zip file with 38 virtual systems. After I uploaded the file to Export/Import site in IDM I run "dry run" or "demo mode" - the blue triangle. After the demo mode run started the application was unavailable with error:
<ResultModels>
<_errors>
<_errors>
<id>480f5db2-d836-4793-9796-339e88148038</id>
<creation>2020-06-09T14:06:36.879+02:00</creation>
<statusEnum>INTERNAL_SERVER_ERROR</statusEnum>
<message>java.io.FileNotFoundException: /opt/tomcat/apache-tomcat-8.5.47/webapps/idm/WEB-INF/lib/junit-jupiter-api-5.3.2.jar (Too many open files)</message>
<parameters/>
<module>core</module>
<statusCode>500</statusCode>
<status>INTERNAL_SERVER_ERROR</status>
</_errors>
</_errors>
</ResultModels>
In the catalina.out log I can see a lot of "Too many open files" exceptions. The demo mode run itself failded with following error:
2020-06-09 14:05:47.157 INFO 328262572 --- [base-task-executor-4] e.b.i.c.m.s.i.DefaultImportManager.internalExecuteImport : Internal import [eu.bcvsolutions.idm.core.api.dto.IdmExportImportDto [id=73daa6 b8-67b1-4d01-a617-315b1ac97c13], dry-run: true] ended 2020-06-09 14:05:47.169 ERROR 328262584 --- [base-task-executor-4] e.b.i.c.s.a.s.AbstractLongRunningTaskExecutor.end : [core:IMPORT_ZIP_EXTRACTION_FAILED:cbce0d75-5015-470b-8488-79dc92b88fc5] Extraction of ZIP import failed! ({}) eu.bcvsolutions.idm.core.api.exception.ResultCodeException: Extraction of ZIP import failed! at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.getParentDtoFromBatch(DefaultImportManager.java:853) at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.lambda$executeImportForType$5(DefaultImportManager.java:279) at java.util.ArrayList.forEach(ArrayList.java:1257) at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.executeImportForType(DefaultImportManager.java:273) at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.lambda$internalExecuteImport$1(DefaultImportManager.java:221) at java.util.ArrayList.forEach(ArrayList.java:1257) at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.internalExecuteImport(DefaultImportManager.java:220) at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager$$FastClassBySpringCGLIB$$6e60f64f.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager$$EnhancerBySpringCGLIB$$87d71c5b.internalExecuteImport(<generated>) at eu.bcvsolutions.idm.core.scheduler.task.impl.ImportTaskExecutor.process(ImportTaskExecutor.java:69) at eu.bcvsolutions.idm.core.scheduler.task.impl.ImportTaskExecutor.process(ImportTaskExecutor.java:32) at eu.bcvsolutions.idm.core.scheduler.api.service.AbstractLongRunningTaskExecutor.call(AbstractLongRunningTaskExecutor.java:212) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.springframework.security.concurrent.DelegatingSecurityContextRunnable.run(DelegatingSecurityContextRunnable.java:84) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.nio.file.FileSystemException: /opt/czechidm/data/temp/3580194995400728524/SysRoleSystemDto: Too many open files at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427) at java.nio.file.Files.newDirectoryStream(Files.java:457) at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:300) at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322) at java.nio.file.FileTreeIterator.<init>(FileTreeIterator.java:72) at java.nio.file.Files.walk(Files.java:3574) at java.nio.file.Files.walk(Files.java:3625) at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.getParentDtoFromBatch(DefaultImportManager.java:842) ... 23 common frames omitted
The imported file contains:
15 files in directory . 38 files in directory ./SysSyncIdentityConfigDto 514 files in directory ./SysSystemAttributeMappingDto 4 files in directory ./IdmFormDefinitionDto 1 files in directory ./IdmExportImportDto 3 files in directory ./IdmFormInstanceDto 38 files in directory ./SysSystemDto 76 files in directory ./SysSystemMappingDto 38 files in directory ./SysSchemaObjectClassDto 1 files in directory ./SysProvisioningBreakConfigDto 1 files in directory ./SysProvisioningBreakRecipientDto 15 files in directory ./IdmFormAttributeDto 526 files in directory ./SysRoleSystemDto 482 files in directory ./SysRoleSystemAttributeDto 478 files in directory ./SysSchemaAttributeDto
2213 files in total
Actions