Project

General

Profile

Actions

Defect #2312

closed

Import of several virtual systems at once causes "Too many open files" errors and unavailability of IdM

Added by Vladimír Kotýnek almost 4 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Vít Švanda
Category:
Export Import
Target version:
Start date:
06/09/2020
Due date:
% Done:

100%

Estimated time:
Affected versions:
Owner:

Description

@10.3.2

I have an export .zip file with 38 virtual systems. After I uploaded the file to Export/Import site in IDM I run "dry run" or "demo mode" - the blue triangle. After the demo mode run started the application was unavailable with error:

<ResultModels>
<_errors>
<_errors>
<id>480f5db2-d836-4793-9796-339e88148038</id>
<creation>2020-06-09T14:06:36.879+02:00</creation>
<statusEnum>INTERNAL_SERVER_ERROR</statusEnum>
<message>java.io.FileNotFoundException: /opt/tomcat/apache-tomcat-8.5.47/webapps/idm/WEB-INF/lib/junit-jupiter-api-5.3.2.jar (Too many open files)</message>
<parameters/>
<module>core</module>
<statusCode>500</statusCode>
<status>INTERNAL_SERVER_ERROR</status>
</_errors>
</_errors>
</ResultModels>

In the catalina.out log I can see a lot of "Too many open files" exceptions. The demo mode run itself failded with following error:

2020-06-09 14:05:47.157  INFO 328262572 --- [base-task-executor-4] e.b.i.c.m.s.i.DefaultImportManager.internalExecuteImport : Internal import [eu.bcvsolutions.idm.core.api.dto.IdmExportImportDto [id=73daa6
b8-67b1-4d01-a617-315b1ac97c13], dry-run: true] ended
2020-06-09 14:05:47.169 ERROR 328262584 --- [base-task-executor-4] e.b.i.c.s.a.s.AbstractLongRunningTaskExecutor.end : [core:IMPORT_ZIP_EXTRACTION_FAILED:cbce0d75-5015-470b-8488-79dc92b88fc5] Extraction of
 ZIP import failed! ({})
eu.bcvsolutions.idm.core.api.exception.ResultCodeException: Extraction of ZIP import failed!
        at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.getParentDtoFromBatch(DefaultImportManager.java:853)
        at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.lambda$executeImportForType$5(DefaultImportManager.java:279)
        at java.util.ArrayList.forEach(ArrayList.java:1257)
        at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.executeImportForType(DefaultImportManager.java:273)
        at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.lambda$internalExecuteImport$1(DefaultImportManager.java:221)
        at java.util.ArrayList.forEach(ArrayList.java:1257)
        at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.internalExecuteImport(DefaultImportManager.java:220)
        at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager$$FastClassBySpringCGLIB$$6e60f64f.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
        at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager$$EnhancerBySpringCGLIB$$87d71c5b.internalExecuteImport(<generated>)
        at eu.bcvsolutions.idm.core.scheduler.task.impl.ImportTaskExecutor.process(ImportTaskExecutor.java:69)
        at eu.bcvsolutions.idm.core.scheduler.task.impl.ImportTaskExecutor.process(ImportTaskExecutor.java:32)
        at eu.bcvsolutions.idm.core.scheduler.api.service.AbstractLongRunningTaskExecutor.call(AbstractLongRunningTaskExecutor.java:212)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.springframework.security.concurrent.DelegatingSecurityContextRunnable.run(DelegatingSecurityContextRunnable.java:84)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.FileSystemException: /opt/czechidm/data/temp/3580194995400728524/SysRoleSystemDto: Too many open files
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
        at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427)
        at java.nio.file.Files.newDirectoryStream(Files.java:457)
        at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:300)
        at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
        at java.nio.file.FileTreeIterator.<init>(FileTreeIterator.java:72)
        at java.nio.file.Files.walk(Files.java:3574)
        at java.nio.file.Files.walk(Files.java:3625)
        at eu.bcvsolutions.idm.core.model.service.impl.DefaultImportManager.getParentDtoFromBatch(DefaultImportManager.java:842)
        ... 23 common frames omitted

The imported file contains:

   15 files in directory .
   38 files in directory ./SysSyncIdentityConfigDto
  514 files in directory ./SysSystemAttributeMappingDto
    4 files in directory ./IdmFormDefinitionDto
    1 files in directory ./IdmExportImportDto
    3 files in directory ./IdmFormInstanceDto
   38 files in directory ./SysSystemDto
   76 files in directory ./SysSystemMappingDto
   38 files in directory ./SysSchemaObjectClassDto
    1 files in directory ./SysProvisioningBreakConfigDto
    1 files in directory ./SysProvisioningBreakRecipientDto
   15 files in directory ./IdmFormAttributeDto
  526 files in directory ./SysRoleSystemDto
  482 files in directory ./SysRoleSystemAttributeDto
  478 files in directory ./SysSchemaAttributeDto

2213 files in total

Actions

Also available in: Atom PDF