Task #1357
openCreate Export system Bulk-Action
60%
Description
- Create a bulk action which will export all entities from the selected system.
Technical design
- Bulk action will extends from AbstractBulkAction<SysSystemDto,SysSystemFilter>
- It will get system id/ids and from that particular system/systems it will export all entities
- Exported data will be saved into CSV file
- User will provide directory, where CSV files will be stored
- Name of CSV will consist of system name and time stamp - this gives us the opportunity to export more systems and also to recognize current export by timestamp
- Next get the configuration of the connector
- Connect to the system and get all entities
- Write entities to file chosen and creating header while doing so
- Header cannot be created before all items are processed - some items have more properties than others in connector object
Related issues
Updated by Marek Klement about 6 years ago
- Related to Task #1230: Virtual system - make first snapshot of data added
Updated by Marek Klement about 6 years ago
- Related to Task #1286: Import entities to system added
Updated by Marek Klement about 6 years ago
- Status changed from New to In Progress
- created class ExportSystemBulkAction in acc module
- Trying to get directory path from frontend
Updated by Marek Klement about 6 years ago
Here is also link to original task: https://wiki.czechidm.com/priv/navrh/import_do_vs#export_uzivatelu_ze_systemu
Updated by Marek Klement about 6 years ago
- now I am able to get the string with directory and dto of system
- will continue with the logic of export -
Updated by Marek Klement about 6 years ago
- % Done changed from 0 to 30
First export from the system achieved!
File:- First of all bulk action validate a given directory
- if the path does not exist, it will create it
- then because of multiselection of systems, it will also create a folder with a name of the system
- in that folder will file with name: <system-name><date><time-millis-of-day>.csv (SomeDifferentVirt2018-11-15-52111169.csv)
- header is created according to the schema of the system
- it will take all attributes of schema and creates a header
- then I handle objects in the system
- iterate header and if there is value, it will put it on a new line, if not then it will put an empty string inside
- multivalued attributes are separated by ","
- create ResultCode messages for some exceptions
- connect these messages with localization
- create test
- create documentation
Updated by Marek Klement about 6 years ago
- if I get null from connector object for one particular item in the header, do I fill it ass empty string or null. The difference is in quotation chars - nothing against ""
- I have a method for filling multivalued - it puts multivalued separator before every item except first - the question is: What if I get null for the last item of these values? it will put ',' in the end - is it even possible?
Updated by Marek Klement almost 6 years ago
- % Done changed from 30 to 60
- All messages in exceptions and names are in locates
- Exceptions are solved by ResultCode
- Test - creates a system, import accounts and export it - deletes all created files after
- Documentation - [[https://wiki.czechidm.com/tutorial/adm/systems_-_export_of_accounts_to_csv|here]]
- Also fixed counter problem in ImportFromCSVToSystemExecutor
Updated by Marek Klement almost 6 years ago
- All messages in exceptions and names are in locates
- Exceptions are solved by ResultCode
- Test - creates a system, import accounts and export it - deletes all created files after
- Documentation - [doc](https://wiki.czechidm.com/tutorial/adm/systems_-_export_of_accounts_to_csv)
- Also fixed counter problem in ImportFromCSVToSystemExecutor
Updated by Marek Klement almost 6 years ago
- Status changed from In Progress to Needs feedback
- Assignee changed from Marek Klement to Alena Peterová
Updated by Alena Peterová almost 6 years ago
- Status changed from Needs feedback to In Progress
- Assignee changed from Alena Peterová to Marek Klement
- make DEFAULT_MULTIVALUED_SEPARATOR configurable (as "Separator of multivalued attributes"). The separator "," is not universal, e.g. when exporting AD, the values of groups contain "cn=groupname,ou=organization,dc=company,dc=tld". We would like to use "\n" (newline) for AD.
- (optional) make DEFAULT_LINE_SEPARATOR configurable (as "Attributes separator"). Although ";" is quite universal
- when the value contains " (double quote), it should be escaped by another double quote (escapechar in CSVWriter). Now it isn't escaped, which could cause some problems when parsing CSV.
- Description - change to "Export all accounts from system"
- file.getParentFile().mkdirs() - check the result. If false, return the ResultCodeException that the file cannot be created (same as when creating new file)
- Permissions - export shouldn't require SYSTEM_UPDATE, SYSTEM_READ should be enough. Please write the required permission to the wiki. (Note: when testing, there is a problem in permissions of bulk actions #1540)
- if there are some unresolved requests, the export has inconsistent data:
- unresolved created accounts aren't exported -> correct
- unresolved updates of accounts are exported already with the changes, which were not implemented yet -> problem, administrators would expect the real state on the system, which doesn't have the changes yet.
- unresolved deletes cause NullPointerException. These accounts should be exported, because they were not deleted yet
- after cancelling a delete request, the account is still exported -> correct
- when a new virtual system is created with a default role, "Behavior of the default role for inactive identities" must be set in the specific settings of the synchronization. Otherwise the synchronization doesn't start. Please set it to DO_NOT_LINK.
Updated by Alena Peterová almost 6 years ago
- Related to Defect #1540: Displaying bulk actions for systems require ROLE_READ and SYSTEM_DELETE added