Create a New Data Set from a Load File

Imports > New Data Set from Load File

Requires Imports - Add/Edit Permissions

Note: Digital Reef now restricts import and reprocessing of data to Projects using Parsing Library V2. You can no longer import, reindex, or reprocess data in a Parsing Library V1 Project.

Using the Imports > New Data Set from Load File option, you can import the data in a Load File to create a new Data Set for the Project. To do so, there must be a Load File Import template available that matches the type of the Load File you want to import. Supported Load Files types include the following:

  • Concordance DAT file (character-delimited)
  • CSV file (character-delimited)

Note: An imported Load File cannot be Shared across Projects in the Organization.

To successfully import a Load File, you must have access to a Load File Import template the matches the type of the Load File and its contents, including encoding and character delimiters (if applicable) and mappings pairing fields discovered in the Load File or a similar one with Digital Reef metadata fields, so that the data in the Load File can be effectively stored and used within Digital Reef. If you do not yet have access to or know of such a template, see Managing Load File Import Templates for information about creating such a template for the Load File you want to import as a new Data Set.

Create a new Data Set from a Load File

To create a new Data Set from a Load File, take the following steps:

  1. Select a Connector from the Connectors table.

  2. In the Folder panel, browse the folders and subfolders available under the Connector until the Load File you are looking for is displayed (potentially among others) and select it; the Path box is populated with the full path.

  3. Give the Data Set a name (which must be unique within the Organization) and optionally a helpful description.

    Note: As with most tree items, Data Set names can include alphanumeric characters, spaces (leading and trailing spaces are ignored), and some supported characters (such as a hyphen, underscore, and apostrophe), as well as characters from some foreign languages (for example, Korean characters). However, the following characters are not supported: ! " # $ % & * + . / : ; < = > ? @ [ \ ] ^ { | } ~ “ ”

  4. Select a Load File Import template from the drop-down. You can edit and save the template (including as a new template) after selecting it by clicking Edit.

  5. Use Index Level to select the appropriate Index representation level. Use the default of Analytic Index if you want to have a Load File take advantage of all analytic capabilities. The different levels of Indexing are as follows:

    • System Metadata – Restricts users to a system (structural metadata) view and a restricted subset of related operations. The Metadata List identifies the system (structural) metadata fields.
    • File Metadata – Restricts users to a metadata-only view of file (embedded) metadata as well as system metadata. This type is also associated with a restricted subset of related operations. When you select File Metadata, RAR, TAR, and ZIP archives are expanded by default to reveal the file metadata for the archive content, but you have the option to disable the expansion of RAR, TAR, and ZIP archives. File Metadata mode always supports the identification and import of Forensic Images (for example, EWF Files that collectively form a disk image).
    • Content Index – Gives users a view of document content and document metadata, thereby providing operations that enable analysis of both content and metadata. This is the only Index level you can later upgrade to an Analytic Index.
    • Analytic Index (default) – Enables users to take advantage of the additional analytic operations such as Document Similarity and Clustering. With this Indexing type, you can use a Project Analytic setting to ignore or include Stop Words for Document Similarity operations and Clustering, if applied.

  6. Optionally review the current Index Settings for the Project, making and saving changes if desired, by clicking Edit for Index Settings.

  7. Optionally disable pattern detection, which is enabled by default. You can also review the current patterns settings for the Project, and make changes if desired, by clicking Edit.

  8. Optionally specify a batch name or number for the new Data Set. If you do not set a batch name or number as part of import, the Data Set name is used. To verify the batch name or number after import, you can view the batch field in the document metadata eDiscovery Properties after import, the Data Set Report > View Details > Scan History after selecting the import from the Scan History tab, or theView Configuration information for a selected Data Set in the Imports Summary. It also appears in the appropriate file manifest upon export.

  9. To define additional eDiscovery options, click Other Legal Discovery Options that provide information to support chain of custody requirements. In the Other Legal Discovery Options popup that displays, you can view or make use of the complete set of eDiscovery options.

When you have finished setting up the Load File Import, click New Data Set from Load File to complete the process. You can monitor your import task in the Work Basket; after the import is complete, the new Data Set is displayed in the Imports Summary.

Load File Import Failures and Warnings

If the imported Load File does not match the type or encoding of the Load File used to create the Load File Import template, the entire import fails. If any child within a given family generates an error, import of the entire family fails.

A Warning icon () for the Work Basket task indicates that the Load File Import task completed with exceptions. You can then right-click the task and use the Download option to download the line-by-line errors in a CSV file. (Lines that do not have errors were successfully imported.)