Manage Load File Import in a Project-Level Load File Template

Home > selected Project > menu or right-click > Settings > Load File Import > selected Load File Import template
Project > Settings > Project Settings tab > Load File Import > selected Load File Import template

Requires Project - Load File Import - View, Add/Edit, Delete Permissions

Users in a role with the appropriate permissions can manage the Load File Field information needed for a Load File Import. At the Project level, this is done in a Load File template. Users with the appropriate permissions can also manage Load File information in an Organization Load File template.

When you select the Project-level Load File Import, a user with the appropriate permissions can use the top-level Templates context menu to perform the following action:

  • Create a Project-level Load File Import template by clicking the (New Template) option, which launches the New Template properties dialog.

For a selected Project-level Load File Import template, a user with the appropriate permissions can click the ellipses and use the context menu to perform the following actions:

  • Save to Template – Launches the Save to Template dialog, which enables you to save current settings to an available template based on your permissions. For example, at the Project level, you can save to a Project-level or Organization-level template, depending on your permissions, or select New Template, which launches the New Template dialog.
  • Load from Template – Launches the Load from Template dialog, which enables you to load the settings from a template. If you are at the Project level, you can load the settings from a Project-level or Organization-level Load File Import template you select using the option (if you have the appropriate permissions). The loaded settings then appear and are saved automatically. You must have Add/Edit permissions for Load File Import in the Project to use the Load from Template option to load settings from a selected template (from a list of available templates).
  • Select Set As Default, which marks the selected template as the default template. This is not available for the Default template of a given type, or for any other template already set as the Default.
  • Edit – Launches the Edit Template dialog, which enables you to edit the template name and/or description of the selected template.
  • Delete – Delete a template, which causes the display of a popup asking you to verify the deletion of the template from the Project.

Note: Save to and Load from operations for this setting observe an "overwrite" behavior. For example, for a Load from operation, your current settings are replaced by the settings from the selected template/settings. Note that some settings, such as Patterns, Tags, Domain Lists, Alias Lists and Excluded Content observe an "append" behavior instead.

Overview of Steps in Setting Up Load File Import Information

This process involves discovering and adding the Load File fields you want to include for a type of Load File, and providing the required field mappings between the Load File fields and the DR Metadata fields. As part of the field mapping process, you work with the appropriate Transformer, which provides details about the mapping required. Defining Load File Field information therefore involves the following steps:

  1. Provide the Load File fields. Click Discover Fields on the toolbar to populate the Load File fields based on the contents of a Load File at a specified location.
  2. Populate the table.
    • Drag-and-drop each Load File field to the left column in the table.
    • For each Load File field added to the table, use the DR Metadata Fields list on the far right to find the appropriate Digital Reef metadata field to which the Load File field can be mapped.

    Note: See Mapping Requirements for information about fields that must be mapped.

  3. Select the appropriate Transformer type for each mapping in the Transformer column of the Field Mapping table.

  4. Save your changes. You will be reminded if your changes do not include any required field mapping.

When you Load File Import information is established, you can then import a Load File, as described in Import Data from a Load File.

Load File Fields

Load File Fields Options

To populate the Load File Fields list, you can use the following options:

  • — Launches a Discover Fields dialog that enables you to select a Connector and Data Area and locate a Load File you want to use.
  • Add Field

For selected Load File Fields in the list, you can perform the following, activated by checkbox selection in the table:

  • — For Load File fields you select with the checkboxes, enables you to delete those fields. You are asked to confirm the deletion.

The table enables you to drag-and-drop Load File Fields and map them to Digital Reef Metadata Fields. Each mapping entry uses to provide details about the mapping required.

Table Columns

The table contains three columns:

  • Load File Field (Left column) — The Load File in a field mapping entry.
  • Transformer — The Transformer used for a field mapping entry. You can edit information for a selected Load file field using one of the following Transformers to provide essential mapping information:
    • — Accommodates changes required for Date/Time formats.
    • — Allows you to specify text information for a Load file field mapping, such as a prefix or suffix, replacement of a character in a path, such as replacing \ with /, or a trim of characters. You can select from a list of available operations and text that applies to the operations.
  • DR Metadata Field (Right column) — The appropriate Digital Reef Metadata field for a mapping entry.

Table Options

The table supports the following, activated by checkbox selection in the table:

  • — For field mappings you select with the checkboxes, enables you to delete those field mappings. You are asked to confirm the deletion.

Note: For any mapping exercise, you must be familiar with the contents of the load file you plan to use. Load File content may vary widely, so it is important to understand the setup of your Load File and then perform the appropriate mapping.

Mapping Requirements and Guidelines

When the load file includes natives or text versions, you will typically have at least one mapping entry to identify the location of the files. Depending on the discovered fields, entries may exist for one or both of the following:

  • the location of native files, using a Load File field such as NativeLink/NativeFile (hereafter referred to as NativeLink).
  • the location of text files from OCR processing, using a Load File field such as TextLink/OCRPath (hereafter referred to as TextLink).

In general, you must also have a mapping entry forextbegdoc to identify the external starting document with a document number. This mapping entry is required for any load file import setup.

If the load file is set up for image processing only, you might not have a mapping entry to darelativepath or ocrpath at all, but you will need to provide the required mapping entries, as described in Mapping to Process Load File Images.

Use the following mapping guidelines:

  • If you are providing only one mapping entry to identify the location of the files (whether it is NativeLink for native files or Textlink for text files from OCR processing), you must map to the Digital Reef darelativepath field (categorized under Digital Reef Properties). If your load file uses the DR\ prefix for the mapping entry, you can use a Text Transformer to trim the first three characters of the DR Base Path (DR\). If your load file does not use the DR\ prefix (and does not require any other adjustment), leave the mapping using a Pass-through Transformer.
  • If you are supplying more than one mapping entry for the location of the files, you can use a Load File field such as NativeLink for native files and map it to darelativepath, and then use a Load File field such as TextLink (for the OCR-processed text files) and map it to the Digital Reef ocrpath field (categorized under Digital Reef Properties). Supplying more than one mapping entry for the location of the files provides additional information.
  • As long as you provide a field mapping to identify the location of a native file and the native is available at that location, the filetype field for the document will report the appropriate filetype; if not, the filetype field will report an Unknown format because there is no native.
  • You must have a mapping entry to the Digital Reef extbegdoc field to ensure that the external starting document has a document number.
  • You may want to supply a mapping entry to the Digital Reef filename field to ensure that you see the expected filename.
  • You cannot provide mapping entries to the following fields (listed under the appropriate properties): contentmd5, docext, filemd5, and filetype. See the section Mapping Needed to Preserve Families for restrictions concerning the mapping of family related fields.

Note: The software can accommodate the situation in which the Load File provides natives for some files, but not all. In this case, a document without a populated NativeLink (or equivalent) field might have a populated TextLink(or equivalent) field that points to extracted text. To ensure that the software can identify the darelativepath information if the NativeLink (or equivalent) field is not populated, you can include an entry that maps the TextLink (or equivalent) field to the Digital Reef ocrpath field.

Viewing Load File Source Information

After import, you can verify Load File source information for a given document in the metadata field loadfiledocsource.This field either provides a semicolon-delimited list of the sources that were used to identify the document (for example, NATIVE;TEXT;IMAGE) or the value NONE if the Load File import did not provide any of those items for a document. Note the following:

  • NATIVE indicates that the native file for a given document was available based on Load File field mapping such as NativeFile/NativeLink to the darelativepath field.
  • TEXT indicates that a text file (for example, an OCR-processed text file) for a given document was available based on Load File field mapping such as TextLink or OCRPath to the ocrpath field.
  • IMAGE indicates that Load File images for a given document were available and processed separately based on an OPT or LFP Image Load File (available at the same location as the DAT Load File upon import).

The values in the loadfiledocsource field determine how representations for a document are made available:

  • When a native file is available (that is, the loadfiledocsource field includes the value NATIVE), that native is used for the native representation. Otherwise, an empty file is generated to represent the native.
  • When a text file is available (that is, the loadfiledocsource field includes the value TEXT), that text file is used for the text representation. Otherwise, text will be derived from the native.
  • When images are available (that is, the loadfiledocsource field includes the value IMAGE), those images are used to provide an image representation in PDF format.

Mapping Needed to Preserve Families

Several field mapping entries are needed in order to preserve families. If you have the Preserve Families option enabled , you must have mapping entries for the following properties:

  • extbegdoc (generally required) — The external starting document with a document number (under Digital_Reef_Properties).
  • extbegattach (required)  — The external starting attachment document with a document number (for example, for an email). This field is typically populated for the beginning attachment for an entire family, but a standalone document may also have this field populated. This field is categorized under Digital_Reef_Properties.

The following fields are recommended to provide optimal family handling:

  • extenddoc (recommended) — The external ending document with a document number (under Digital_Reef_Properties). If this field is populated, the software will also populate the extdocrange and extdocattachrange fields.
  • extendattach (recommended) — The external ending attachment document with a document number (under Digital_Reef_Properties). If this field is populated, the software will attempt to preserve families when a family would otherwise be considered broken and generate an error during the Load File import.

Note: Even if you decide to clear the Preserve Families option, you can still map all of these family fields and the software will use them to preserve the family information. Without all of these family field mapping entries present, all documents are treated as individual documents (without any MAG or DAG relationships). Do not provide mapping entries for the attachments and parent fields, as the software will automatically populate these fields to preserve families.

Mapping Needed to Process Load File Images

To process the images for an OPT or LFP Image Load File (available at the same location as the DAT Load File upon import), you must enable and apply the Process Load File Images option on the Load File Settings tab, and you must supply a mapping entry for the following:

  • extbegdoc (generally required) — The external starting document with a document number (under Digital_Reef_Properties).

Optionally, you can provide a mapping entry for the following Digital Reef Properties:

  • extenddoc (optional) — The external ending document with a document number (under Digital_Reef_Properties).
  • extnumpages (optional) — The number of pages detected for a multi-page document (the number of pages for an attachment in a Load File Image PDF). A Load File field such as PgCount can be mapped to this field.

Upon successful processing of the images for an OPT or LFP Image Load File, the software creates a PDF that represents all of the images associated with a given Load File document.

After a successful import of a Load File with processed images, you can check the stored_image metadata field, which identifies External to indicate that the images were generated externally. You can also view the PDF created to contain the images for a document in the Image tab/view mode of the Document Viewer. The Image tab only appears for a document that has stored images as a result of a Load File Import with processed images, External Image Import, or an Export that requests PDFs.

Custom Fields

You can also map Load File fields to Custom Fields that you have added (Custom Fields are Digital Reef Custom Properties). When you map a Load File field to a Custom Field, you select the field and drag the field to the mapping table. If you provide the Custom Field in a the mapping entry will contain the prefix drcustom- (for example, if your field is custom1, the entry will show drcustom-custom1). This is to avoid potential field name collisions.

Sample Load File Field Mappings

The following lists sample Load file Settings and sample Load File Field Mappings for the default all

Load File Field ---> DR Metadata Field

NativeFile ---> darelativepath

BegDoc# ---> extbegdoc

EndDoc# ---> extenddoc

BegAttach ---> extbegattach

Filename ---> filename

OCRPath ----> ocrpath

DR Metadata Fields

The right side of the screen displays an scrollable list of the Digital Reef Metadata fields. .

  • Name — Displays the Digital Reef name for the field.

To help you focus on a subset of fields, you can use the following to temporarily filter the list of available fields:

If you do not save your changes to the Load File information before navigating away, you will be prompted to

If you have not defined the required mapping entries, you see an invalid mapping message when you try to save the changes. For example, if Preserve Families is enabled, you will see a message if you have not provided the required family-related mapping entries. Without the required information, your changes cannot be saved and they will be discarded if you navigate away. For your convenience after you see a mapping error, you can modify the current content in the mapping table and then save the appropriate mapping entries.

  • – Saves your Load File Import changes.
  • – Discards your Load File Import changes.

Next Steps: Add Data from a Load File

After you have defined your Load File information, you can import a  Load File. See Add Data from a Load File for more information.