Manage Patterns in a Template

Home > selected Organization > menu or right-click > Settings > eDiscovery Templates > Patterns
Project > Settings drop-down > Organization Settings > eDiscovery Templates > Patterns

Requires Organization - Patterns - View, Add/Edit Permissions

A Pattern (regular expression) is a sequence of characters typically used to perform a pattern match and identify patterned data during the parsing process. The Digital Reef software includes a number of predefined Patterns, called System Patterns A set of preconfigured Patterns (also known as regular expressions) that match specific types of data. The content of a System Pattern cannot be edited and a System Pattern cannot be deleted. They can be enabled and disabled with or without storing values and can be copied to serve as the basis for Custom (user-defined) Patterns. They must be enabled before they are available for use. A subset of System Patterns are enabled automatically., which are included in the Default Patterns template. The email, UNC, and URI System Patterns are enabled by default in the Default Patterns template, so in new Projects that reference the Default template, these Patterns store values by default. Users with permissions can also create Custom Patterns A locally defined pattern (also known as a regular expression) identified by its name, that is used during the parsing process to match specific data patterns. Custom Patterns can be created, deleted, enabled, and disabled. Custom Patterns must be enabled before data is added or reprocessed..

Note: Patterns apply to initial import, a Pattern update, or reprocessing. If you change the System Patterns or Custom Patterns for your Project, you can either use the Reprocess option from results to reprocess and pick up the latest Pattern changes, or you can update the Patterns using the standalone Update Patterns option (for example, by right-clicking on a Data Set in the tree). Otherwise, your Pattern changes will have no effect on the existing Data Set documents.

When you select a template type such as Patterns from the Organization Settings, a user with the appropriate permissions can view a list of Patterns templates and use the top-level Templates context menu to perform the following action:

Create a template by clicking the (New Template) option, which launches the New Template dialog. When you add a new Patterns template from Organization Settings, it will be empty, which enables you to define your own Custom Patterns instead of starting with all of the Patterns included in the Default Patterns template.

For a selected template, a user with the appropriate permissions can click the ellipses and use the context menu to perform the following actions:

Save to Template – Launches the Save to Template dialog, which enables you to save current settings to an available template, or select New Template, which launches the New Template .
Load from Template – Launches the Load from Template dialog, which enables you to load the settings from a selected System template or Organization template. The loaded settings and fields then appear.

Select Set As Default, which marks the selected template as the default template. This is not available for the Default template of a given type, or for any other template already set as the Default.

Edit – Launches the Edit Template dialog, which enables you to edit the template name and/or description of the selected template.
Delete – Delete a template, which causes the display of a popup asking you to verify the deletion of the template from the Organization.

Note: Save to and Load from Template operations for this setting observe an "append" behavior with regard to Custom Patterns. For example, for a Load from operation, your current Custom Pattern settings are preserved and only new, unique items from the source template/settings are added. Items with any name collisions are not added. Note that other settings, such as Index settings and Analytic settings, observe an "overwrite" behavior instead.

Patterns Summary

The Patterns summary shows information about each Pattern, as follows:

Name – The name of the Pattern. For data processed prior to 4.3.11.0, the Pattern name serves as a searchable Token Name, and is the name you include in searches and that can appear in the Top Terms list for a Cluster. For newly processed, updated, or reprocessed data as of Release 4.3.11.0, Tokens do not apply, and you use the Pattern name in a pattern metadata field search to find documents that match an enabled Pattern (using the format pattern::<pattern_name>). Example: ssn.
Searchable Token Name (optional column, not shown by default) – In data processed prior to 4.3.11.0, this column displays how to search for the token using the 'token-<token_name>' format. This format requires you to place the search within single quotes and specify the Token name in lowercase, since the software normalizes a Token name to lowercase. Example: 'token-ssn'. In newly processed, updated, or reprocessed data as of Release 4.3.11.0, you do not use this format and instead search for an enabled Pattern using the pattern metadata field and the Pattern name (pattern::<pattern_name>, such as pattern::email).

Description – A short description of the Pattern.
Enable – A check mark indicates that this Pattern is enabled.

Store Value – When enabled, as indicated by a check mark, you can search for an enabled Pattern as well as individual Pattern values.For data processed prior to 4.3.11.0, both the tokens and the individual values are added to the system dictionary and are available for search and clustering operations. For documents processed as of 4.3.11.0, this means you can search for an enabled Pattern using the pattern field and search for a Pattern value using the patternvalue field. If values are not stored for an enabled Pattern, documents processed prior to 4.3.11.0 have the token applied, which means that you can identify the documents that contain matching data but you cannot search for specific values that triggered the match.For documents processed in 4.3.11.0, this means you can search for an enabled Pattern using the pattern metadata field, but you cannot search for specific Pattern values using the patternvalue field. To change settings for a Pattern, edit the Pattern.

Created By – The login name of the user who created the Pattern. System indicates a System Pattern.
Pattern – The contents of the Pattern.

Store Values Tips:

Control characters, such as newline, tab, and so forth, can be stored but they are not searchable or displayed in a Cluster’s top terms. When you search for Pattern matches that include control characters, you need to use wildcards to represent the control characters.

Patterns that can return long matches might not be the best candidates for storing values. Patterns that are concise and that would not span lines or have a lot of embedded control characters are better candidates for value storage and subsequent searching.

New and Selected Pattern Options

To add a new Pattern, use the top-level New Pattern option.

For a selected Pattern, right-click the Pattern or click the ellipses at the far right to see a menu with the following options, as long as you have permissions to perform those actions (actions that are not permitted will be grayed out):

Note: The Copy, Edit, and Delete options require that you first select an item in the list. See Add, Edit or Copy a Pattern for more information about adding, editing, or copying a Pattern.

Copy – Creates a new Custom Pattern by copying a System Pattern or an existing Custom Pattern. A user with permissions can perform a copy of regex content with standard Ctrl-C operations.

Edit – Enables you to edit a selected Pattern. A user with appropriate permissions can edit all fields for a Custom Pattern. For a System Pattern, you can edit only the Enable and Store Value options.
Delete – Deletes a Custom Pattern upon confirmation. A user with permissions can delete a Custom Pattern. System Patterns are not eligible for deletion.

Usage Notes

Parsed Patterns can affect all data in a Project. The enabled Patterns affect how data is parsed.

Enabling Patterns and/or storing values of Patterns may increase processing overhead and the disk space required to support the Data Set.

In general, you should enable Patterns before adding documents to a Data Set or before reindexing a Data Setbecause that is when the initial parsing operation takes place.