Use the Job Management Queue

Home or Project Workspace > Status drop-down > Job Management...

Requires System-level Job Management - View permissions to view the Job information and Job Management - Add/Edit permissions to manage the Jobs or perform bulk actions

All System Users have the View permission for Job Management by default and can therefore view the Job Management Queue and use the Create Job Notification and View Live Log options. Users in the System Manager role have the Add and Edit permissions by default and can therefore (unless these permissions are removed) perform actions that control the state and order of one or more Jobs (for example, canceling a Job). The System Administrator role always has Add/Edit permissions. For information about System-level permissions, see View and Manage System Role-Based Permissions.

The Job Management Queue screen opens in a new browser tab. When you are done viewing the information, you can either close the tab or use it to view other Status menu items, such as Active User Sessions.

Job Management Options

While viewing the Job Management Queue, you can select the following from the toolbar:

Update – Click this button to update the information on demand (only requires View permissions).
Cancel Selected Jobs... – Click this button to cancel one or more selected Jobs in the grid (requires Add/Edit permissions). You will see a Cancel Multiple Jobs popup that asks you to confirm if you want to cancel the number of Jobs you selected. Click Cancel Jobs to continue or Don't Cancel Jobs to keep the task in its current state.
Create Job Notification... – Click this button to set up Job notifications for one or more selected Jobs in the grid (requires Add/Edit permissions).
Change Project Service... – Click this button to change the Service for Projects associated with one or more selected Jobs in the grid (requires Add/Edit permissions).

To individually select one or more Jobs, select the appropriate checkboxes in the grid. You can also select the checkbox in the list header to select all listed jobs.

If you have Add/Edit permissions for Job Management you can change the processing order of the current Jobs by dragging one or more eligible Jobs into the desired order. You must click Apply for a change in Job order to take effect.

Job Hover Options

If you have Add/Edit permissions for Job Management, depending on the state of a Job you can use the actions available when you hover over a Job:

(Pause) – Suspends a running Job temporarily. Note that you cannot pause a Workflow. (You can, however, pause a search from a Workflow.)

(Resume) –Enables you to resume a Job that has been paused. For a Job that has been suspended due to a failure, this button enables you to resume the task after the failure has been addressed. This prevents you from having to restart the Job.

(Cancel)) – Lets you cancel a Job in progress. The Job must be in a phase that allows cancellation. If you are permitted to cancel the operation, a Cancel confirmation popup appears. This popup displays the Job name, in quotes, to help you confirm that you have selected the appropriate Job to cancel (for example, Cancel "Generating View Manifest for download"?). Click Cancel Job to continue or Don't Cancel Job to keep the task in its current state. For an export, this clears the checkpoint information and deletes the open volume; the Job can no longer be resumed. You are free to export the same view again. If a System Administrator called sysadmin1 cancels a Job, the Server Log file will contain an entry with User: sysadmin1@null (since the System Administrator is not associated with a given Organization). When you cancel an OCR Job, you will see a warning that the OCR text already generated will not be preserved. In this case, you may want to evaluate the current progress before proceeding.

Apply or Discard Changes Options

If you do not save your changes before navigating away, you will be prompted to either save your changes or discard your changes.

Apply – Saves your changes.
Discard Changes – Discards your changes.

When you select a single Job (using the checkbox or clicking the row for the Job), you can also select one of the following context menu options (only View permissions are required for these options):

Create Job Notification... – For a selected (single) Job, select this option to request an email notification (to you and to any additional email addresses you supply). This option is available from either the ellipsis menu or from a right-click on the Job.
View Live Log... – For a selected (single) Job, select this option to view the log file information for the Job’s parent Service Request Identifier (SRI) in a new tab. This option is available from either the ellipsis menu or from a right-click on the Job.

Note: When multiple Jobs are selected, the context menu and its options will not be available.

Selecting a single Job using the checkbox or clicking the row will automatically display the Job Details. Clearing the checkbox for a Job or clicking the row again will deselect the Job and close the Details area.

Job Management Columns

The following columns are shown by default and display information about the various Jobs supported by Digital Reef.

Process — Identifies the process for a Job. An icon indicates the current status of the Job:
- Running – The Job is running. You can monitor the progress using the % column. The Job is currently executing (that is, it is in progress). Use the % Complete column to monitor the Job's progress.
- Queued – The Job is in a queue, waiting for system (grid) resources to become available to accommodate it. You may see tasks queued up behind a long-running task, waiting for the server to allocate processing time. Example: An Import Job may be queued behind multiple Import Jobs.
- Blocked – A Job is currently blocked because another Job in the same Project is causing changes to the underlying data, and those changes must be done in an exclusive manner. For example, if a tagging Job is requested and a request to add more documents to Project Data is made, the Job to add more documents may become blocked if the tagging Job is in a phase where the underlying data is being changed. Also, if you set up an import with the Add to Project Data option, the Adding documents to Project Data Job will be in the Blocked state because it cannot run until the import Job completes. A Job that is in the Blocked state cannot be deleted, but it can be canceled and then deleted.
- Paused – The Job is suspended (in response to a user request to pause the Job). The Job retains this state, using no resources, until a user resumes the Job. A paused Job retains all state settings applied to it before the Job was paused.
- Paused on Failure – The system suspended the Job in response to a failure (for example, a lack of disk space). Hovering over the icon changes it to the information icon, which you can then click to get details about the failure. Once the failure has been resolved, you can resume the Job where it left off, instead of having to restart the entire Job. You cannot delete an operation that is paused, but you can cancel it (even if it is in a critical state of processing) and then delete it. Note that under certain circumstances, when a Job is paused on failure, locking may prevent subsequent Jobs such as export or tagging from being scheduled until the Job completes with success or failure (that is, subsequent Jobs will appear as queued).
- Pending Cancellation – The system is working to cleanly cancel the Job and release any resources used by the Job. Depending on the state of the Job and the number of resources associated with it, canceling a Job can take some time.
- Pausing – The system is suspending all activity associated with the Job. Depending on the state of the Job and the number of resources associated with it, pausing a Job can take some time.
Type — The type of Job (for example, Import, Export, Search, Workflow, or Tag).
Username — The user who initiated the Job.
Organization — The Organization associated with the User who triggered the Job.
Project Name - Description — The name of the Project associated with the Job. If the Project has a description, a hyphen appears after the Project name, followed by the description.
Service — The name of the Service associated with the Job (for example, Digital Reef Default, or the name of a configured Service). This column information can help you review the Services for the listed Jobs and decide whether you should switch any Projects to another Service.
Rate (Requested / Current) — The Requested Rate and Current Rate as a percentage of the resources available for the Job. For example, if a Job was configured to use half of the available resources, the Requested Rate would be 50% and the Current Rate would typically be 50%. If you have Add/Edit permissions for Job Management you can change Requested Rate for a Job using the controls or by entering a value. If you change to a value outside the valid range of 1 to 100, such as 0, the value will return to a valid value when you move elsewhere in the screen. You must click Apply for a Rate change to take effect. Hovering over this field displays information about processing Cores including total Cores, available Cores, and assigned Cores (the number assigned to the Job).
Priority — The current resource Priority setting for this Job (Low, Normal, or High). If you have Add/Edit permissions for Job Management you can change the Priority by clicking one of the buttons. This may affect how a Job is allocated to a resource; for example, changing an Export Job from Normal to High will cause it to be allocated to an Express Node, if available. You do not have to click Apply; Priority changes take effect immediately.
Job State — The current state of the Job (for example, INDEX_JOB), which can help you track progress and monitor the different states of a user's request in the System (identified by the Service Request Identifier). This was previously known as the Current FSM State (Finite State Machine State).
Task Duration — The length of time that the Job has been in process in the format dd hh:mm:ss, depending on the available information. If the day value is zero, then it is not shown. This column is updated continuously. Sample task duration: 00:02:55.
Task % — The percentage complete for the Job so far.
Results — The number of documents produced as a result of the Job. The number produced displayed here is representative of the Job type. For example, for an import, this number would display the number of files imported so far. For a Search, it would display the number of matching search results so far.

The Process, Type, Username, Organization, and Project Name columns also support a filter row text box, shown for these columns by default:

Note: You can right-click a column and select which columns you want to see. If you have filtered column information, you can also right-click a column and select Reset All Filters to clear/reset all filtering.

Filter row text box — You can use the Filter row text box under the appropriate column. (The icon indicates that filtering is available.) Using the Filter box enables you to pinpoint the items you want to work with based on a Filter term item containing one or more characters you enter. You can explicitly apply a filter by typing one or more characters in the text box and clicking Enter (the return key). If you type one or more characters in the text box, the software will automatically apply the filter for you, and the text box changes to a yellow background color. For any applied filter, you can then clear the filter by removing the text in the box and clicking Enter, by removing the text from the box, or by clicking the that appears at the far right of the Filter box. Clearing a filter restores the list to its original state.

Job Details

To display detailed information for a given Job, select that Job in the Job List and the information appears in the bottom half of the screen.

Note: You can resize the Job Details area to control the display of this area versus the main Job list. To reduce the size, click the up arrow that appears on a gray panel resize bar (center of screen). To expand the area, click the down arrow that appears on the gray panel resize bar (center of screen).

Description: — A user-specified description of the Job (for example, Creating representation Analytic Index for <data_set_name>).

Additional Information:

Started: — The date and time that the Job was submitted in the format yyyy-MM-dd HH:mm:ss.
DR Service: — The Digital Reef processing node service assigned to the Job (for example, Digital Reef Entire Realm).
Nodes: — The number of processing nodes associated with and available to the task.
Express Node: — Whether or not the processing node associated with the Job is an Express Node (that is, whether the node is Express Node capable or Not capable).

Track Progress of Phases for Key Jobs

You can view both general and detailed information about the different phases of a selected Job.

Viewing the detailed information, just like viewing the Task Details in the Work Basket of a Project, can help you track phases of a Job/task, get more information about options in effect for the Job. The details for a given Job will include sections specific to the Job as well as sections that apply to all tasks. For example, for key operations such as Import and Export, you can view the scheduling information, to see how resources were allocated.

Note: The currently running phase for a Job includes (Running) after the name of the phase. For example, if the Copy Source Files phase of the Project Export Job is running, you will see Copy Source Files (Running). You can also see the percent complete for a given phase. A value of 100 for a phase indicates that it is 100 percent complete.

Common Sections of the Job Details

Some common sections of the Job Details are as follows:

AE Resource Stats — Displays the Analytic Nodes that have been involved in completing a Job. The primary Node is identified as the Master AE Resource. If additional Nodes are involved, they are identified individually as Helper AE Resources.
Execution Summary — Displays the amounts of time that a Job has been running, queued, or blocked. For SRI Prefix, you may see one Service Request Identifier (SRI) or a comma-separated list of associated SRIs, ending with a colon (:).
General Information — Displays the Service Request Identifier (SRI) and Task Handle information. In the event of a failure, you can use this information to help debug the source of the error.
Service Node State —The status of the Job (for example, Current State RUNNING). This information also appears in the Status column of the Job List.

Service Node Debug State — The current debug status of the task, which includes the Current Job State (previously known as the Current FSM State), Instance ID, and associated SRIs. A sample Current Job State for an Import Job would be INDEX_JOB.

Scheduling Summary — Displays a summary of the scheduling information for the Job, such as the resources requested and allocated (for example, for a sample Import task, the scheduling summary information might include Master Component Cores Requested 0.50 cores, Master Component Memory Requested 256 MB, Master Component Memory Allocation 256 MB, and Master Component Cores Allocated 0.50 cores).
Scheduling Info — Displays scheduling information for components involved in the Job. These will vary based on the type of Job (for example, you will see Export-related components such as the ARCA agent for Export, Doc Manager components for Import, and vector-related VCA components when vectors are produced for operations such as near-duplicate processing).

The following sections show the different phases for a number of Jobs.

Project Delete Job Details

The following details are reported for a Project Delete Job:

Project Delete Start - Tracks the start of the Project Delete process.
Cancel Project SRIs - Tracks the canceling of SRIs for the Project.
Delete Project Synthetic Documents. - Tracks the deletion of any Synthetic Documents in the Project.
Delete Folders - Tracks the deletion of all Folders in the Project.
Delete Comparisons - Tracks the deletion of any Comparisons in the Project, as defined in eDiscovery.
Delete Project Data and the Discard Pile - Tracks the deletion of Project Data and the Discard Pile.
Delete All Imports - Tracks the deletion of all items under Imports (any Load File Imports, all Data Sets created in the Project, and references to any Shared Data Sets).
Delete Data Sets - Tracks the deletion of all Data Sets in the Project.
Delete Organization Template References - Tracks the deletion of the Project's references to Organization templates.
Delete Service Tier Objects for Project - Tracks the deletion of Service Tier objects for the Project.
Delete Project Files - Tracks the deletion of Project-related files.
Delete Project Work Basket - Tracks the deletion of the Project Work Basket.

Project Export Job Details

The following details are reported for a Project Export Job:

Project Export Start — Tracks the start of the Project Export process.
Activate Project Data Area — Tracks the activation of the Project Data Area.
Copy Files — Tracks the copy of files to the target Project Data Area.
Copy Work Basket — Tracks the copy of the Project Work Basket.
Copy Database Stage 1 — Tracks the first stage of the Service Tier Copy Database operation.
Copy Views — Tracks the copy of all views in the Project.
Copy Database Stage 2 — Tracks the second stage of the Service Tier Copy Database operation.
Activate Data Areas — Tracks activation of Data Areas in the Project.
Copy Source Files — Tracks the copy of the source files used by the Project. You can also view the number of results for this phase.
Project Export Complete — Tracks the completion of the Project Export process.

Import Job Details

The following details are reported for a Data Set Import Job (Creating representation <representation_level> for <dataset_name>) Job:

Phase (1) Usage Results: INDEX_JOB — Reports statistics about the initial indexing phase.
Phase (2) Usage Results: INDEX_MERGE_JOB — Reports statistics about the merge Job for the Index.
Phase (3) Usage Results: EXTRACT_VECTORS_JOB — Reports statistics about the extraction of vectors for an Analytic Index.
Phase (4) Usage Results: VECTORS_COMMIT_JOB — Reports statistics about the save of vectors for an Analytic Index.
Representation Configuration — Reports the Index Settings for the Import Job, including the current OCR Processing, Language Detection, Numeric Quantities, Currency, Numeric Terms, Pattern Detection, and Stop Words in vector settings.

As indicated, you can view each Phase of the Import task, and some Phases are transitory. For example, you may see an in-progress Import Job at the Analytic Index representation level report the following phase:

Phase (1) Usage Results: CORPUS_VECTOR_DATA_REP_CREATE_JOB

You may also see the following phase reported early in the Import process:

Phase (1) Usage Results: DIRECTORY_JOB — Reports statistic about the initial directory Job.

An Import Job will also include sections for related component work, such as Document Manager (DocMgr) operations.

Search Job Details

The following details are reported for a query-based (term) search:

Description — Identifies the query as part of the Task description.
Search Target — Identifies the name of the view searched.

A query-based search also includes a Search Progress section, which lists information for the settings that apply to the search. An in-progress search will include the percentage complete that applies to each setting when you launch the popup, while a completed search will show 100 for the percentage complete:

Include Metadata and/or Synonyms — Tracks the expansion of the query to include metadata and synonyms, if applicable.
Perform Wildcard Expansion — Tracks the expansion of the query based on the use of wildcards, such as * (asterisk) and ? (question mark).
Search Index — Tracks the progress of the Search (that is, search of the terms in the index). This will not be populated if the Generate Reports option is selected for the search.
Include Families — Tracks the inclusion of families (MAGs and DAGs), if the Include Families option is selected for the search and the Generate Reports option is not selected for the search. Otherwise, it is not populated.
Search Index and Generate Reports — Tracks the requested generation of the reports as part of the Search process (so the user does not have to wait for them to be generated later). This field will not be populated unless the user has selected the Generate Reports option for the Search. If it is populated, the standalone Search Index field (and Include Families field, if the option is selected) will not be populated.
Create Per-Clause Report — Tracks the creation of the per-clause report for the clauses included in the query.
No. of Results — For an in-progress Search (issued without Generate Reports), indicates the number of results found so far.

In general, the details for a query-based search issued without the Generate Reports option will include the following phase:

Phase (1) Usage Results: LUCENE_SEARCH_JOB

Additional phases depend on your selected settings. For example, if you perform a search with the Include Families option but without the Generate Reports option, you will also see the following phase when you view the details (for example, for a completed search):

Phase (2) Usage Results: MAG_EXPANSION_JOB

If you perform a search with the Generate Reports option, you will see the following "combined" phase, which includes the search, MAG expansion, and report generation (so you will not see the LUCENE_SEARCH_JOB or MAG_EXPANSION_JOB phases):

Phase (1) Usage Results: SEARCH_AND_CALCULATE_REPORT_JOB

Reprocessing Details

The Reprocessing Details section of the details for a reprocessing Job will display entries for the reprocessing options (separate lines for each option).

Example of Reprocessing Details when Reprocess Documents Children is selected:

Apply New Tag to Children False

Apply Parent Tags to Children True

Document Timeout 5 Minutes

Message Archive Timeout 10 Minutes

Reprocess Documents with Children True

Scan Date 2017-11-15-19-28-08

Sync Document Children to Project Data True

Update Native Files False

If you run the Reprocess operation with Reprocess Documents Only, the Extract from Container entry also appears as True or False, depending on the selection made for the Reprocess options.

You can also use view details for a Load from External Area task to see the options selected. For example:

Example of Details when Load External OCR Text is selected:

Load External OCR Text True

Scan Date 2017-11-15-19-28-08

Export Details

For an Export Job, you can see sections that reflect the different phases of Export:

For an in-progress eDiscovery Export, you will see sections that reflect the different Export settings selected:

Export Output Format Options — For settings such as the Documents to Export criteria (such as tag criteria), the type of manifest (e.g., DAT or CSV), and the date/time format and time zone information.
Export Manifest Options — For manifest settings such as BegAttach starts with and ThreadGroup includes Attachments.
Export Duplicate Processing — For the Duplicates Processing mode and the Near-Duplicate settings.
Export Identifier Controls — For the Volume label, ID Prefix, Starting ID, Pad Size, and Page Level Numbering settings.
Export Email Options — For the email-related settings such as Separate Email Attachments, Include Container References, and Remove Attached Archives.
Additional File Options — For the production formats selected.

An in-progress eDiscovery Export (for Export production) will also report current phases. For example:

Phase (1) Usage Results: EXPORT_VIEW_CREATE_VOLUME_JOB
Phase (2) Usage Results: EXPORT_VIEW_ADD_PREPARE_JOB
Phase (3) Usage Results: EXPORT_VIEW_ADD_COMMIT_JOB
Phase (1) Usage Results: COLLECT_DOCUMENTS_COLLECT_JOB
Phase (2) Usage Results: PREPARE_JOB

An in-progress eDiscovery Export (for staging) will report current phases. For example:

Phase (1) Usage Results: COLLECT_DOCUMENTS_COLLECT_JOB
Phase (2) Usage Results: PREPARE_JOB
Phase (3) Usage Results: PROCESS_JOB
Phase (4) Usage Results: PREPARE_COMMIT_JOB
Phase (1) Usage Results: EXPORT_VIEW_ADD_PREPARE_JOB

Note that a completed Export task that used tagging criteria to determine the documents to Export would report the following phases in the Project Work Basket:

Phase (1) Usage Results: TAG_REASON_JOB
Phase (2) Usage Results: EXPORT_PRODUCE_JOB
Phase (3) Usage Results: EXPORT_COMMIT_JOB
Phase (4) Usage Results: EXPORT_MANIFEST_JOB

An eDiscovery Export will also include various sections for component work (for example, ARCA work and VCA work).

Tag Apply Details

An in-progress Tag Apply Job will also report current phases. For example, for tagging of selected searches from a Bulk Search:

Phase (1) Usage Results: MERGE_RESULT_VIEWS
Phase (2) Usage Results: APPLY_PREPARE_JOB
Phase (3) Usage Results: APPLY_COMMIT_JOB

For a regular Tag Apply Job (for example, outside of a Bulk Search), you would see the APPLY_PREPARE_JOB and the APPLY_COMMIT_JOB.