Guides
Home

Workflow Jobs and Operations

Understand workflow jobs

Overview

A workflow is a standard and repeatable process that manages, transforms, or writes data. Such workflows may consist of multiple subprocesses that are performed in a specific sequence —in a pipeline— by which the output of one step is used in the next step of the process. Such workflows tend to be rather long-running (time-consuming) and thereby pose a problem for RESTful APIs as long-running processes can be blocking.

Workflow job responses

In the Risk Modeler and Data Bridge APIs, long-running processes are managed as workflow jobs that are scheduled and processed by a workflow engine. These APIs return a 202 Accepted response code that enables the workflow engine to manage workflow jobs asynchronously and accept requests for other processes without requiring that connections to persist until the process is completed.

Most requests require little time to complete; the API can immediately return a 200 OK response code that indicates to the client that its request was successful. But in some cases, the client may make a request that initiates a long-running and time-consuming process, e.g. the creation of the core objects, data migration, the running reports, and risk analysis. In such cases, the API adds a workflow job to the workflow engine queue and returns a 202 Accepted response code to the client.

The 202 Accepted response code indicates that a request has been accepted for processing, that the API has added a workflow job to workflow engine queue, and provides the client with a means of tracking the status of the workflow job. The platform does not notify the client when the workflow job is complete. Rather, operations that return the workflowId of the job enabling the client to track the progress of the workflow job.

On submission, an operation that initiates a workflow job returns a 202 Accepted response status code and a URL in the Location response header that specifies ID number of the workflow job.

The Location header of the response returns information about the job including its type, ID, and a URL for retrieving information about the status of the job. 

202 ACCEPTED
Location: https://api-euw1.rms-npe.com/riskmodeler/v1/workflows/9050766

The Location header returns a URL that represents the endpoint Get Workflow or Operation, which accepts a workflowId as a path parameter.

Standard workflows

A standard workflow is a system-defined process that the Intelligent Risk Platform manages as a workflow job on the workflow engine.

The Risk Modeler and Data Bridge APIs expose operations that enable tenants define and initiate workflow jobs. These processes are typically time-consuming and expensive and include creation of the core objects, running reports, or risk analysis.

The table lists workflow job types and the Risk Modeler API operations that may be used to initiate those jobs:

JobResources
ALMPOST v2/exposurebatches, POST v1/aggregate-portfolios/{id}/process, POST v2/aggregate-portfolios/{id}/process
ALM_IMPORTPOST v1/imports
BULK_EDITPOST v1/globaledit
CLIMATE_CHANGEPOST /v1/analyses/{id}/climate-change, POST /v2/analyses/{id}/climate-change
CONVERT_ACCOUNT_CURRENCYPOST v1/accounts/{id}/convertcurrency, POST v2/accounts/{id}/convertcurrency
CONVERT_AGGREGATE_PORTFOLIO_CURRENCYPOST v1/aggregate-portfolios/{id})/convertcurrency, POST v2/accounts/{id}/convertcurrency, POST v2/aggregate-portfolios/{id}/convertcurrency
CONVERT_EVENT_RATE_LOSSPOST v1/analyses/{id}/convert-event-rate-loss, POST v2/analyses/{id}/convert-event-rate-loss
CONVERT_PORTFOLIO_CURRENCYPOST v1/portfolios/{id}/convertcurrency,POST v2/portfolios/{id}/convertcurrency
CONVERT_RESULT_CURRENCYPOST v1/analyses/{id}/convertcurrency, POST v2/analyses/{id}/convertcurrency
CONVERT_TREATY_CURRENCYPOST v1/treaties/{id}/convertcurrency
COPY_ACCOUNTPOST v1/accounts/{id}/copy, POST v2/accounts/{id}/copy
COPY_AGGPORTPOST v1/aggregate-porfolios/{id}/copy, POST v2/aggregate-porfolios/{id}/copy
COPY_PORTFOLIOPOST v1/portfolios/{id}/copy, POST v2/portfolios/{id}/copy
CREATE_EDMPOST v1/datasources
DELETE_ACCOUNTDELETE v1/accounts/{id}, DELETE v2/accounts/{id}
DELETE_AGGREGATE_PORTFOLIODELETE v1/aggregate-porfolios/{id}, DELETE v2/aggregate-portfolios/{id}
DELETE_EDMDELETE v1/datasources/{datasourcename}
DELETE_PORTFOLIODELETE v1/portfolios/{id}, DELETE v2/portfolios/{id}
DELETE_RDMPOST v1/exports, POST v2/exports
DEREGISTER_DATA_BRIDGE_EDMDELETE v1/data-bridge/de-register/{datasourcename}, DELETE v2/data-bridge/de-register/{datasourcename}
DLMDEPRECATED POST v1/accounts/{id}/process, POST v2/accounts/{id}/process, DEPRECATED POST v1/portfolios/{id}/process, POST v2/portfolios/{id}/process, POST v1/exposurebatches, POST v2/exposurebatches, POST v1/workflows
DOWNLOAD_EDMPOST v1/datasources
DOWNLOAD_LOCATION_RESULTSPOST v1/exports, POST v2/exports
DOWNLOAD_RDMPOST v1/exports, POST, v2/exports
EDM2EDMPOST v1/exports, POST v2/exports
EDM_UPGRADEPOST v1/datasources
EXPOSURE_BATCHESPOST v1/workflows
EXPOSURE_BATCH_EDITPOST v3/exposurebatches
EXTRACT_ELTPOST v1/analysis-groups, PUT v1/analysis-groups/{id}
GEOCODINGPOST v1/accounts/{id}/geohaz, POST v2/accounts/{id}/geohaz, POST v1/portfolios/{id}/geohaz, POST v2/portfolios/{id}/geohaz, POST v1/exposurebatches, POST v2/exposurebatches
GEOHAZPOST v1/workflows
GROUPINGPOST v1/analysis-groups, POST v1/analysis-groups/{id}/create-independent-group, PUT v1/analysis-groups/{id}, POST v2/analysis-groups, POST v2/analysis-groups/{id}/create-independent-group, PUT v2/analysis-groups/{id}, POST v1/workflows
GROUP_JOBTYPE_REGROUPPOST v1/analysis-groups/{id}/regroup, POST v2/analysis-groups/{id}/regroup
GROUPING_LOSS_POSTPROCESSORPOST v1/analysis-groups, PUT v1/analysis-groups
HAZARDPOST v1/accounts/{id}/geohaz, POST v2/accounts/{id}/geohaz, POST v1/portfolios/{id}/geohaz, POST v2/portfolios/{id}/geohaz, POST v1/exposurebatches, POST v2/exposurebatches
HDDEPRECATED POST v1/accounts/{id}/process, POST v2/accounts/{id}/process, DEPRECATED POST v1/portfolios/{id}/process, POST v2/portfolios/{id}/process, POST v1/exposurebatches, POST v2/exposurebatches
HD_ALMPOST v1/aggregate-portfolios/{id}/process, POST v2/aggregate-portfolios/{id}/process, POST v2/exposurebatches
HD_ALM_GROUPINGPOST v1/aggregate-portfolios/{id}/process, POST v2/aggregate-portfolios/{id}/process, POST v2/exposurebatches
HD_ENGINEDEPRECATED POST v1/accounts/{id}/process, POST v2/accounts/{id}/process, DEPRECATED POST v1/portfolios/{id}/process, POST v2/portfolios/{id}/process, POST v1/exposurebatches, POST v2/exposurebatches
HD_GROUPINGPUT v1/analysis-groups/{id}, POST v1/analysis-groups
HD_TO_RDMDEPRECATED POST v1/exports, POST v2/exports
MAP_PERSECTIVEPOST v1/analyses/{id}/map-perspective, POST v2/analyses/{id}/map-perspective
MARGINAL_IMPACTPOST /v1/analyses/{id}/marginal-impact, POST /v2/analyses/{id}/marginal-impact
MRI_IMPORTPOST v1/imports
PATEPOST v1/workflows, POST v1/analyses/{id}/pate, POST v2/analyses/{id}/pate
PROCESS_RDMPOST v1/uploads/{uploadId}/rdm
REGISTER_DATA_BRIDGE_EDMPOST v2/data-bridge/register
RENAME_ANALYSISPUT /v1/analyses/{id}/rename-analysis, PUT /v2/analyses/{id}/rename-analysis
REPORT_GENERATIONPOST v1/accounts/{id}/summary_report, POST v2/accounts/{id}/summary_report, POST v2/analysis/{id}/summary_report, POST v1/portfolios/{id}/summary_report
RERUNPOST /v1/analyses/{id}/rerun-ep, POST /v2/analyes/{id}/rerun-ep
SIMULATE_LOSSESPOST /V1/analyses/{id}/simulate-losses, POST /v2/analyses/{id}/simulate-losses
SIMULATIONPOST /V1/analyses/{id}/simulate-losses, POST /v2/analyses/{id}/simulate-losses
UDW_POSTPROCESSORPOST v1/accounts/{id}/geohaz
UPDATE_RESULTS_COUNTDELETE v1/analyses, DELETE v1/analyses/{id}, DELETE v2/analyses, DELETE v2/analyses/{id}
UPLOAD_EDMPOST v1/uploads/{uploadId}/edm
UPLOAD_RDMPOST v1/uploads/{uploadId}/rdm

📷

NOTE

  • Some workflow job types may be initiated by multiple operations. For example, v1 and v2 of the Analyze Climate Change operation both initiate a CLIMATE_CHANGE job type.
  • Some operations enable you to initiate multiple types of workflow jobs. For example, the Import Exposures from File operation may initiate either an ALM_IMPORT or MRI_IMPORT job depending on the values specified in the request body.

A Get Workflow or Operation request returns the name, workflow job type, status, and other information about a specific job submitted to the Intelligent Risk Platform.

Custom workflows

A custom workflow is a user-defined workflow that defines a series subprocesses called operations that are processed in order.

The Process custom workflow operation enables you to submit a custom workflow job consisting of multiple operations to the workflow engine for processing.

Custom workflows enable you to define the operations and their sequence with the job. Four types of operations are supported: geohaz, group, process, and batch. Each operation in the workflow is submitted to workflow engine in sequence enabling you to create, geohazard, and process an exposure in a single workflow job. The user-defined workflow manages the both the sequencing of operations within the workflow job and the reporting of operation and workflow status.

Polling workflow job and operation statuses

The Get workflow or operation operation enables you to retrieve information about workflow jobs or operations including the status of the workflow. The operation returns a 202 Accepted response and the ID number of the workflow job.

  • A workflow job is a process that runs on the workflow engine.
  • An operation is a subprocess that runs within a user-defined workflow. Account workflow jobs support geohaz and process operations. Portfolio workflow jobs support geohaz, group, and process operations.

To retrieve information about a workflow job, specify the workflow ID in the endpoint:

curl --request GET \
     --url 'https://{host}/riskmodeler/v1/workflows/5673' \
     --header 'Accept: application/json' \
     --header 'Authorization: XXXXXXXXXX'

To retrieve information about an operation, specify the ID of an operation as a path parameter and the ID of the parent workflow using the parent query parameter:

curl --request GET \
     --url 'https://{host}/riskmodeler/v1/workflows/5673?parent=89993' \
     --header 'Accept: application/json' \
     --header 'Authorization: XXXXXXXXXX'

If a parent workflowId is specified as a query parameter, the id path parameter identifies the requestId of an operation within that workflow.

The response body returns information about the operation in the jobs array:

{
  "userName": "[email protected]",
  "status": "FINISHED",
  "name": "Create Group: New Group from Batch",
  "type": "GROUPING",
  "jobs": [
     {
     "id": "caece2d3-0854-402c-9428-336be01f54c0",
     "status": "Succeeded",
     "name": "GROUPING",
     "input": {
     },
     "output": {
       "groupName": "New Group from Batch",
       "groupDescription": "",
       "analysisId": "68977",
       "currency": "USD",
       "regionPerils": "NAEQ"
       },
       "percentComplete": 100
     }
  ],
  "summary": {
    "analysisId": "68977",
    "groupName": "New Group from Batch",
    "groupDescription": "",
    "regionPerils": "NAEQ",
    "currency": "USD"
  },
  "progress": 100,
  "messages": []
}

The response returns information that will enable you to assess the current status of the specified workflow job or operation.

  • The userName attribute identifies the name of principal, the user or API key that submitted the request.
  • The status attribute indicates the current status of the workflow job. One of QUEUED, RUNNING, FINISHED, FAILED, CANCELLED, PENDING, CANCELLING, or CANCEL_REQUESTED.
  • The name attribute identifies the name of the workflow job. If the workflow job is a standard workflow, the name is determined by the type and the parameters specified in the request. If the workflow job is a custom workflow, the name is specified in the request.
  • The type attribute identifies the workflow job type, e.g. GROUPING, MRI_IMPORT.
  • The jobs attribute defines an array of jobs. Each job object consists of an id, status, name, input, output, and percentComplete.
  • The summary attribute is an object that consists of job-specific data.
  • The progress attribute displays the percent of the job that is complete.
  • The messages attribute is an array of messages.

A successful response returns the Workflow object, which provides detailed information about this workflow job including the submitTime, startTime, type, job details, and its status:

  • QUEUED
  • RUNNING
  • FINISHED
  • FAILED
  • CANCELLED
  • PENDING
  • CANCELLING
  • CANCEL_REQUESTED

Updated 7 months ago