Upload Mapping File
Map relations between uploaded account, location, and reinsurance data and EDM database fields
Overview
The MRI import process relies on an import mapping file to match columns of data in the uploaded source files to EDM database columns. The import mapping file enables the operation to ensure that structured data in the source file is imported into the correct destination field in the EDM.
The import mapping file also enables you to specify which properties will be imported into the EDM from the uploaded account, location, and reinsurance source files.
Step 5.1: Create import mapping file
An import mapping file defines the relationships between account, location and reinsurance data and between columns in the uploaded account, location, and reinsurance resource files and the corresponding columns in the EDM.
The MRI validator does not distinguish between the different language conventions, and may produce erroneous warning messages as a result. Warning messages do not impact the accuracy of the import job, which can still complete successfully.
The mapping file must be in the MIDI file format (MFF
).
Step 5.2: Get S3 path and credentials
The Get S3 path and credentials enables you to fetch the path to an S3 bucket and temporary security credentials that will enable you to upload an flat file of exposure data.
The operation requires that you specify the bucketId
of an Amazon S3 bucket as a path parameter.
curl --location --request POST 'https://{host}/riskmodeler/v1/storage/{{bucketId}}/path' \
--header 'Content-Type: application/json' \
--header 'Authorization: {api_key}'
The request package identifies the fileInputType
, fileName
, fileSize
, and fileType
.
{
"fileInputType": "MRI",
"fileType": "mapping",
"fileSize": 9105,
"fileName": "mapping.txt"
}
The fileType
and fileName
attributes are required.
- The
fileInputType
attribute identifies the job type (ALM
orMRI
). - The
fileName
attribute specifies the name of the flat file to be uploaded. - The
fileSize
attribute specifies the size of the flat file in kilobytes. - The
fileType
attribute specifies the type of data contained in the flat file. One ofaccount
(account data),risk
(location data),reins
(reinsurance data), ormapping
(mapping data).
If successful, the response returns a 201
status code and base64 encoded temporary security credentials from the AWS Security Token Service.
Step 5.3: Upload mapping file to S3
The RMS Risk Modeler API does not provide a custom operation for uploading flat file data to S3 buckets. You must use the AWS APIs to manage this process.
In this procedure, you will use the S3 bucket path and temporary user credentials to upload account data to the S3 bucket.
First, you must decode to the accessKeyId
, secretAccessKey
, sessionToken
, and s3Path
values and pass the decoded values to a S3 client. The sample code is in Java 8.
private static String base64Decode(String text) {
return new String(Base64.getDecoder().decode(text));
}
Pass the decoded accessKeyId
, secretAccessKey
, and sessionToken
to the Amazon getS3Client()
method to create an Amazon S3 client.
private static AmazonS3 getS3Client(String accessKey, String secretKey, String sessionToken){
BasicSessionCredentials sessionCredentials = new BasicSessionCredentials(
accessKey,
secretKey,
sessionToken);
return AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(sessionCredentials))
.withRegion(Regions.EU_WEST_1)
.build();
}
Amazon TransferManager is a high-level utility for managing transfers to Amazon S3 that makes extensive use of Amazon S3 multipart uploads.
Once you have the Amazon S3 client, you can pass the the s3Client, bucketName
, key
, and the filePath
to the TransferManager
.
private static void upload(AmazonS3 s3Client, String bucketName, String key, String filePath) {
try {
TransferManager tm = TransferManagerBuilder.standard()
.withS3Client(s3Client)
.build();
// TransferManager processes all transfers asynchronously,
// so this call returns immediately.
Upload upload = tm.upload(bucketName, key, new File(filePath));
System.out.println("Object upload started");
// Optionally, wait for the upload to finish before continuing.
upload.waitForCompletion();
System.out.println("Object upload complete");
}catch( Exception ex){
System.out.println(ex.getMessage());
}
}
The parameters are derived from previous steps:
- The
bucketName
can be extracted from the the initial section of the decoded s3Path. If thes3Path
isrms-mi/preview/tenant/50000/import/mri/3929
, thebucketName
is rms-mi. - The
key
combines the remaining portion of thes3Path
with thefileId
,fileName
in the following pattern:s3Path/fileId-fileName
. For example, preview/tenant/50000/import/mri/3929/12373-fileName. - The
filePath
specifies the absolute path to flat file you want to upload.
Updated 11 months ago