Overview of the Data Extraction API’s

Overview of the Data Extraction API’s

This article covers:

• Overview of the Data Extraction API’s
• Setting up the API
• API key management  
• Set up data requirements via your own integration tools
• Hourly Data Extract Format
• IQ Data Extraction Format
• Module Data Extraction Format
• Amendments to Module Extracts
• API Schema
• FAQs

Overview of the Data Extraction API’s

Assure’s data extraction API (Application Programming Interface) allows you to securely and automatically extract your information from Assure into another application, your company’s business intelligence tools, so you can analyse and publish your safety data alongside other company information. Once set up, no user intervention is required.  
  
The Assure API can be accessed via the internet as a secure RESTful API which is widely supported by integration platforms, ETL tools and all programming and scripting languages, so it can be easily set up to integrate with external systems.  
  
Please contact your CDM (Customer Development Manager) or CSM (Customer Success Manager) should you be interested in procuring this feature.

Setting up the API

Once requested Evotix will enable the API and set up your data extraction requirements

The following elements can be defined and must be provided by the customer. 

• Output format can be made available in JSON (for JSONLines), CSV format and PARQUET format
• Module records you want to include in your data extraction. 

Check out this article for details. (Data Extract Article)

API key management   
Customers can then create the API Keys to access the API so you control access to your data and can remove this at any time. Check out this article for details (API Key Management Article)   
Up to 30 API Keys can be in use concurrently, and these can be added and removed at any time in line with your IT or security policies. 

Set up data requirements via your own integration tools

As a final step, you need to set up the data request in your own integration tool. As this is customer specific, our team cannot provide support for this step. This step and the previous step should be supported by your IT team.  
You will need to use the correct request URL to identify the dataset you need. The following datasets are available. To use these URLs, replace <stack> with the Assure stack name which you use. This will be the same as the stack name you use to access the Assure Web UI. For example, if you access the Assure Web UI with https://uk2.sheassure.net/ then your <stack> value is uk2.

Hourly Data Extract Format

Hourly data extracts are generated every hour and include all newly created or updated data from midnight (the time of the last daily extract) up to the previous hour.

The URL for accessing the extract file remains the same as the daily file, for both Module and IQ data. To receive the hourly extract, you must include the following header and value in your request:

x-extract-schedule: hourly

Please note: 

• If this header is not present, you will receive the daily extract.
• If the header is present but the value is set to daily, you will also receive the daily extract.
• Only requests with the header value set to hourly will receive the hourly updates.
• The setting must also be enabled in the Assure UI, please refer to this knowledgebase article for further information

Availability: 
Hourly extract files will be available by the 30-minute mark past the hour (e.g. 1:30, 2:30, etc.). Each file includes data up to the start of the hour (e.g. the 1:30 file includes data up to 1:00).

Make sure your system is configured to retrieve the file after it becomes available.

IQ Data Extraction Format For IQ data and IQ Additional fields

In order to access the complete IQ data for each of the modules, please add "/iq" to the end of the URL for that module. E.g.  incident-event/iq

To access the IQ additional fields, please add “/iq-additional-fields” to the end of the URL for that module. E.g. incident-event/iq-additional-fields

Module Data Extraction Format

To efficiently support larger module data extracts, the export format for Data Extracts has been updated from JSON to JSON Lines.

If you are using JSON lines to export data, any empty field in the record in Assure (e.g. a name field that has no text) will be omitted from the output.

Once set up your own integration tools will request data from the Assure Customer API (1). After authentication, the data set that you have defined in step 2 will be retrieved (2) and returned to you in the defined format.

Amendments to Module Extracts

Additions to the module extracts are sometimes required to add additional fields in as new functionality is delivered. Updates made to the released extracts can be found here.

API Schema

A comprehensive OpenAPI schema which details all the available datasets for extraction is available here - https://github.com/Evotix/public-assure-customerapi. This OpenAPI schema is automatically updated as new modules become available for extraction. All Assure record fields will be included in each data extract. To ensure a standard data structure that is not impacted by configuration changes to captions, the columns fields are standardised based on the default Assure captions, not the customer configured captions. A CSV export is available via the Assure UI which contains a mapping between the API field names and the captions displayed in the Assure UI to aid with understanding how the fields in the extracts related to the fields in the UI.  

FAQ’S

What data is included?  
All records are included in one call, including any iQ form data that the customer chooses to include.  
Each Dataset includes all the records and fields held in Assure for the selected module so you can perform any filtering and analysis required in your own BI tools.  
For iQ template data, you can control what is included or you can export all module data by adding ‘/iq’ to the end of the URL. (Data Extract Article)

What terminology and structure should I use?  
Assure default terminology and fieldnames are used in the dataset. A field mapping sheet is available to download which will show the dataset field names and their corresponding Assure captions. (Data Extract Article)

Why are my iQ fields not coming through?  
The iQ template question needs a Data Extract field export code to be set to enable iQ data for that question to be included in the module export. This must be set on each iQ question or it will not be included. Please ensure that a standard naming convention is used here to ensure simplicity. 
The Data Extract export code does not need to be set for data to be included in the complete IQ data export for each module. 

Which Approvals are included in my Extract? 

Approvals are only included within the Approvals Extract where the module has been enabled for extraction. If a module has not been enabled, then the Approvals for that module will be excluded from the Approvals Extract.

Which Actions are included in my Extract? 
All Actions are included with the Actions Extract, regardless of the Action's module being enabled for extraction.

 Endpoint fields in Actions Extract 
The Actions extract contains two endpoint fields - actionModuleEndpoint and actionSubModuleEndpoint. 
These fields have two main purposes:

• A static module reference that is not modified by the caption of the module's name.
• The endpoint reference that you can use to get the respective module records.

As the Actions Extract contains all of the configured Actions, some of these endpoints may not be activated by Evotix yet. Please refer to the list of modules above for the latest list of available endpoints. 
Additionally and in case the module in question is in the list, you may need to enable it for extraction. More information can be found in this article: Setting up your API Data Extraction requirements.

How are Time Zones handled in the Data Extracts? 

In Assure, there are some modules that use a date field and a time field in the module record. Where a date field and time field is used in Assure, these two fields are merged to become one single field in the Data Extract. 

The time field can be:

• Completed, including those that have been set to 00:00(AM) or;
• hidden (and not used), and is therefore marked as 00:00(AM) by Assure in the Data Extracts. 

To help you identify the values that you can convert to another time zone, we have included a column  (e.g. hazardSpottedAtType or incidentOccurredAtType) which contains 2 possible values:

• UTCTIME - This value indicates that this date-time field is in UTC time zone, since there is a `time` component in the record and thus, it can be safely converted to another time zone. This applies to the completed time fields, including those that have been set to 00:00(AM). 
• DATEONLY - This value indicates that this date-time field did not have a `time` component recorded, therefore it should not be converted to another time zone. This applies where time fields have not been completed or are hidden. 

Why am I seeing differing dates in my data extract?

Some fields in your data extract may have dates in them that do not line up in a correct order, for example the module record creation date (createdAtUTC), the iQ Questionnaire start date (iqQuestionnaireStartedAtUTC) and the iQ Questionnaire completion date(iqQuestionnaireCompletedAtUTC). 

If these records were raised through AssureGo+ there can be inconsistencies in these dates, this is because AssureGo+ does not create the module record until it reaches Assure. This can happen whether the record has been auto-processed or has been in the portal queue.

Why can I not see all data in my hourly extracts?

Hourly extracts are generated every hour and include all newly created or modified data from the last daily extract up to the previous hour.

I set up Hourly extracts, why am I not seeing the updates I have made?

If you're not seeing the hourly updates, it’s likely due to one of the following reasons:

1. Missing or incorrect header:
   • To receive hourly extracts, you must include the header: x-extract-schedule: hourly
   • If this header is missing, or if it’s set to daily instead of hourly, the system will return the daily extract only.
2. Request timing:
   • Hourly files become available 30 minutes past each hour (e.g. 1:30, 2:30, etc.). 
3. The setting must also be enabled in Assure or you will receive  404 error. Details on how to set this up are here. 

Double-check your request headers and timing and ensure your retrieval setup matches the expected file availability schedule.