Sensor Data API
The Sensor Data API provides a unified method for retrieving time series sensor data (and related metadata) within iTwin and other platforms. The Sensor Data API will simplify the process of connecting sensor data from sources, including IoT Devices, IaaS IoT Hubs, Enterprise Systems, and other generic repositories. The availability of this data will enable an ecosystem of developers, vendors, and users to develop OT workflows that create new use cases for digital twins in their products.
Sensor Data consists of sequences of numeric values representing measurements from observing sensor data over a specified time. It includes both the current real-time value as well as historical values.
The Two Models
There are two data models to be aware of when it comes to registering a dataSource on Sensor Data Service (SDS).
- Connection holds all the sensitive information and is only viewable and mutable by SDS developers. It has the following properties:
- scope – scope needed on token being used to access data source. For example,
sensemetrics-sensor-data:read
scope needs to exist in the IMS token being sent to Sensemetrics - type – type of the connection. One of
SELF_SERVICE
orEAGLE_IO
- authType – type of authentication used for the connection. One of
NONE
,JWT_TOKEN
,IMS_TOKEN
,HTTP_BASIC
,API_KEY
, orCLIENT_CREDENTIALS
- restUrl – connection endpoint where all the REST requests will be forwarded (e.g.: https://www.example.com)
- wsUrl (Optional) – connection endpoint where all the Web Socket requests will be forwarded (e.g.: wss://www.example.com:1111)
- authorization (Optional) – API Key, Bearer token, or any other authentication token that is needed to access the connection located in above specified url endpoint
- username (Optional) – Username needed to access the connection for
HTTP_BASIC
andCLIENT_CREDENTIALS
authTypes - password (Optional) – Password needed to access the connection for
HTTP_BASIC
andCLIENT_CREDENTIALS
authTypes
- scope – scope needed on token being used to access data source. For example,
- DataSource holds all the meta-data and non-sensitive information on a data source. It is viewable and mutable by users. It has the following properties:
- bucketId - id of bucket
- bucketName – name of bucket
- iTwinId – iTwinId associated with the bucket
- connectionId – id of the connection the data source points to with all the sensitive information (This is provided to the user, read below on how this is generated)
Data Sources
A DataSource is a connection to an existing store of sensor data. It defines a relationship between a specific iTwin and a bucket (collection) of physical sensors and their time series data. Once a DataSource is defined, subsequent calls completely hide the underlying source of data from the calling application.
Connection Types
These are the values that are supported in the type
property for a Connection:
- SELF_SERVICE
- EAGLE_IO
Connection Auth Types
These are the values that are supported in the authType
property for a Connection:
- IMS_TOKEN: Uses the Authorization Bearer token by the calling client, exchanges that token using a delegation client, then passes this new token to the connection. For example, Sensemetrics supporting IMS tokens.
- HTTP_BASIC: Uses the Authorization: Basic header after Base64 encoding the username/password
- API_KEY: Uses the Authorization ApiKey
header - JWT_TOKEN: Non-expiring JWT token for testing which uses Authorization Bearer header. This is for connections that do not want to give a username/password or need something quick. Renewal may or may not be implemented in the future for this auth type if the connection supports it
- CLIENT_CREDENTIALS: Uses the Authorization Bearer header after obtaining a Client Credentials token from the endpoint of the data source using the given username/password
Connection Property Combinations
When creating a DataSource with a certain type
or authType
there are certain required properties that must be provided. Here is a list of the currently supported permutations:
Here are the supported type
and authType
combinations:
Typical DataSource Usage
Creating a Connection and a DataSource is usually a one-time event that does not have to be done at runtime. You can think of it as "setting up a connection" or "registering a connection." This is usually done beforehand as part of a project setup/onboarding process by someone (an admin or customer success team member) who has some knowledge of how to connect to the DataSource and what the actual underlying DataSource supports.
The person creating a Connection and a DataSource typically needs to know the following at a minimum:
- the type of integration to the external application. SELF_SERVICE is the most common
- the host/port of the external application that provides the data
- the type of authentication supported by the external application
- a UUID that represents a bucket (or collection or grouping) of sensors on the external application. In some applications this may be known as a "project" or "site." Different applications have different ways of grouping sensors together.
- a friendly display name for the bucket
- the associated iTwinId that this DataSource represents
Step and actions to register a new Connection
Let's assume Sensemetrics wants to integrate its data sources with SDS
- A developer from Sensemetrics starts a new conversation on this channel by posting a message "!Register New Connection"
- Power Automate responds to that message and directly messages the developer from Sensemetrics with a link to a Microsoft Forms where he will have to fill out information pertaining to the Connection model above
- Once the Sensemetrics developer submits the form, Power Automate creates a work item on Azure DevOps with details from that form titled "[BE] SDS | New Connection Registration Request" and tagged "newconnectionrequest" then assigns it to a SDS developer
- Once a SDS developer reviews the request, he will generate a
connectionId
andscope
(if authType isIMS_TOKEN
) and provide these values to the Sensemetrics developer - Finally, the Sensemetrics developer can use the Sensor Data API and
connectionId
to fill in the details for the DataSource model to register all their data sources - For
IMS_TOKEN
auth type data sources, the IMS token being sent to the data source's restUrl will only have the scope provided to them by SDS developer. Meaning, requests coming from SDS to Sensemetrics will have an IMS token with only the scopesensemetrics-sensor-data:read
and Sensemetrics has to validate and allow only incoming requests from SDS having that scope. Any other caller besides SDS should not be able to call Sensemetrics' SDS dedicated endpoints without that scope in their IMS token
Access Control and micro-permissions for iTwins
All Sensor Data API endpoints require that the user have micro-permissions to the associate iTwin. Specifically, if calling read endpoints, the user must have sensor_data_read
or sensor_data_manage
micro-permission to the associated iTwin. Similarly, if calling modify endpoints, the user must have sensor_data_manage
micro-permission to the associated endpoint. For example, if a user is calling a GET operation on /sensors
endpoint for an iTwin with id abc-xyz
. That user must have sensor_data_read
and/or sensor_data_manage
micro-permission(s) to that iTwin. This is true for /datasources
and /observations
endpoints too.
If the required micro-permissions are missing Sensor Data API responds with a 403
response.
All users must first contact the administrator of their iTwin project to have the micro-permissions granted before calling any Sensor Data API endpoints.
Typical Workflow
A typical workflow involves the following:
- Registering a Connection
- Registering a DataSource
- Getting the Sensors for that DataSource
- Getting Observations for a Sensor
Steps
Register a new Connection with steps detailed above and acquire a new
connectionId
andscope
(if applicable) values for your data source registrationsRegister a new DataSource that maps an iTwinId to a bucketId. Take note of the returned dataSourceId.
Note: Step 1 can be skipped if you already have a dataSourceId that you would like to use.
curl -L -X POST 'https://api.bentley.com/sensor-data/datasources' \
-H 'Authorization: Bearer <changeme>' \
-H 'Content-Type: application/json' \
--data-raw '{
"bucketId": "<changeme>",
"bucketName": "Hoover Dam Sensors",
"iTwinId": "<changeme>",
"connectionId: "<newly_acquired_connectionId>"
}'
- Using the dataSourceId returned from Step 1, get the available Sensors for that DataSource. Take note of 1 sensorId and 1 metric and 1 unit for that sensorId.
curl -L -X GET 'https://api.bentley.com/sensor-data/sensors?iTwinId=<changeme>' \
-H 'Authorization: Bearer <changeme>'
- Get sensor readings for the sensorId obtained in Step 2 for a given metric and unit. Use iTwinId from Step 1. Adjust startDate and endDate using the standard ISO8601 date and time format.
curl -L -g -X GET 'https://api.bentley.com/sensor-data/observations?iTwinId=<changeme>&sensorId=<changeme>&metric=<changeme>&unit=<changeme>&startDate=2011-12-03T10:15:30Z&endDate=2022-09-22T10:15:30Z' \
-H 'Authorization: Bearer <changeme>'
Sample JSON Payloads
Data Sources
Example JSON
{
"id": "6304AD5893266431E1E9AD71",
"bucketId": "6304AD5893266431E1E9AD72",
"bucketName": "My Sensors",
"iTwinId": "28ba3a8e-bf86-47b6-a37c-6600d9e2e328",
"connectionId": "641a59f27478902f2f3aabcde"
}
{
"id": "6304AD5893266431E1E9AD81",
"bucketId": "vistaDataVisionBucketId",
"bucketName": "VDV Sensors",
"iTwinId": "28ba3a8e-bf86-47b6-a37c-6600d9e2e328",
"connectionId": "641abcde07478902f2f3a4390"
}
{
"id": "6304AD5893266431E1E9AD91",
"bucketId": "eagleBucketId",
"bucketName": "Eagle Sensors",
"iTwinId": "28ba3a8e-bf86-47b6-a37c-6600d9e2e328",
"connectionId": "641f6ff27400fa123523a4390"
}
Sensors
Sensors represent the physical sensor gathering data measurements. For example, a voltage sensor that reads voltage.
Example JSON
{
"id": "sensorId",
"dataSourceId": "6304AD5893266431E1E9AD71",
"type": "SELF_SERVICE",
"bucketId": "6304AD5893266431E1E9AD72",
"iTwinId": "28ba3a8e-bf86-47b6-a37c-6600d9e2e328",
"metricsAndUnits": {
"standard": true,
"metrics": [
{
"metric": "f",
"displayName": "Frequency",
"notation": "f",
"defaultUnit": "Hz",
"units": [
{
"unit": "Hz",
"displayName": "hertz",
"notation": "Hz"
},
{
"unit": "digits",
"displayName": "digits",
"notation": "digits"
}
]
},
{
"metric": "T",
"displayName": "Temperature",
"notation": "T",
"defaultUnit": "C",
"units": [
{
"unit": "C",
"displayName": "celsius",
"notation": "°C"
},
{
"unit": "F",
"displayName": "fahrenheit",
"notation": "°F"
},
{
"unit": "K",
"displayName": "kelvin",
"notation": "°K"
}
]
}
]
},
"metadata": {
// free-form key-value pair map
}
}
Observations
Observations are representations of sensor readings. They include a timestamp (epoch) and an IEEE-754 floating point value.
Example JSON
{
"page": 0,
"pageSize": 10000,
"pageTotal": 100,
"metric": "",
"unit": "",
"observations": [
{
"timestamp": 1676415254000,
"value": 100.0
}
]
}