Sensor Data API

The Sensor Data API provides a unified method for retrieving time series sensor data (and related metadata) within iTwin and other platforms. The Sensor Data API will simplify the process of connecting sensor data from sources, including IoT Devices, IaaS IoT Hubs, Enterprise Systems, and other generic repositories. The availability of this data will enable an ecosystem of developers, vendors, and users to develop OT workflows that create new use cases for digital twins in their products.

Sensor Data consists of sequences of numeric values representing measurements from observing sensor data over a specified time. It includes both the current real-time value as well as historical values.

The Two Models

There are two data models to be aware of when it comes to registering a dataSource on Sensor Data Service (SDS).

1. Connection holds all the sensitive information and is only viewable and mutable by SDS developers. It has the following properties:
   • scope – scope needed on token being used to access data source. For example, sensemetrics-sensor-data:read scope needs to exist in the IMS token being sent to Sensemetrics
   • type – type of the connection. One of SELF_SERVICE or EAGLE_IO
   • authType – type of authentication used for the connection. One of NONE, JWT_TOKEN, IMS_TOKEN, HTTP_BASIC, API_KEY, or CLIENT_CREDENTIALS
   • restUrl – connection endpoint where all the REST requests will be forwarded (e.g.: https://www.example.com)
   • wsUrl (Optional) – connection endpoint where all the Web Socket requests will be forwarded (e.g.: wss://www.example.com:1111)
   • authorization (Optional) – API Key, Bearer token, or any other authentication token that is needed to access the connection located in above specified url endpoint
   • username (Optional) – Username needed to access the connection for HTTP_BASIC and CLIENT_CREDENTIALS authTypes
   • password (Optional) – Password needed to access the connection for HTTP_BASIC and CLIENT_CREDENTIALS authTypes
2. DataSource holds all the meta-data and non-sensitive information on a data source. It is viewable and mutable by users. It has the following properties:
   • bucketId - id of bucket
   • bucketName – name of bucket
   • iTwinId – iTwinId associated with the bucket
   • connectionId – id of the connection the data source points to with all the sensitive information (This is provided to the user, read below on how this is generated)

Data Sources

A DataSource is a connection to an existing store of sensor data. It defines a relationship between a specific iTwin and a bucket (collection) of physical sensors and their time series data. Once a DataSource is defined, subsequent calls completely hide the underlying source of data from the calling application.

Connection Types

These are the values that are supported in the type property for a Connection:

• SELF_SERVICE
• EAGLE_IO

Connection Auth Types

These are the values that are supported in the authType property for a Connection:

• IMS_TOKEN: Uses the Authorization Bearer token by the calling client, exchanges that token using a delegation client, then passes this new token to the connection. For example, Sensemetrics supporting IMS tokens.
• HTTP_BASIC: Uses the Authorization: Basic header after Base64 encoding the username/password
• API_KEY: Uses the Authorization ApiKey header
• JWT_TOKEN: Non-expiring JWT token for testing which uses Authorization Bearer header. This is for connections that do not want to give a username/password or need something quick. Renewal may or may not be implemented in the future for this auth type if the connection supports it
• CLIENT_CREDENTIALS: Uses the Authorization Bearer header after obtaining a Client Credentials token from the endpoint of the data source using the given username/password

Connection Property Combinations

When creating a DataSource with a certain type or authType there are certain required properties that must be provided. Here is a list of the currently supported permutations:

Here are the supported type and authType combinations:

Typical DataSource Usage

Creating a Connection and a DataSource is usually a one-time event that does not have to be done at runtime. You can think of it as "setting up a connection" or "registering a connection." This is usually done beforehand as part of a project setup/onboarding process by someone (an admin or customer success team member) who has some knowledge of how to connect to the DataSource and what the actual underlying DataSource supports.

The person creating a Connection and a DataSource typically needs to know the following at a minimum:

• the type of integration to the external application. SELF_SERVICE is the most common
• the host/port of the external application that provides the data
• the type of authentication supported by the external application
• a UUID that represents a bucket (or collection or grouping) of sensors on the external application. In some applications this may be known as a "project" or "site." Different applications have different ways of grouping sensors together.
• a friendly display name for the bucket
• the associated iTwinId that this DataSource represents

Step and actions to register a new Connection

Let's assume Sensemetrics wants to integrate its data sources with SDS

1. A developer from Sensemetrics starts a new conversation on this channel by posting a message "!Register New Connection"
2. Power Automate responds to that message and directly messages the developer from Sensemetrics with a link to a Microsoft Forms where he will have to fill out information pertaining to the Connection model above
3. Once the Sensemetrics developer submits the form, Power Automate creates a work item on Azure DevOps with details from that form titled "[BE] SDS | New Connection Registration Request" and tagged "newconnectionrequest" then assigns it to a SDS developer
4. Once a SDS developer reviews the request, he will generate a connectionId and scope (if authType is IMS_TOKEN) and provide these values to the Sensemetrics developer
5. Finally, the Sensemetrics developer can use the Sensor Data API and connectionId to fill in the details for the DataSource model to register all their data sources
6. For IMS_TOKEN auth type data sources, the IMS token being sent to the data source's restUrl will only have the scope provided to them by SDS developer. Meaning, requests coming from SDS to Sensemetrics will have an IMS token with only the scope sensemetrics-sensor-data:read and Sensemetrics has to validate and allow only incoming requests from SDS having that scope. Any other caller besides SDS should not be able to call Sensemetrics' SDS dedicated endpoints without that scope in their IMS token

Access Control and micro-permissions for iTwins

All Sensor Data API endpoints require that the user have micro-permissions to the associate iTwin. Specifically, if calling read endpoints, the user must have sensor_data_read or sensor_data_manage micro-permission to the associated iTwin. Similarly, if calling modify endpoints, the user must have sensor_data_manage micro-permission to the associated endpoint. For example, if a user is calling a GET operation on /sensors endpoint for an iTwin with id abc-xyz. That user must have sensor_data_read and/or sensor_data_manage micro-permission(s) to that iTwin. This is true for /datasources and /observations endpoints too.

If the required micro-permissions are missing Sensor Data API responds with a 403 response.

All users must first contact the administrator of their iTwin project to have the micro-permissions granted before calling any Sensor Data API endpoints.

Typical Workflow

A typical workflow involves the following:

1. Registering a Connection
2. Registering a DataSource
3. Getting the Sensors for that DataSource
4. Getting Observations for a Sensor

Steps

1. Register a new Connection with steps detailed above and acquire a new connectionId and scope (if applicable) values for your data source registrations

2. Register a new DataSource that maps an iTwinId to a bucketId. Take note of the returned dataSourceId.

Note: Step 1 can be skipped if you already have a dataSourceId that you would like to use.

curl -L -X POST 'https://api.bentley.com/sensor-data/datasources' \
-H 'Authorization: Bearer <changeme>' \
-H 'Content-Type: application/json' \
--data-raw '{
    "bucketId": "<changeme>",
    "bucketName": "Hoover Dam Sensors",
    "iTwinId": "<changeme>",
    "connectionId: "<newly_acquired_connectionId>"
}'

3. Using the dataSourceId returned from Step 1, get the available Sensors for that DataSource. Take note of 1 sensorId and 1 metric and 1 unit for that sensorId.

curl -L -X GET 'https://api.bentley.com/sensor-data/sensors?iTwinId=<changeme>' \
-H 'Authorization: Bearer <changeme>'

4. Get sensor readings for the sensorId obtained in Step 2 for a given metric and unit. Use iTwinId from Step 1. Adjust startDate and endDate using the standard ISO8601 date and time format.

curl -L -g -X GET 'https://api.bentley.com/sensor-data/observations?iTwinId=<changeme>&sensorId=<changeme>&metric=<changeme>&unit=<changeme>&startDate=2011-12-03T10:15:30Z&endDate=2022-09-22T10:15:30Z' \
-H 'Authorization: Bearer <changeme>'

Sample JSON Payloads

Data Sources

Example JSON

{
  "id": "6304AD5893266431E1E9AD71",
  "bucketId": "6304AD5893266431E1E9AD72",
  "bucketName": "My Sensors",
  "iTwinId": "28ba3a8e-bf86-47b6-a37c-6600d9e2e328",
  "connectionId": "641a59f27478902f2f3aabcde"
}

{
  "id": "6304AD5893266431E1E9AD81",
  "bucketId": "vistaDataVisionBucketId",
  "bucketName": "VDV Sensors",
  "iTwinId": "28ba3a8e-bf86-47b6-a37c-6600d9e2e328",
  "connectionId": "641abcde07478902f2f3a4390"
}

{
  "id": "6304AD5893266431E1E9AD91",
  "bucketId": "eagleBucketId",
  "bucketName": "Eagle Sensors",
  "iTwinId": "28ba3a8e-bf86-47b6-a37c-6600d9e2e328",
  "connectionId": "641f6ff27400fa123523a4390"
}

Sensors

Sensors represent the physical sensor gathering data measurements. For example, a voltage sensor that reads voltage.

Example JSON

{
  "id": "sensorId",
  "dataSourceId": "6304AD5893266431E1E9AD71",
  "type": "SELF_SERVICE",
  "bucketId": "6304AD5893266431E1E9AD72",
  "iTwinId": "28ba3a8e-bf86-47b6-a37c-6600d9e2e328",
  "metricsAndUnits": {
    "standard": true,
    "metrics": [
      {
        "metric": "f",
        "displayName": "Frequency",
        "notation": "f",
        "defaultUnit": "Hz",
        "units": [
          {
            "unit": "Hz",
            "displayName": "hertz",
            "notation": "Hz"
          },
          {
            "unit": "digits",
            "displayName": "digits",
            "notation": "digits"
          }
        ]
      },
      {
        "metric": "T",
        "displayName": "Temperature",
        "notation": "T",
        "defaultUnit": "C",
        "units": [
          {
            "unit": "C",
            "displayName": "celsius",
            "notation": "°C"
          },
          {
            "unit": "F",
            "displayName": "fahrenheit",
            "notation": "°F"
          },
          {
            "unit": "K",
            "displayName": "kelvin",
            "notation": "°K"
          }
        ]
      }
    ]
  },
  "metadata": {
    // free-form key-value pair map
  }
}

Observations

Observations are representations of sensor readings. They include a timestamp (epoch) and an IEEE-754 floating point value.

Example JSON

{
  "page": 0,
  "pageSize": 10000,
  "pageTotal": 100,
  "metric": "",
  "unit": "",
  "observations": [
    {
      "timestamp": 1676415254000,
      "value": 100.0
    }
  ]
}