Device Schema Standard

This page details the standards any SMDH device schema must follow in order to work with the MDEP device registration API system. Not following this standard will cause problems with streamed message ingestion into MDEP.

What is a JSON schema?

A JSON schema is a method of defining what structure a JSON object must conform to. For further information please read this guide.

Why does MDEP require every device to provide a JSON schema?

In MDEP we use these schemas to understand the MQTT message data before it arrives and create the correct data storage locations with the correct layout beforehand. Without JSON schemas, automation and data standardisation would be impossible.

By asking for a JSON schema we also ensure the streamed data ingested into MDEP is of a high quality for the purposes of data science.

Schema Standard

We have built a custom standard which is built of JSON_schema_draft-7. This page will only detail any changes we have made to draft 7 or any items which will affect how you create a schema for a SMDH device.

General structure

Below is the general boilerplate JSON schema structure (You can copy this as the starting point for your schema):

{
    "title": "",
    "$schema": "http://json-schema.org/draft-07/schema#",
    "description": "",
    "type": "object",
    "properties": {},
    "required": []`
}

The following is a description for each field:

• title

  This is the most important field and consistent reuse is encouraged. Give a name for your json schema, this should be on a per message type basis example “Temperature”, “PowerMonitoring”.

  Restrictions

  Max character length of 60 characters.

  Tip

  Different sensor types can support the same schema - consider different temperature monitoring sensor manufacturers supporting the same SMDH schema standard.

  Automatic schema versioning

  The title should not change for any future version you may make of this schema - regard it as a static lookup value. The title should not contain any versioning values within the title’s string value. For example, if you created a new version of your sensor with some field changes but it's still the same type of sensor, the “title” field in your JSON schema should still not be changed. We automatically version schemas based on changes to the other fields of the schema.

  The only time you should change this field is if the device is measuring something completely different.

  If you are making a new schema for a new device type, make sure it has a different title value than other schemas you have defined for other device types.

• $schema

  Keep as set in example above, it allows us to know the schema type.

• description

  Tell us about this schema. What is the device type. Any additional information you think is important.

  Tip

  This field can be helpful to provide context for the schema and to help you identify it.

• type

  Keep as object.

• properties

  See the properties section below.

• required

  Please only complete this after completing the properties section. For more information, see the required section below.

Properties

The properties section defines the format of the sensor data to be sent to MDEP. We recommend you gain a reasonable understanding of JSON before beginning this section. A good source of information can be found here. Keys We will start with defining the JSON keys that will be included in the JSON object sent by the device. Ordering of keys is not significant. Key value pairs are nested within the “properties” key as shown below (\<Key> represents a placeholder):

"properties": {
    "<Key>": {},
    "<Key>": {},
    ...
}

The placeholder <Key> should be replaced with an actual key name that is specified in the corresponding schema document.

Example

If we take an example MQTT message a device might send:

Example message

{
    "current": 0.3,
    "power": 23,
    "time": "800202180"
}

We look for the keys, in this example “current”, “power” and “time” and we add them to our properties section

Schema

"properties": {
    "current": {},
    "power": {},
    "time": {}
}

Values

This section will take you through how to define the value of each key you have created in the keys section. This section is split based on the value's data type.

• String
• Stringed Integer
• Stringed float
• Stringed decimal
• Stringed boolean
• Integer
• Decimal
• Date/Time/Timestamp

If your data type is not in this list or doesn't fit with any current data type please contact support@smdh.uk.

String

A string can be a combination of one or more utf-8 characters, with a maximum byte size of 256. Bytes can be worked out as 4 bytes (headers) + total bytes for characters, where each character could be 1 to 4 bytes in size.

Below is an example of a string being defined

Template

"<key>": {
    "type": "string"
}

Key	Value
type	Keep as “string”

You can also set a regular expression (regex) which will allow you to precisely define the string format (allowed sequences of characters). This is not required but will help us tell you if sensors are sending incorrect data.

Template

"<key>": {
    "type": "string",
    "pattern": "<regex>"
}

Key	Value
type	Keep as “string”
pattern (optional)	Input your regex

Example

"email": {
    "type": "string",
    "pattern": "/^([a-z0-9_\.-]+)@([\da-z\.-]+)\.([a-z\.]{2,5})$/"
}

If you require more than 256 bytes you can use the “maxlength” key and this can be a maximum of 6000.

Template

"<key>": {
    "type": "string",
    "pattern": "<regex>",
    “maxlength”: <1-6000>
}

Key	Value
type	Keep as “string”
pattern (not required, i.e. optional)	Input your regex
maxlength (not required, i.e. optional)	A number between 1 and 6000 for the max number of bytes of the string. If this isn't inputted we automatically set the maxlength at 256.

Example

"email": {
    "type": "string",
    "pattern": "/^([a-z0-9_\.-]+)@([\da-z\.-]+)\.([a-z\.]{2,5})$/"
    "maxlength": 300
}

Stringed Integer

This is a string containing an integer. This was added due to some MQTT systems only allowing strings to be sent.

Warning

Please try and use normal integers first, only if that fails should you use this option.

Your stringed integer should follow all requirements set out in the integer section.

Template

"<key>": {
    "type": "string",
    "pattern": "<regex>",
    "mvDType": "int"
}

Key	Value
type	Keep as “string”
pattern (not required, i.e. optional)	Input your regex,
mvDType	Keep as “int”

Stringed Float

This is a string containing a float. This was added due to some MQTT systems only allowing strings to be sent.

Float represents the double-precision floating point format, according to the IEEE Standard 754 for Binary Floating-Point Arithmetic. It has a precision of about 15 digits, and a range of around 1E-307 to 1E+308.

Warning

Please try and use normal floats first, only if that fails should you use this option.

Your stringed float should follow all requirements set out in the float section.

Template

"<key>": {
    "type": "string",
    "pattern": "<regex>",
    "mvDType": "float"
}

Key	Value
type	Keep as “string”
pattern (not required i.e. optional)	Input your regex,
mvDType	Keep as “float”

Stringed decimal

This is a string containing a decimal. This was added due to some MQTT systems only allowing strings to be sent.

Warning

Please try and use normal decimal first, only if that fails should you use this option.

Your stringed decimal should follow all requirements set out in the decimal section.

Template

"<key>": {
    "type": "string",
    "pattern": "<regex>",
    "mvDType": "decimal"
}

Key	Value
type	Keep as “string”
pattern (not required i.e. optional)	Input your regex,
mvDType	Keep as “decimal”

Stringed Boolean

This is a string that is a boolean. This was added due to some MQTT systems only allowing strings to be sent.

Your boolean must be a string and allowed values are “0” or “1”.

Template

"<key>": {
    "type": "string",
    "enum": ["1","0"],
    "mvDType": "bool"
}

Key	Value
type	Keep as “string”
enum	Keep as set above
mvDType	Keep as “decimal”

Integer

An integer has to be a number between -2147483648 to +2147483647.

Template

"<key>": {
    "type": "integer"
}

Key	Value
type	Keep as “integer”

Decimal

A decimal has to be 128-bit signed integers with up to 38 digits of precision.

Template

"<key>": {
    "type": "number"
}

Date/Time/Timestamp

If possible, we strongly urge the use of ISO8601 in UTC for timestamps using the following pattern (YYYY-MM-DDTHH:mm:ss.mmm ±offset). Other ISO8601 formats are possible, but we recommend this pattern for consistency.

Example

In the example below, an optional decimal fraction (denoted by a comma or dot) has been added to the lowest order time element. There is no limit on the number of decimal places for the decimal fraction. The optional offset is a duration measured from UTC. Offset has the format ±hh:mm (a plus or minus sign followed by hours : mins). Therefore, if the timestamp is already in UTC, the offset should be omitted or be +00:00 (no deviation from UTC).

2024-04-08T21:10:12.438031+00:00

Given the wide variety of sensor types and the variable formats we have encountered, it was not been possible to standardise on a particular format or set of formats.

Template

"<key>": {
    "type": "string",
    "format": "date-time"
}

Example

Below you can find an example properties section:

"properties": {
    "time": {
        "type": "string",
        "format": "date-time"
    },
    "machine": { "type": "string" },
    "current": {
        "type": "string",
        "pattern": "^([0-9]|[1-9][0-9]|1[0-9][0-9]|200)$",
        "mvDType": "int"
    },
    "power": {
        "type": "string",
        "pattern": "^(0|[1-9][0-9]{0,4}|1[0-9][0-9]{0,4}|200000)$",
        "mvDType": "int"
    }
}

What does this properties section define?

It states that the MQTT message will contain four items (keys) “time”, “machine”, “current” and “power” and that the:

• “time” value will be a string in a date-time format
• “machine” value will be a string
• “current” value will be a string but must conform to a regex pattern and the data type we should store this as should be an int (mvDType)
• “power” value will be a string but must conform to a regex pattern and the data type we should store this as should be an int (mvDType)

Required

The “required” section is where you state which of the “properties” keys are required. A key given in the “properties” section but not in the “required” section is optional. Validation will fail if required properties are omitted. Ordering of required keys is not significant.

Example

"required": [
    "<key>",
    "<key>",
    ...
]

Example Messages & Schemas

Example 1 - Temperature Sensor

MessagesSchema

Below are five example messages from a temperature sensor.

Messages

{
    "time": "2024-04-08T21:10:08.702860+00:00",
    "temp": "11.6875",
    "Threshold": "50",
    "AlertVal": "0",
}
{
    "time": "2024-04-08T21:10:10.552214+00:00",
    "temp": "11.6934",
    "Threshold": "50",
    "AlertVal": "0",
}
{
    "time": "2024-04-08T21:10:12.438031+00:00",
    "temp": "11.7003",
    "Threshold": "50",
    "AlertVal": "0",
}
{
    "time": "2024-04-08T21:10:14.290971+00:00",
    "temp": "11.6938",
    "Threshold": "50",
    "AlertVal": "0",
}
{
    "time": "2024-04-08T21:10:16.140404+00:00",
    "temp": "11.6919",
    "Threshold": "50",
    "AlertVal": "0",
}

Schema

{
    "title": "temperature sensor",
    "$schema": "http://json-schema.org/draft-07/schema#",
    "description": "A representation of temperature sensor data",
    "type": "object",
    "properties": {
        "temp": {
            "type": "string",
            "pattern": "^-?\\d{1,3}(\\.\\d{0,4})?$",
            "mvDType": "float"
        },
        "AlertVal": {
            "type": "string",
            "enum": ["1", "0"],
            "mvDType": "bool"
        },
        "Threshold": {
            "type": "string",
            "pattern": "^-?\\d{1,3}(\\.\\d{0,4})?$",
            "mvDType": "float"
        },
        "time": {
            "type": "string",
            "format": "date-time"
        }
    },
    "required": [
        "temp",
        "AlertVal",
        "Threshold",
        "time"
    ]
}

Example 2 - Air Quality Sensor (integer)

MessagesSchema

Below are five example messages from an air quality sensor.

Messages

{
    "TVOC": 76,
    "CO2": 507,
    "AQI": 2,
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.184812+00:00",
}
{
    "TVOC": 76,
    "CO2": 507,
    "AQI": 2,
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.384886+00:00",
}
{
    "TVOC": 76,
    "CO2": 507,
    "AQI": 2,
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.584882+00:00",
}
{
    "TVOC": 76,
    "CO2": 507,
    "AQI": 2,
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.784880+00:00",
}
{
    "TVOC": 70,
    "CO2": 495,
    "AQI": 2,
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.984890+00:00",
}

Schema

{
    "title": "airquality sensor",
    "$schema": "<http://json-schema.org/draft-07/schema#>",
    "description": "A representation of airquality sensor data",
    "type": "object",
    "properties": {
        "TVOC": {"type": "integer"},
        "CO2": {"type": "integer"},
        "AQI": {"type": "integer"},
        "machine": {"type": "string"},
        "time": {
            "type": "string",
            "format": "date-time"
        }
    },
    "required": [
        "TVOC",
        "CO2",
        "AQI",
        "machine",
        "time"
    ]
}

Example 3 - Air Quality Sensor (stringed integer)

MessagesSchema

Below are five example messages from an air quality sensor.

Messages

{
    "TVOC": "76",
    "CO2": "507",
    "AQI": "2",
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.184812+00:00",
}
{
    "TVOC": "76",
    "CO2": "507",
    "AQI": "2",
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.384886+00:00",
}
{
    "TVOC": "76",
    "CO2": "507",
    "AQI": "2",
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.584882+00:00",
}
{
    "TVOC": "76",
    "CO2": "507",
    "AQI": "2",
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.784880+00:00",
}
{
    "TVOC": "70",
    "CO2": "495",
    "AQI": "2",
    "machine": "Machine_1",
    "time": "2024-03-26T07:27:34.984890+00:00",
}

Schema

{
    "title": "airquality sensor",
    "$schema": "<http://json-schema.org/draft-07/schema#>",
    "description": "A representation of airquality sensor data",
    "type": "object",
    "properties": {
        "TVOC": {
            "type": "string",
            "mvDType": "int"
        },
        "CO2": {
            "type": "string",
            "mvDType": "int"
        },
        "AQI": {
            "type": "string",
            "mvDType": "int"
        },
        "machine": {
            "type": "string"
        },
        "time": {
            "type": "string",
            "format": "date-time"
        },
    },
    "required": [
        "TVOC",
        "CO2",
        "AQI",
        "machine",
        "time",
    ]
}