Leveraging the TravelTime Extension for Reproducibility and Data Lineage

STA API time-travel extension

This extension assists istSTA users in accessing historical time travel data. It enables users to retrieve information from a web service as it appeared at a specific moment in time, using a new query parameter called as_of.

Additionally, it introduces a new entity called Commit, which enables data lineage, allowing users to trace data changes. From a scientific perspective, this extension enables FAIR data management by allowing datasets to be permanently cited. This is achieved by using a combination of the service address (in red), the request that returns the dataset (in green), and the dataset's status at a specific time instant (in orange) as a Persistent Identifier for reference.

Example: https://<base_url>/<version>/<entity>?$expand=<entity>&$as_of=<date_time>

Definition

The time-travel extension adds the following optional query parameters to any STA request:

Parameter	Type	Description
as_of	ISO 8601 date-time	a date-time parameter to specify the exact moment for which the data is requested
from_to	ISO 8601 period	a period parameter to specify the time interval for which the data is requested

The time-travel extension introduces a new entity, Commit, with the following properties:

Properties	Type	Multiplicity and use	Description
author	string(128)	One (mandatory)	Authority, Username or User Profile Link
encodingType	string	One (optional)	The encoding type of the message (default is text).
message	string(256)	One (mandatory)	Commit message detailing the scope, motivation, and method of the transaction.
date	ISO 8601 date-time	One (mandatory)	A date-time that specifies the exact moment when the commit was executed.

Commits are related to SensorThings API entities with a one-to-zero-or-one (1:0..1) relationship.

Preliminary Steps

This section contains the preliminary steps to set up the base URL and import necessary libraries.

Replace IST_SOS_ENDPOINT in the following script with your istSOS base URL (http://localhost:8018/istsos4/v1.1 or https://istsos.org/v4/v1.1).

import json
import re
from datetime import datetime, timedelta
import pytz

import requests
from IPython.display import Markdown, display

IST_SOS_ENDPOINT = "http://localhost:8018/istsos4/v1.1"

Login as editor

username = input("Enter your username: ")
password = input("Enter your password: ")

if not username or not password:
    print("Username or password is empty")

else:
    data = {
        "username": username,
        "password": password,
        "grant_type": "password",
    }

    response = requests.post(IST_SOS_ENDPOINT + "/Login", data=data)
    if response.status_code == 200:
        token = response.json()["access_token"]
        print(
            f"Token expires at: { datetime.fromtimestamp(response.json()['expires_in'])}"
        )
        prefix = username + "-"
        print("Your station name will be prefixed with: " + prefix)
    else:
        result = json.dumps(response.json(), indent=2)
        display(Markdown(f"```json\n{result}\n```"))

Create a Thing

body = {
    "name": f"{prefix}Lugano Lakee",
    "description": "The Apline Lake located in Southern Switzerland",
    "properties": {
        "Max depth": "288 m",
    },
}

response = requests.post(
    IST_SOS_ENDPOINT + "/Things",
    data=json.dumps(body),
    headers={
        "Content-type": "application/json",
        "Authorization": f"Bearer {token}",
        "Commit-message": "Create new thing",
    },
)

if response.status_code == 201:
    print(f"Thing created successfully ({response.headers['location']})")
    match = re.search(r"\((\d+)\)", response.headers["location"])
    if match:
        thing_id = int(match.group(1))
    else:
        print("No number found in parentheses.")
else:
    result = json.dumps(response.json(), indent=2)
    display(Markdown(f"```json\n{result}\n```"))

Take a rest and whait e few minutes... ;-)

Update the Thing

body = {
    "properties": {
        "Catchment area": "565.6 km²",
        "Surface Area": "38.7 km²",
        "Avg. Depth": "124 m",
        "Max depth": "288 m",
        "Water Volume": "6.5 km³",
        "Surface elevation": "271 m",
        "Primary inflows": [
            "Vedeggio",
            "Cassarate",
            "Cuccio",
            "Laveggio",
            "Magliasina",
            "Bolletta",
            "Scairolo",
        ],
        "Primary outflows": "Tresa",
    },
}

response = requests.patch(
    f"{IST_SOS_ENDPOINT}/Things({thing_id})",
    data=json.dumps(body),
    headers={
        "Content-type": "application/json",
        "Authorization": f"Bearer {token}",
        "Commit-message": "Corrected properties",
    },
)

if response.status_code == 200:
    print(f"Thing properties updated successfully")
    datetime_before_update = datetime.now(pytz.utc) - timedelta(seconds=1)
    datetime_before_update = datetime_before_update.replace(tzinfo=None)
    datetime_before_update = datetime_before_update.isoformat() + "Z"
else:
    result = json.dumps(response.json(), indent=2)
    display(Markdown(f"```json\n{result}\n```"))

Retrieve the Thing

Retrieve the current state of the Thing

response = requests.get(
    f"{IST_SOS_ENDPOINT}/Things({thing_id})?$expand=Commit",
    headers={
        "Authorization": f"Bearer {token}",
    },
)
result = json.dumps(response.json(), indent=2)
display(Markdown(f"```json\n{result}\n```"))

Retrieve the Thing's state at a specific point in time (before update)

response = requests.get(
    f"{IST_SOS_ENDPOINT}/Things({thing_id})?$expand=Commit&$as_of={datetime_before_update}",
    headers={
        "Authorization": f"Bearer {token}",
    },
)
result = json.dumps(response.json(), indent=2)
display(Markdown(f"```json\n{result}\n```"))

Retrieve the historical states of the Thing

datetime_now = datetime.now(pytz.utc) - timedelta(seconds=1)
datetime_now = datetime_now.replace(tzinfo=None)
datetime_now = datetime_now.isoformat() + "Z"
response = requests.get(
    f"{IST_SOS_ENDPOINT}/Things({thing_id})?$expand=Commit&$from_to={datetime_before_update}/{datetime_now}",
    headers={
        "Authorization": f"Bearer {token}",
    },
)
result = json.dumps(response.json(), indent=2)
display(Markdown(f"```json\n{result}\n```"))