Accessing data¶
Accessing the data is possible using the SOS Standard requests, WA REST requests or directly using the Data Viewer or Data Editor.
Let’s try to request data using the different interfaces…
Getting observations in the SOS 1.0.0 way¶
Quick view on the GetObservation request using the GET method:
| Parameter | Description | Definition | Multiplicity and Use |
| request | getObservation is designed to request detailed sensor metadata | getObservation | One (mandatory) |
| service | Service type identifier | SOS | One (mandatory) |
| version | Specification version for operation | 1.0.0 | One (mandatory) |
| srsName | Defines the spatial reference system that should be used for any geometries that are returned in the response. | This must be one of the advertised values in the offering specified in gml:srsName elements. | One (Optional) |
| offering | Specifies the offering URI advertised in the GetCapabilities document. All of the following parameters are dependent on the selected offering. | This must match the gml:name of the offering or be constructed as a URL with a fragment identifier resolving to the offering gml:id. | One (mandatory) |
| eventTime | Specifies the time period(s) for which observations are requested. This allows a client to request observations from a specific instant, multiple instances or periods of time in the past, present and future. The supported range is listed in the selected offering capabilities | ISO 8601 Examples: Time instant: 2015-06-11T17:30:00+0200 Time period 2015-06-11T14:30:00+0200/ 2015-06-11T17:30:00+0200 | Zero or many (Optional) |
| procedure | The procedure parameter specifies the sensor system(s) for which observations are requested. It defines a filter for the procedure property of the observations. | comma separated valid sensors from the GetCapabilities | Zero or many (Optional |
| observedProperty | One or many (mandatory) | ||
| featureOfInterest | Specifies the feature for which observations are requested. This can either be represented by a reference to a feature ID advertised in the capabilities document or can be a spatial constraint. | Zero or many (Optional) | |
| responseFormat | Specifies the desired resultFormat MIME content type for transport of the results (e.g. TML, O&M native format, or MPEG stream out-of-band). The supported output formats are listed in the selected offering capabilities. Desired output format of the getObservation operation. This can be a MimeType or QName for example. | text/xml;subtype=”sensor ML/1.0.0” | One (mandatory) |
Procedure-time-property filters¶
Requesting rainfall observations from sensor P_LUGANO between 2015-01-01T00:00:00+01 and 2015-02-4T17:00:00+01:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&procedure=urn:ogc:def:procedure:x-istsos:1.0:P_LUGANO&eventTime=2015-06-01T00:00:00+0200/2015-06-03T00:00:00+0200&observedProperty=urn:ogc:def:parameter:x-istsos:1.0:meteo:air:rainfall&responseFormat=text/xml;subtype="sensorML/1.0.1"&service=SOS&version=1.0.0Time-property filters¶
Requesting rainfall observations from all the stations between 2014-06-01T00:00:00+02 and 2014-06-03T00:00:00+02:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&eventTime=2015-06-01T00:00:00+0200/2015-06-03T00:00:00+0200&observedProperty=urn:ogc:def:parameter:x-istsos:1.0:meteo:air:rainfall&responseFormat=text/xml;subtype="sensorML/1.0.0"&service=SOS&version=1.0.0Time-property-area filters¶
Requesting the last meteo observations applying a spatial intersection with a BBOX:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&observedProperty=meteo&responseFormat=text/xml;subtype="sensorML/1.0.1"&service=SOS&version=1.0.0&featureOfInterest=<ogc:BBOX><ogc:PropertyName>the_geom</ogc:PropertyName><gml:Box srsName='EPSG:4326'><gml:coordinates>18,45 30,55</gml:coordinates></gml:Box></ogc:BBOX>Time-property-distance filters¶
Requesting the last meteo observations applying a distance filter:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&observedProperty=temperature&responseFormat=text/xml;subtype="sensorML/1.0.1"&service=SOS&version=1.0.0&featureOfInterest=<ogc:DWithin><ogc:PropertyName>SHAPE</ogc:PropertyName><gml:Point srsName="EPSG:4326"><gml:coordinates decimal="." cs="," ts=" ">8.961,46.027</gml:coordinates></gml:Point><ogc:Distance>10</ogc:Distance></ogc:DWithin>Now try to change the distance from 10 to 100.
Time-property filters and result values¶
Requesting rainfall observations from all the stations between 2014-06-01T00:00:00+02 and 2014-06-03T00:00:00+02 and where the rain value is greater than 0:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&eventTime=2015-06-01T00:00:00+0200/2015-06-03T00:00:00+0200&observedProperty=urn:ogc:def:parameter:x-istsos:1.0:meteo:air:rainfall&responseFormat=text/plain&result="<ogc:PropertyIsGreaterThan><ogc:PropertyName>rainfall</ogc:PropertyName><ogc:Literal>0</ogc:Literal></ogc:PropertyIsGreaterThan>"Getting observations using the istSOS extending features¶
In this part of the tutorial we will explore the extending features of istSOS, developed for making user life easier and to fulfill data experts requirements.
GetObservation with simple names¶
According to the standard observedProperties and procedures are accessible using a Unique Resource Identifier (URI). This is those used when the observed property and the procedure was created.
In this tutorial we used for example:
istSOS is not strict and allows to specify in a GetObservation request just the or procedure name and/or observed property in the request: as a result for the desired procedures all the observed properties with that words will be selected (LIKE ‘%XXX%’ SQL query).
In a suggested hierarchical usage of URIs, this allows to quickly access to
all the subdomain properties, so for example using
observedProperty=urn:ogc:def:parameter:x-istsos:1.0:meteo in a getObservation
request in istSOS will return all the available observations which measure
meteo parameters (rainfall, windspeed, humidity, etc..).
Let’s try:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&procedure=P_LUGANO&eventTime=2015-06-01T00:00:00+0200/2015-06-03T00:00:00+0200&observedProperty=rainfall&responseFormat=text/xml;subtype="sensorML/1.0.1"&service=SOS&version=1.0.0Requesting all the rainfall observations between 2015-06-01T00:00:00+02 and 2015-06-03T00:00:00+02:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&eventTime=2015-06-01T00:00:00+0200/2015-06-03T00:00:00+0200&observedProperty=rainfall&responseFormat=text/xml;subtype="sensorML/1.0.1"&service=SOS&version=1.0.0GetObservation with specific time zone¶
istSOS support time zones. Whenever in getObservation request the eventTime is specified with a time zone (e.g.: +0700) the response will be returned with the same time zone.
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&procedure=P_LUGANO&eventTime=2015-06-01T00:00:00+0500/2015-06-03T00:00:00+0500&observedProperty=rainfall&responseFormat=text/xml;subtype="sensorML/1.0.1"&service=SOS&version=1.0.0GetObservation in CSV or JSON¶
In addition to the mandatory text/xml;subtype=”sensorML/1.0.0” istSOS support also application/json and text/csv (for simplification also text or json )
Data in CSV:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&eventTime=2015-06-01T00:00:00+0200/2015-06-03T00:00:00+0200&observedProperty=rainfall&responseFormat=text/plain&service=SOS&version=1.0.0Data in JSON:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&eventTime=2015-06-01T00:00:00+0200/2015-06-03T00:00:00+0200&observedProperty=rainfall&responseFormat=application/json&service=SOS&version=1.0.0GetObservation with data aggregation on the fly¶
When executing a getObservation request istSOS offer an extra feature. Adding vendor specific parameters aggregateInterval, aggregateFunction, aggregatenodata and aggregatenodataqi you can request data already aggregated by istSOS.
- aggregateInterval: ISO 8601 Durations (P1DT = 1 Day, PT12H = 12 hours)
- aggregateFunction: AVG, SUM, MAX, MIN
For example we can Request maximal daily temperature observation:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&procedure=T_LUGANO&eventTime=2015-05-04T00:00:00+01/2015-05-14T00:00:00+01&observedProperty=temperature&aggregateInterval=PT24H&aggregateFunction=MAX&responseFormat=text/plain&service=SOS&version=1.0.0GetObservation in CSV with qualityIndex¶
in addition to the mandatory text/xml;subtype=”sensorML/1.0.0” istSOS support also application/json and text/csv (for simplification also text or json)
Data in CSV with qualityIndex:
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&eventTime=2015-06-01T00:00:00+0200/2015-06-03T00:00:00+0200&observedProperty=rainfall&responseFormat=text/plain&service=SOS&version=1.0.0&qualityIndex=TrueData in CSV with qualityFilter:
Note that when applying a qualityFilter istSOS returns only the observations that has all the observedProperties values with associated qualityIndex satisfying the criteria.
e.g.: suppose you have a procedure observing rainfall, temperature and humidity. Your single observation will have the triplet of values and quality index:
time1, rain value, rain qi, temperature val, emperature qi, humidity val, humidity qi
time2, rain value, rain qi, temperature val, emperature qi, humidity val, humidity qi
time3, rain value, rain qi, temperature val, emperature qi, humidity val, humidity qi
time4, rain value, rain qi, temperature val, emperature qi, humidity val, humidity qi
| Operator | istSOS behaviour |
| > | only records where all qi are > of the filter value are returned |
| >= | only records where all qi are >= of the filter value are returned |
| < | only records where all qi are < of the filter value are returned |
| <= | only records where all qi are <= of the filter value are returned |
| = | all the records that has at least one qi equal to the filter value |
http://localhost/istsos/demo?service=SOS&request=GetObservation&offering=temporary&eventTime=2015-06-03T15:00:00+01:00/2015-06-03T16:00:00+01:00&observedProperty=temperature&responseFormat=text/plain&service=SOS&version=1.0.0&procedure=T_LUGANO&qualityIndex=True&qualityfilter=>110Getting observations using the WA REST¶
Composing a WA REST request is all about building the correct path url.
Note
Executing a service request, you will receive a list of istSOS service instances: http://localhost/istsos/wa/istsos/services
Executing a procedures get list operation, you will receive a list of procedures belonging to a specific service:
http://localhost/istsos/wa/istsos/services/demo/procedures/operations/getlist
Getting observations with the Data Viewer¶
The istSOS web administration pages interact with the service making use of WA REST. The Data Viewer panel is implemented as an example of data visualization.
View observation
To open your Web Viewer follow this link: http://localhost/istsos/admin
And then in the “Data Management” tab press the “Data Viewer” button.
Note
Up to now the viewer permit to display data of a single observationProperties only, you can select and display multiple procedures but with the same observed property.
Go ahead and take some confidence with the Data Viewer.
Editing observations with the Data Editor¶
Hey! The procedure T_LUGANO has some problems.. let’s fix it!
From 2015-06-02T02:40:00 to 2015-06-02T07:20:00 there are no data values:
urn:ogc:def:parameter:x-istsos:1.0:time:iso8601,urn:ogc:def:parameter:x-istsos:1.0:meteo:air:temperature
2015-06-02T02:30:00.000000+0200,12.150000
2015-06-02T02:40:00.000000+0200,-999.9
2015-06-02T02:50:00.000000+0200,-999.9
2015-06-02T03:00:00.000000+0200,-999.9
We can correct them using the Data Editor!
Load the data
From the Web Admin:
- Go to Data Management
- Press the Data Editor button
- Like in the Data Viewer sequentially choose
- the service demo,
- the offering temporary
- and then “Add” BELLINZONA, LOCARNO and T_LUGANO
- On the right panel choose the Property: air-temperature
- Press “Plot”, the last week of measurements is loaded and displayed
Editing with the “Calculator”
On the left panel there is the “Editor” tab:
- Select T_LUGANO from the combo list
- The press “Start editing”, the grid is now displayed
- At the bottom-right corner of the chart there are 3 buttons “Day”, “Week” and “All”
- Click on “Day”, the chart is zoomed to contain only one day of data
- Drag the timeline bar on the right where you will see that T_LUGANO has no data
- Click on the chart to select the last observation before the “nodata” hole, a green line is displayed and in the Editing Grid the corresponding row is selected.
- Now go to the Editing Grid panel
- Click the first row where data are NaN,
- Scroll to the last NaN record and holding the SHIFT Key click on it
- The press the “Calculator” button
With the Calculator we are able to correct an interval of data in a single action. It is possible to set a numeric value or also use a function using data from the other loaded procedures.
Let’s build a function that make the average of the data from BELLINZONA and LOCARNO and then removes to units:
((BELLINZONA+LOCARNO)/2)-2
Select the quality index. In this case we can choose a QI 500 (manually adjusted).
