EVOLUTION-MANAGER

Edit File: drv_selafin.html

<html>
<head>
<title>Selafin files</title>
<style type="text/css">

</style>
</head>

<h1>Selafin files</h1>

<p>OGR supports reading from 2D Selafin/Seraphin files. Selafin/Seraphin is the generic output and input format of geographical files in the open-source <a href="http://www.opentelemac.org">Telemac hydraulic model</a>. The file format is suited to the description of numerical attributes for a set of point features at different time steps. Those features usually correspond to the nodes in a finite-element model. The file also holds a connectivity table which describes the elements formed by those nodes and which can also be read by the driver.</p>

<p>The driver supports the use of VSI virtual files as Selafin datasources.</p>

<p>The driver offers full read-write support on Selafin files. However, due to the particular nature of Selafin files where element (polygon) features and node (point) features are closely related, writing on Selafin layers can lead to counter-intuitive results. In a general way, writing on any layer of a Selafin data-source will cause side effects on all the other layers. Also, it is very important <b>not to open the same datasource more than once in update mode</b>. Having two processes write at the same time on a single datasource can lead to irreversible data corruption. The driver issues a warning each time a datasource is opened in update mode.</p>

<h2>Magic bytes</h2>

<p>There is no generic extension to Selafin files. The adequate format is tested by looking at a dozen of magic bytes at the beginning of the file:
<ul>
<li>The first four bytes of the file should contain the values (in hexadecimal): 00 00 00 50. This actually indicates the start of a string of length 80 in the file.</li>
<li>At position 84 in the file, the eight next bytes should read (in hexadecimal): 00 00 00 50 00 00 00 04.</li>
</ul>
</p>

<p>Files which match those two criteria are considered to be Selafin files and the driver will report it has opened them successfully.</p>

<h2><a id="format">Format</a></h2>

<p>Selafin format is designed to hold data structures in a portable and compact way, and to allow efficient random access to the data. To this purpose, Selafin files are binary files with a generic structure.</p>

<h3>Elements</h3>
<p>Selafin files are made of the juxtaposition of elements. Elements have one of the following types:
<ul>
	<li>integer,</li>
	<li>string,</li>
	<li>floating point values,</li>
	<li>arrays of integers,</li>
	<li>arrays of floating point values.</li>
</ul></p>

<table id="tableslots">
	<tr>
		<th>Element</th>
		<th>Internal representation</th>
		<th>Comments</th>
	</tr>
	<tr>
		<td>Integer</td>
		<td> <table class="slots"><tr><td>a</td><td>b</td><td>c</td><td>d</td></tr></table> </td>
		<td>Integers are stored on 4 bytes in big-endian format (most significant byte first). The value of the integer is 2<sup>24</sup>.a+2<sup>16</sup>.b+2<sup>8</sup>.c+d.</td>
	</tr>
	<tr>
		<td>Floating point</td>
		<td> <table class="slots"><tr><td>a</td><td>b</td><td>c</td><td>d</td></tr></table> </td>
		<td>Floating point values are stored on 4 bytes in IEEE 754 format and under big-endian convention (most significant byte first). Endianness is detected at run time only once when the first floating point value is read.</td>
	</tr>
	<tr>
		<td>String</td>
		<td> <table class="slots"><tr><td style="width:80px">Length</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td><td>...</td><td style="width: 80px">Length</td></tr></table> </td>
		<td>Strings are stored in three parts:
			<ul>
				<li>an integer holding the length (in characters) of the string, over 4 bytes;</li>
				<li>the sequence of characters of the string, each character on one byte;</li>
				<li>the same integer with the length of the string repeated</li>
			</ul>
		</td>
	</tr>
	<tr>
		<td>Array of integers</td>
		<td> <table class="slots"><tr><td style="width:80px;">Length</td><td style="width:80px">1</td><td style="width:80px">2</td><td style="width:80px">3</td><td style="width:80px">...</td><td style="width: 80px">Length</td></tr></table> </td>
		<td>Arrays of integers are stored in three parts:
			<ul>
				<li>an integer holding the length (in bytes, thus 4 times the number of elements) of the array, over 4 bytes;</li>
				<li>the sequence of integers in the array, each integer on 4 bytes as described earlier;</li>
				<li>the same integer with the length of the array repeated</li>
			</ul>
		</td>
	</tr>
	<tr>
		<td>Array of floating point values</td>
		<td> <table class="slots"><tr><td style="width:80px;">Length</td><td style="width:80px">1</td><td style="width:80px">2</td><td style="width:80px">3</td><td style="width:80px">...</td><td style="width: 80px">Length</td></tr></table> </td>
		<td>Arrays of floating point values are stored in three parts:
			<ul>
				<li>an integer holding the length (in bytes, thus 4 times the number of elements) of the array, over 4 bytes;</li>
				<li>the sequence of floating point values in the array, each one on 4 bytes as described earlier;</li>
				<li>the same integer with the length of the array repeated</li>
			</ul>
		</td>
	</tr>
</table>

<h3>Full structure</h3>
<p>The header of a Selafin file holds the following elements in that exact order:
<ul>
	<li>a <em>string</em> of 80 characters with the title of the study; the last 8 characters shall be "SERAPHIN" or "SERAFIN" or "SERAFIND";</li>
	<li>an <em>array of integers</em> of exactly 2 elements, the first one being the number of variables (attributes) <i>nVar</i>, and the second is ignored;</li>
	<li><em><i>nVar</i> strings</em> with the names of the variables, each one with length 32;</li>
	<li>an <em>array of integers</em> of exactly 10 elements:
		<ul>
			<li>the third element is the x-coordinate of the origin of the model;</li>
			<li>the fourth element is the y-coordinate of the origin of the model;</li>
			<li>the tenth element <i>isDate</i> indicates if the date of the model has to be read (see later);</li>
                        <li>in addition, the second element being unused by hydraulic software at the moment, it is used by the driver to store the spatial reference system of the datasource, in the form of a single integer with the EPSG number of the projection;</li>
		</ul></li>
	<li>if <i>isDate</i>=1, an <em>array of integers</em> of exactly 6 elements, with the starting date of the model (year, month, day, hour, minute, second);</li>
	<li>an <em>array of integers</em> of exactly 4 elements:
		<ul>
			<li>the first element is the number of elements <i>nElements</i>,
			<li>the second element is the number of points <i>nPoints</i>,
			<li>the third element is the number of points per element<i>nPointsPerElement</i>,
			<li>the fourth element must be 1;</li>
		</ul></li>
	<li>an <em>array of integers</em> of exactly <i>nElements*nPointsPerElement</i> elements, with each successive set of <i>nPointsPerElement</i> being the list of the number of the points  (number starting with 1) constituting an element;</li>
	<li>an <em>array of integers</em> of exactly <i>nPoints</i> elements ignored by the driver (the elements shall be 0 for inner points and another value for the border points where a limit condition is applied);</li>
	<li>an <em>array of floating point values</em> of exactly <i>nPoints</i> elements with the x-coordinates of the points;</li>
	<li>an <em>array of floating point values</em> of exactly <i>nPoints</i> elements with the y-coordinates of the points;</li>
</ul></p>

<p>The rest of the file actually holds the data for each successive time step. A time step contains the following elements:
<ul>
	<li>a <em>array of floating point values</em> of exactly 1 element, being the date of the time step relative to the starting date of the simulation (usually in seconds);</li>
	<li><em><i>nVar</i> array of floating point values</em>, each with exactly <i>nPoints</i> elements, with the values of each attribute for each point at the current time step.</li>
</ul></p>

<h2>Mapping between file and layers</h2>

<h3>Layers in a Selafin datasource</h3>
<p>The Selafin driver accepts only Selafin files as data sources.</p>

<p>Each Selafin file can hold one or several time steps. All the time steps are read by the driver and two layers are generated for each time step:
<ul>
	<li>one layer with the nodes (points) and their attributes: its name is the base name of the data source, followed by "_p" (for Points);</li>
	<li>one layer with the elements (polygons) and their attributes calculated as the averages of the values of the attributes of their vertices: its name is the base name of the data source, followed by "_e" (for Elements).</li>
</ul></p>

<p>Finally, either the number of the time step, or the calculated date of the time step (based on the starting date and the number of seconds elapsed), is added to the name. A data source in a file called Results may therefore be read as several layers:
<ul>
	<li><tt>Results_p2014_05_01_20_00_00</tt>, meaning that the layers holds the attributes for the nodes and that the results hold for the time step at 8:00 PM, on May 1st, 2014;</li>
	<li><tt>Results_e2014_05_01_20_00_00</tt>, meaning that the layers holds the attributes for the elements and that the results hold for the time step at 8:00 PM, on May 1st, 2014;</li>
	<li><tt>Results_p2014_05_01_20_15_00</tt>, meaning that the layers holds the attributes for the elements and that the results hold for the time step at 8:15 PM, on May 1st, 2014;</li>
	<li>...</li>
</ul>
</p>

<h3>Constraints on layers</h3>
<p>Because of the <a href="#format">format of the Selafin file</a>, the layers in a single Selafin datasource are not independent from each other. Changing one layer will most certainly have side effects on all other layers. The driver updates all the layers to match the constraints:
<ul>
	<li>All the point layers have the same number of features. When a feature is added or removed in one layer, it is also added or removed in all other layers.</li>
	<li>Features in different point layers share the same geometry. When the position of one point is changed, it is also changed in all other layers.</li>
	<li>All the element layers have the same number of features. When a feature is added or removed in one layer, it is also added or removed in all other layers.</li>
	<li>All the polygons in element layers have the same number of vertices. The number of vertices is fixed when the first feature is added to an element layer, and can not be changed afterwards without recreating the datasource from scratch.</li>
	<li>Features in different element layers share the same geometry. When an element is added or removed in one layer, it is also added or removed in all other layers.</li>
	<li>Every vertex of every feature in an element layer has a corresponding point feature in the point layers. When an element feature is added, if its vertices do not exist yet, they are created in the point layers.</li>
	<li>Points and elements layers only support attributes of type "REAL". The format of real numbers (width and precision) can not be changed.</li>
</ul></p>

<h2>Layer filtering specification</h2>
<p>As a single Selafin files may hold millions of layers, and the user is generally interested in only a few of them, the driver supports syntactic sugar to filter the layers before they are read.</p>

<p>When the datasource is specified, it may be followed immediately by a <em>layer filtering specification.</em>, as in <tt>Results[0:10]</tt>. The effects of the layer filtering specification is to indicate which time steps shall be loaded from all Selafin files.</p>

<p>The layer filtering specification is a comma-separated sequence of range specifications, delimited by square brackets and maybe preceded by the character 'e' or 'p'. The effect of characters 'e' and 'p' is to select respectively either only elements or only nodes. If no character is added, both nodes and elements are selected.
Each range specification is:
<ul>
	<li>either a single number, representing one single time step (whose numbers start with 0),</li>
	<li>or a set of two numbers separated by a colon: in that case, all the time steps between and including those two numbers are selected; if the first number is missing, the range starts from the beginning of the file (first time step); if the second number is missing, the range goes to the end of the file (last time step);</li>
</ul>
Numbers can also be negative. In this case, they are counted from the end of the file, -1 being the last time step.</p>

<p>Some examples of layer filtering specifications:
<table id="tablesexamples">
<tr><td style="font-family:courier">[0]</td><td>First time step only, but return both points and elements</td></tr>
<tr><td style="font-family:courier">[e:9]</td><td>First 10 time steps only, and only layers with elements</td></tr>
<tr><td style="font-family:courier">[p-4:]</td><td>Last 4 time steps only, and only layers with nodes</td></tr>
<tr><td style="font-family:courier">[3,10,-2:-1]</td><td>4<sup>th</sup>, 11<sup>th</sup>, and last two time steps, for both nodes and elements</td></tr>
</table></p>

<h2><a id="DCO">Datasource creation options</a></h2>
<p>Datasource creation options can be specified with the "<tt>-dsco</tt>" flag in ogr2ogr.</p>
<dl>
	<dt>TITLE</dt>
	<dd>Title of the datasource, stored in the Selafin file. The title must not hold more than 72 characters. If it is longer, it will be truncated to fit in the file.</dd>
	<dt>DATE</dt>
	<dd>Starting date of the simulation. Each layer in a Selafin file is characterized by a date, counted in seconds since a reference date. This option allows providing the reference date. The format of this field must be YYYY-MM-DD_hh:mm:ss. The format does not mention the time zone.</dd>
</dl>

<p>An example of datasource creation option is: <tt>-dsco TITLE="My simulation" -dsco DATE=2014-05-01_10:00:00</tt>.

<h2>Layer creation options</h2>
<p>Layer creation options can be specified with the "<tt>-lco</tt>" flag in ogr2ogr.</p>
<dl>
	<dt>DATE</dt>
	<dd>Date of the time step relative to the starting date of the simulation (see <a href="#DCO">Datasource creation options</a>). This is a single floating-point value giving the number of seconds since the starting date.</dd>
</dl>

<p>An example of datasource creation option is: <tt>-lco DATE=24000</tt>.

<h2>Notes about the creation and the update of a Selafin datasource</h2>
<p>The driver supports creating and writing to Selafin datasources, but there are some caveats when doing so.</p>

<p>When a new datasource is created, it does not contain any layer, feature or attribute.</p>

<p>When a new layer is created, it automatically inherits the same number of features and attributes as the other layers of the same type (points or elements) already in the datasource. The features inherit the same geometry as their corresponding ones in other layers. The attributes are set to 0. If there was no layer in the datasource yet, the new layer is created with no feature and attribute.In any case, when a new layer is created, two layers are actually added: one for points and one for elements.</p>

<p>New features and attributes can be added to the layers or removed. The behaviour depends on the type of layer (points or elements). The following table explains the behaviour of the driver in the different cases.</p>

<table id="tableoperations">
	<thead><tr><th>Operation</th><th>Points layers</th><th>Element layers</th></tr></thead>
	<tbody>
		<tr>
			<td>Change the geometry of a feature</td>
			<td>The coordinates of the point are changed in the current layer and all other layers in the datasource.</td>
			<td>The coordinates of all the vertices of the element are changed in the current layer and all other layers in the datasource. It is not possible to change the number of vertices. The order of the vertices matters.</td>
		</tr>
		<tr>
			<td>Change the attributes of a feature</td>
			<td>The attributes of the point are changed in the current layer only.</td>
			<td>No effect.</td>
		</tr>
		<tr>
			<td>Add a new feature</td>
			<td>A new point is added at the end of the list of features, for this layer and all other layers. Its attributes are set to the values of the new feature.</td>
			<td>The operation is only allowed if the new feature has the same number of vertices as the other features in the layer. The vertices are checked to see if they currently exist in the set of points. A vertex is considered equal to a point if its distance is less than some maximum distance, approximately equal to 1/1000<sup>th</sup> of the average distance between two points in the points layers. When a corresponding point is found, it is used as a vertex for the element. If no point is found, a new is created in all associated layers.</td>
		</tr>
		<tr>
			<td>Delete a feature</td>
			<td>The point is removed from the current layer and all point layers in the datasource. All elements using this point as a vertex are also removed from all element layers in the datasource.</td>
			<td>The element is removed from the current layer and all element layers in the datasource.</td>
		</tr>
	</tbody>
</table>

<p>Typically, this implementation of operations is exactly what you'll expect. For example, ogr2ogr can be used to reproject the file without changing the inner link between points and elements.</p>

<p>It should be noted that update operations on Selafin datasources are very slow. This is because the format does no allow for quick insertions or deletion of features and the file must be recreated for each operation.</p>

<h2>VSI Virtual File System API support</h2>

<p>The driver supports reading and writing to files managed by VSI Virtual File System API, which include "regular" files, as well as files in the /vsizip/ (read-write) , /vsigzip/ (read-only) , /vsicurl/ (read-only) domains.</p>

<h2>Other notes</h2>
<p>There is no SRS specification in the Selafin standard. The implementation of SRS is an addition of the driver and stores the SRS in an unused data field in the file. Future software using the Selafin standard may use this field and break the SRS specification. In this case, Selafin files will still be readable by the driver, but their writing will overwrite a value which may have another purpose.</p>

</body>
</html>