EVOLUTION-MANAGER

Edit File: drv_pg_advanced.html

<html>
<head>
<title>PostgreSQL / PostGIS - Advanced Driver Information</title>
</head>

<h1>PostgreSQL / PostGIS - Advanced Driver Information</h1>

The information collected in that page deal with advanced topics, not found in the
<a href="drv_pg.html">OGR PostgreSQL driver Information</a> page.

<h2>Connection options related to schemas and tables</h2>

Starting with GDAL 1.8.0, the database opening should be significantly faster than in previous versions, so using tables= or
schemas= options will not bring further noticeable speed-ups.<p>

Starting with GDAL 1.6.0, the set of tables to be scanned can be overridden by specifying
<i>tables=[schema.]table[(geom_column_name)][,[schema2.]table2[(geom_column_name2)],...]</i> within the connection string.
If the parameter is found, the driver skips enumeration of the tables as
described in the next paragraph.<p>

Starting with GDAL 1.7.0, it is possible to restrict the schemas that will be scanned while establishing
the list of tables. This can be done by specifying <i>schemas=schema_name[,schema_name2]</i> within the connection string.
This can also be a way of speeding up the connection to a PostgreSQL database if there are a lot of schemas.
Note that if only one schema is listed, it will also be made automatically the active schema
(and the schema name will not prefix the layer name). Otherwise, the active schema is still 'public',
unless otherwise specified by the <i>active_schema=</i> option.<p>

Starting with GDAL 1.7.0, the active schema ('public' being the default) can be overridden by specifying
<i>active_schema=schema_name</i> within the connection string. The active schema is the schema where tables
are created or looked for when their name is not explicitly prefixed by a schema name.
Note that this does not restrict the tables that will be listed (see <i>schemas=</i> option above).
When getting the list of tables, the name of the tables within that active schema will
not be prefixed by the schema name. For example, if you have a table 'foo' within the public
schema, and a table 'foo' within the 'bar_schema' schema, and that you specify
active_schema=bar_schema, 2 layers will be listed : 'foo' (implicitly within 'bar_schema') and 'public.foo'.<p>

<h2>Multiple geometry columns</h2>

Starting with GDAL 1.6.0, the PostgreSQL driver supports accessing tables with multiple PostGIS geometry columns.

OGR 1.11 supports reading, updating, creating tables with multiple PostGIS
geometry columns (following
<a href="http://trac.osgeo.org/gdal/wiki/rfc41_multiple_geometry_fields">RFC 41</a>)

For such a table, a single OGR layer will be reported with as many geometry
fields as there are geometry columns in the table.<p>

For backward compatibility, it is also possible to query a layer with GetLayerByName()
with a name formatted like 'foo(bar)' where 'foo' is a table and 'bar' a geometry column.<p>

For such a table, there will be as many layers reported as the number of geometry columns listed for that
table in the <i>geometry_columns</i> table.
For example, if a table 'foo' has 2 geometry columns 'bar' and 'baz', 2 layers will be reported :
'foo(bar)' and 'foo(baz)'. For backward compatibility, if a table has only one geometry column,
the layer name is the table name. Also if a table 'foo' has several geometry columns,
with one being called 'wkb_geometry', the layer corresponding to this geometry column will
be simply reported as 'foo'. Be careful - the behaviour in creation, update or deletion of layers
that are based on tables with multiple PostGIS geometry column is known to have (not well-defined)
side-effects on the other layers as they are closely tied. Thus, that capability should currently
be thought as mostly read-only.<p>

<h2>Layers</h2>
Starting with GDAL 1.6.0, even when PostGIS is enabled, if the user defines the environment variable
<pre>PG_LIST_ALL_TABLES=YES</pre> (and does not specify tables=), all regular user tables and named views
will be treated as layers. However, tables with multiple geometry column will only be reported
once in that mode. So this variable is mainly useful when PostGIS is enabled to find out tables
with no spatial data, or views without an entry in <i>geometry_columns</i> table.<p>

In any case, all user tables can be queried explicitly with GetLayerByName()<p>

Regular (non-spatial) tables can be accessed, and will return features with
attributes, but not geometry.  If the table has a "wkb_geometry" field, it will
be treated as a spatial table.  The type of the field is inspected to
determine how to read it.  It can be a PostGIS <b>geometry</b> field, which
is assumed to come back in OGC WKT, or type BYTEA or OID in which case it
is used as a source of OGC WKB geometry.<p>

Starting with GDAL 1.6.0, tables inherited from spatial tables are supported.<p>

If there is an "ogc_fid" field, it will be used to set the feature id of
the features, and not treated as a regular field. <p>

The layer name may be of the form "schema.table". The schema must exist, and the
user needs to have write permissions for the target and the public schema.<p>

<p>Starting with GDAL 1.7.0, if the user defines the environment variable
<pre>PG_SKIP_VIEWS=YES</pre> (and does not specify tables=), only the regular
user tables will be treated as layers. The default action is to include the
views. This variable is particularly useful when you want to copy the data into
another format while avoiding the redundant data from the views.

<h2>Named views</h2>

When PostGIS is enabled for the accessed database, named views are supported, provided that
there is an entry in the <i>geometry_columns</i> tables. But, note that the AddGeometryColumn() SQL
function doesn't accept adding an entry for a view (only for regular tables). So, that must usually
be done by hand with a SQL statement like :
<pre>"INSERT INTO geometry_columns VALUES ( '', 'public', 'name_of_my_view', 'name_of_geometry_column', 2, 4326, 'POINT');"</pre>

Starting with GDAL 1.6.0, it is also possible to use named views without inserting a row in the geometry_columns
table. For that, you need to explicitly specify the name of the view in the "tables=" option of the
connection string. See above. The drawback is that OGR will not be able to report a valid SRS and figure out
the right geometry type.

<h2>Retrieving FID of newly inserted feature</h2>

Starting with OGR 1.8.0, and for PostgreSQL >= 8.2 databases, the FID of a feature (i.e. usually the
value of the OGC_FID column for the feature) inserted into a table with CreateFeature(), in non-copy
mode, will be retrieved from the database and can be obtained with GetFID(). One side-effect of this
new behaviour is that you must be careful if you re-use the same feature object in a loop that makes
insertions. After the first iteration, the FID will be set to a non-null value, so at the second iteration,
CreateFeature() will try to insert the new feature with the FID of the previous feature, which will fail as
you cannot insert 2 features with same FID. So in that case you must explicitly reset the FID before calling CreateFeature(),
or use a fresh feature object.<p>

Snippet example in Python :
<pre>
    feat = ogr.Feature(lyr.GetLayerDefn())
    for i in range(100):
        feat.SetFID(-1)  # Reset FID to null value
        lyr.CreateFeature(feat)
        print('The feature has been assigned FID %d' % feat.GetFID())
</pre>

or :
<pre>
    for i in range(100):
        feat = ogr.Feature(lyr.GetLayerDefn())
        lyr.CreateFeature(feat)
        print('The feature has been assigned FID %d' % feat.GetFID())
</pre>

OGR &lt; 1.8.0 behaviour can be obtained by setting the configuration option OGR_PG_RETRIEVE_FID to FALSE.<p>

<h2>Issues with transactions</h2>

<p>Note: this section mostly applies to GDAL 2.0, that implements
<a href="http://trac.osgeo.org/gdal/wiki/rfc54_dataset_transactions">RFC 54 - Dataset transactions</a>
Previous versions had different behaviour which made it impractical to handle
both reading and writing with the same OGR datasource. Reading several layers
in a interleaved way was also not working properly. The new below behaviour should
enable more powerful uses, but might cause subtle problems for existing code
that relied on implicit transactions being regularly flushed by the PG driver in GDAL 1.X</p>

<p>Efficient sequential reading in PostgreSQL requires to be done within a transaction
(technically this is a CURSOR WITHOUT HOLD).
So the PG driver will implicitly open such a transaction if none is currently
opened as soon as a feature is retrieved. This transaction will be released if
ResetReading() is called (provided that no other layer is still being read).</p>

<p>If within such an implicit transaction, an explicit dataset level StartTransaction()
is issued, the PG driver will use a SAVEPOINT to emulate properly the transaction
behaviour while making the active cursor on the read layer still opened.</p>

<p>If an explicit transaction is opened with dataset level StartTransaction()
before reading a layer, this transaction will be used for the cursor that iterates
over the layer. When explicitly committing or rolling back the transaction, the
cursor will become invalid, and ResetReading() should be issued again to restart
reading from the beginning.</p>

<p>As calling SetAttributeFilter() or SetSpatialFilter() implies an implicit
ResetReading(), they have the same effect as ResetReading(). That is to say,
while an implicit transaction is in progress, the transaction will be committed
(if no other layer is being read), and a new one will be started again at the next
GetNextFeature() call. On the contrary, if they are called within an explicit
transaction, the transaction is maintained.</p>

<p>With the above rules, the below examples show the SQL instructions that are
run when using the OGR API in different scenarios.</p>
<pre>

lyr1->GetNextFeature()             BEGIN (implicit)
                                   DECLARE cur1 CURSOR FOR SELECT * FROM lyr1
                                   FETCH 1 IN cur1

lyr1->SetAttributeFilter('xxx')
     --> lyr1->ResetReading()      CLOSE cur1
                                   COMMIT (implicit)

lyr1->GetNextFeature()             BEGIN (implicit)
                                   DECLARE cur1 CURSOR  FOR SELECT * FROM lyr1 WHERE xxx
                                   FETCH 1 IN cur1

lyr2->GetNextFeature()             DECLARE cur2 CURSOR  FOR SELECT * FROM lyr2
                                   FETCH 1 IN cur2

lyr1->GetNextFeature()             FETCH 1 IN cur1

lyr2->GetNextFeature()             FETCH 1 IN cur2

lyr1->CreateFeature(f)             INSERT INTO cur1 ...

lyr1->SetAttributeFilter('xxx')
     --> lyr1->ResetReading()      CLOSE cur1
                                   COMMIT (implicit)

lyr1->GetNextFeature()             DECLARE cur1 CURSOR  FOR SELECT * FROM lyr1 WHERE xxx
                                   FETCH 1 IN cur1

lyr1->ResetReading()               CLOSE cur1

lyr2->ResetReading()               CLOSE cur2
                                   COMMIT (implicit)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ds->StartTransaction()             BEGIN

lyr1->GetNextFeature()             DECLARE cur1 CURSOR FOR SELECT * FROM lyr1
                                   FETCH 1 IN cur1

lyr2->GetNextFeature()             DECLARE cur2 CURSOR FOR SELECT * FROM lyr2
                                   FETCH 1 IN cur2

lyr1->CreateFeature(f)             INSERT INTO cur1 ...

lyr1->SetAttributeFilter('xxx')
     --> lyr1->ResetReading()      CLOSE cur1
                                   COMMIT (implicit)

lyr1->GetNextFeature()             DECLARE cur1 CURSOR  FOR SELECT * FROM lyr1 WHERE xxx
                                   FETCH 1 IN cur1

lyr1->ResetReading()               CLOSE cur1

lyr2->ResetReading()               CLOSE cur2

ds->CommitTransaction()            COMMIT

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ds->StartTransaction()             BEGIN

lyr1->GetNextFeature()             DECLARE cur1 CURSOR FOR SELECT * FROM lyr1
                                   FETCH 1 IN cur1

lyr1->CreateFeature(f)             INSERT INTO cur1 ...

ds->CommitTransaction()            CLOSE cur1 (implicit)
                                   COMMIT

lyr1->GetNextFeature()             FETCH 1 IN cur1      ==> Error since the cursor was closed with the commit. Explicit ResetReading() required before

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

lyr1->GetNextFeature()             BEGIN (implicit)
                                   DECLARE cur1 CURSOR FOR SELECT * FROM lyr1
                                   FETCH 1 IN cur1

ds->StartTransaction()             SAVEPOINT savepoint

lyr1->CreateFeature(f)             INSERT INTO cur1 ...

ds->CommitTransaction()            RELEASE SAVEPOINT savepoint

lyr1->ResetReading()               CLOSE cur1
                                   COMMIT (implicit)
</pre>

<p>Note: in reality, the PG drivers fetches 500 features at once. The FETCH 1
is for clarity of the explanation.</p>

<h3>Advanced Examples</h3>

<ul>
<li>
<p>
This example shows using ogrinfo to list only the layers specified by the <i>tables=</i> options. (Starting with GDAL 1.6.0)<p>

<pre>
ogrinfo -ro PG:'dbname=warmerda tables=table1,table2'
</pre>
</li>

<li>
<p>
This example shows using ogrinfo to query a table 'foo' with multiple geometry columns ('geom1' and 'geom2'). (Starting with GDAL 1.6.0) <p>

<pre>
ogrinfo -ro -al PG:dbname=warmerda 'foo(geom2)'
</pre>
</li>

<li>
<p>
This example show how to list only the layers inside the schema apt200810 and apt200812.
The layer names will be prefixed by the name of the schema they belong to. (Starting with GDAL 1.7.0)<p>

<pre>
ogrinfo -ro PG:'dbname=warmerda schemas=apt200810,apt200812'
</pre>
</li>

<li>
<p>
This example shows using ogrinfo to list only the layers inside the schema named apt200810.
Note that the layer names will not be prefixed by apt200810 as only one schema is listed. (Starting with GDAL 1.7.0)<p>

<pre>
ogrinfo -ro PG:'dbname=warmerda schemas=apt200810'
</pre>
</li>

<li>
<p>
This example shows how to convert a set of shapefiles inside the apt200810 directory into an
existing Postgres schema apt200810. In that example, we could have use the schemas= option instead. (Starting with GDAL 1.7.0)<p>

<pre>
ogr2ogr -f PostgreSQL "PG:dbname=warmerda active_schema=apt200810" apt200810
</pre>
</li>

<li>
<p>
This example shows how to convert all the tables inside the schema apt200810 as a set of shapefiles inside the apt200810 directory.
Note that the layer names will not be prefixed by apt200810 as only one schema is listed (Starting with GDAL 1.7.0)<p>

<pre>
ogr2ogr apt200810 PG:'dbname=warmerda schemas=apt200810'
</pre>
</li>

<li>
<p>
This example shows how to overwrite an existing table in an existing schema. Note the use of -nln to specify the qualified layer name.<p>

<pre>
ogr2ogr -overwrite -f PostgreSQL "PG:dbname=warmerda" mytable.shp mytable -nln myschema.mytable
</pre>

Note that using -lco SCHEMA=mytable instead of -nln would not have worked in that case
(see <a href="http://trac.osgeo.org/gdal/ticket/2821">#2821</a> for more details).<p>

If you need to overwrite many tables located in a schema at once, the -nln option is not
the more appropriate, so it might be more convenient to use the active_schema connection
string (Starting with GDAL 1.7.0). The following example will overwrite, if necessary,
all the PostgreSQL tables corresponding to a set of shapefiles inside the apt200810 directory :

<pre>
ogr2ogr -overwrite -f PostgreSQL "PG:dbname=warmerda active_schema=apt200810" apt200810
</pre>

</li>

</ul>

<ul>
<li> <a href="drv_pg.html">OGR PostgreSQL driver Information</a><p>
</ul>

</html>