ogr-decode

Last updated · how this page is sourced

Converts vector land-cover data (coastlines, water, land use) from any format GDAL can read into the intermediate polygon form tg-construct consumes. It is the stage that gives FlightGear terrain its surface, as opposed to its shape, and it is where a TerraGear build stops being cheap.

The tool is called ogr-decode. Some documentation says it was “renamed to poly-decode in recent versions”. It was not: poly-decode appears nowhere in the repository’s entire git history.

ReadsAny OGR datasource; shapefile, GeoPackage, PostGIS
WritesOne directory of intermediate polygons per material, under the work directory
Measured8 s for ten layers over one 1°×1° square; but see what it costs downstream

Usage

ogr-decode [options...] <work_dir> <datasource> [<layername>...]
Clip your data before you decode it. This is the one thing on this page that will cost you an hour if you skip it.

--spat looks like a bounding box. It is not: it calls GDAL’s SetSpatialFilterRect, which selects features whose bounding box intersects the rectangle and does nothing to their geometry. Natural Earth’s land layer is eleven features, one of which spans the globe. Asked for one degree square in California with --spat, the tool duly selected that feature and started tessellating every continent in it. It was still going eleven minutes later, on CLIPPED row 167 of 214 - center lat is -69.0625: Antarctica.

Clip the geometry first with ogr2ogr. The same job then took 0.02 seconds.

A worked example

Land cover for N37W123, the square used throughout this site, from OpenStreetMap and Natural Earth.

1. Clip every layer to the region

ogr2ogr -f "ESRI Shapefile" -clipsrc -123 37 -122 38 \
        clipped/landmass.shp ne_10m_land.shp ne_10m_land

ogr2ogr -f "ESRI Shapefile" -clipsrc -123 37 -122 38 \
        clipped/water.shp   norcal/gis_osm_water_a_free_1.shp
ogr2ogr -f "ESRI Shapefile" -clipsrc -123 37 -122 38 \
        clipped/landuse.shp norcal/gis_osm_landuse_a_free_1.shp
ogr2ogr -f "ESRI Shapefile" -clipsrc -123 37 -122 38 \
        clipped/natural.shp norcal/gis_osm_natural_a_free_1.shp

Clipping the three OSM layers took four seconds and reduced a 3.1 GB regional extract to a few thousand features.

2. Decode each material

OSM’s fclass values are not TerraGear area types, so each run selects a set of classes with --where and assigns one area type. The valid names are in /usr/local/share/TerraGear/default_priorities.txt: read it before choosing.

ogr-decode --area-type Default --max-segment 500 \
  $WORK/Landmass clipped/landmass.shp landmass

ogr-decode --area-type Lake --max-segment 500 \
  --where "fclass IN ('water','reservoir','riverbank','dock','pond')" \
  $WORK/Lake clipped/water.shp water

ogr-decode --area-type Marsh --max-segment 500 \
  --where "fclass LIKE 'wetland%'" \
  $WORK/Marsh clipped/water.shp water

ogr-decode --area-type EvergreenForest --max-segment 500 \
  --where "fclass = 'forest'" \
  $WORK/Forest clipped/landuse.shp landuse

ogr-decode --area-type Urban --max-segment 500 \
  --where "fclass IN ('residential','retail','commercial')" \
  $WORK/Urban clipped/landuse.shp landuse

And so on for grass, scrub, industrial, crops and sand. The full script is published with the build record.

This mapping is San Francisco’s. Ten area types out of the 96 TerraGear defines, chosen because they are what this square is made of. Run somewhere with different ground and it starts to show: in a German square the largest class by count is meadow, which is lumped in with mown grass above, and vineyards come out as ploughed field. There is a fourteen-layer version built on a second square: along with why counting polygons is the wrong measure, and how to tell a gap in your data from ground nobody has mapped.
Ten layers, one 1°×1° square. Eight seconds in total.
DirectoryArea typeFeaturesTime
LandmassDefault20 s
LakeLake2,0321 s
MarshMarsh4480 s
ForestEvergreenForest7,2463 s
GrassGrassland12,0761 s
ScrubScrub5,5641 s
UrbanUrban8,1201 s
IndustrialIndustrial7381 s
CropDryCrop1,2780 s
SandSand2010 s
Land cover map of San Francisco Bay: blue ocean and bay, mauve urban
       areas on the peninsula and east bay, dark green forest in the coastal
       hills, olive scrub and grass inland.
The ten decoded layers, drawn in the order tg-construct resolves them. This is what the terrain’s surface is made of: the real coastline, San Francisco and Oakland in urban grey, the Santa Cruz Mountains in forest, the salt ponds at the south end of the bay, and the Crystal Springs reservoirs as thin lines down the peninsula.

3. Build with them

Drop --ignore-landmass and add the directories to the load list:

tg-construct --work-dir=$WORK --output-dir=$OUT \
             --min-lon=-123 --max-lon=-122 --min-lat=37 --max-lat=38 \
             --priorities=/usr/local/share/TerraGear/default_priorities.txt \
             --threads \
             SRTMGL1 Landmass Lake Marsh Forest Grass Scrub Urban \
             Industrial Crop Sand AirportArea AirportObj

What land cover costs

The decode itself is trivial. What it does to the construction stage is not; same square, same elevation, same airports, the only difference being the ten layers above:

Without land coverWith
tg-construct15 s286 s
Peak memory81 MB2,217 MB
Output3.5 MB50 MB
Vertices115,9891,265,905

Nineteen times the wall clock, twenty-seven times the memory, fourteen times the disk. If you are sizing a machine for a large build, this is the number that matters: not the area on its own. See Hardware and build times.

Options

OptionWhat it does
--area-type typeMaterial for every object from this source
--area-type-column colnameTake the material from an attribute; overrides the above
--line-width widthWidth in metres for line features
--line-width-column colnameTake line width from an attribute
--point-width widthSize in metres of the square generated from a point
--point-width-column colnameTake point width from an attribute
--max-segment lengthMaximum segment length in metres; subdivides long edges
--where queryAttribute filter, SQL WHERE syntax
--spat xmin ymin xmax ymaxSelects features by bounding box. Does not clip geometry: see above
--start-record nBegin at a record number, first is 0
--continue-on-errorsCarry on when the input looks wrong
--log-level prioritybulk, debug, info, warn, alert

What it replaced

tgvpf, documented on terragear.org through the 2000s as “generate scenery polygons from VPF data”. It read VMAP0; the public vector dataset of the time; in Vector Product Format, and took a 10°×10° --chunk or explicit longitude and latitude bounds. ogr-decode does the same job through GDAL, which is why it accepts shapefiles, GeoPackages and PostGIS rather than one format.

Related

cliff-decode handles cliff lines specifically. poly2ogr goes the other way, turning TerraGear’s intermediate polygons back into an OGR datasource for inspection in a GIS viewer; though on this build it ran for 25 minutes on a 100 KB input with memory climbing past 2.5 GB, and was killed. That is an observation, not a diagnosis.