Custom OpenStreetMap Rendering – OpenTrackMap Experience

After 5 years of its existence, the OpenSteetMap [1] is becoming to be an important and valuable source of a geographic data for all people on the world. Although initially targeted to provide a map of cities for routing services, it can be exploited to other and often unexpected purposes. Such an utilization is an effort to map a network of hiking tracks of the Czech Tourist Club [2]. To support and apply this endeavour, the OpenTrackMap [3] project was started. Its aim is to primarily provide a customized rendering style for Mapnik renderer which emphasizes map features important to tourists and displays a layer with hiking tracks. This article presents obstacles which such project must face and it can be used as a tutorial for other projects of similar type. Motivation and Background Small portable devices equipped with GPS chips and high bandwidth Internet connection are more frequently becoming our attendants on trips to nature. Although they are sometimes more powerful than personal computers used in near past, they have not enough processing power to interactively render vector geographic data of larger areas with good visual quality. For this reason Web-based raster geographic data services such as TMS [4], WMS [5,6] or Slippy Map [7] combined with geocoding [8] and routing services are more suitable for this platform. Rendering requests are sent to a server and rendered tiles are then transferred back to a client as images using HTTP protocol. This transfers heavy load of data rendering to distant servers. Although servers like [9] provides hiking maps of Czech Republic, they are almost impossible to use on the portable devices with an open source software due to licensing or technical Geinformatics FCE CTU 2009 5 Bartoň R.: Custom OpenStreetMap Rendering – OpenTrackMap Experience restrictions. In the field of the open source, the OpenSteetMap is only available source of geographic data of the entire planet. On the other hand, portion of already mapped tracks of CTC hiking tracks network in the OpenStreetMap is only partial. This implies need of publicly accessible and regularly updated service which displays hiking tracks from the OpenSteetMap for Czech Republic. It should provide fast enough feedback for mappers at home and it could also be used in a terrain. First project to develop fully open source portable device with GPS capabilities is Openmoko [10]. So a primary target of the OpenTrackMap project is to provide maps for a TangoGPS application on the Openmoko platform.


Motivation and Background
Small portable devices equipped with GPS chips and high bandwidth Internet connection are more frequently becoming our attendants on trips to nature.Although they are sometimes more powerful than personal computers used in near past, they have not enough processing power to interactively render vector geographic data of larger areas with good visual quality.
For this reason Web-based raster geographic data services such as TMS [4], WMS [5,6] or Slippy Map [7] combined with geocoding [8] and routing services are more suitable for this platform.Rendering requests are sent to a server and rendered tiles are then transferred back to a client as images using HTTP protocol.This transfers heavy load of data rendering to distant servers.
Although servers like [9] provides hiking maps of Czech Republic, they are almost impossible to use on the portable devices with an open source software due to licensing or technical restrictions.In the field of the open source, the OpenSteetMap is only available source of geographic data of the entire planet.On the other hand, portion of already mapped tracks of CTC hiking tracks network in the OpenStreetMap is only partial.This implies need of publicly accessible and regularly updated service which displays hiking tracks from the OpenSteetMap for Czech Republic.It should provide fast enough feedback for mappers at home and it could also be used in a terrain.
First project to develop fully open source portable device with GPS capabilities is Openmoko [10].So a primary target of the OpenTrackMap project is to provide maps for a TangoGPS application on the Openmoko platform.

Similar Projects
Before getting down to start developing the OpenTrackMap, proper research of existing projects with same scope was made.Most similar of them is an openstreetmap.cz[11] with its display of hill shading and Czech hiking tracks layer.Unfortunately, it is not regularly updated a it does not provide a baked layer with all map features together.Moreover, its rendering style is not high quality.
Projects like a Freemap Slovakia [12] or an OSMC Reit-und Wanderkarte [13] are more mature but they covers different countries while projects [14,15,16,17] are also customized OpenStreetMap renderings but aimed to different audience.

Requested Features
With respect to objectives of the project, following list with requested features can be written up: Display hiking tracks of the CTC network with their color, that is use red, green, blue and yellow lines for them.Properly display two to four parallel tracks using dashed lines of appropriate colors.Distinguish between regular, local, learning, skiing and horse tracks using shields with established symbols [18,19,20].
Overlay a layer with contour lines generated from publicly available elevation data.
Add a layer with hill shading also extracted from elevation data.Display additional point objects attractive for hikers (i.e. guideposts, castles, peaks, pubs, etc.) with appropriate icons that are not included in a default OpenStreetMap rendering style.
Provide both single TMS service for each layer and a complete TMS service with all layers baked into a single layer.
Setup automatic data synchronization from a main OpenStreetMap data repository with reasonable update interval and delay.

Available Renderes
From the list of known renderers of OpenSteetMap data, only three are more interesting.

Mapnik
The Mapnik [21] is the most features and also the most used renderer available.Basically, it is a C++ library with Python bindings generator accompanied with rendering scripts using the library.Appearance of the rendered map is specified in custom XML style document [22] but a CSS like styling syntax can be also used and then compiled to the default XML style with a Cascadenik tool [23].There is also a simple GUI application for Mapnik styles preview called a Mapnik Viewer [24].On-demand tile rendering is possible with mod tile [25] or TileLite [26].
Possible input vector formats of the Mapnik are following:

PostGIS database.
Oracle Spatial database.

SQLite database.
And many others through OGR library [28] like GRASS vector format, GPX or KML.
Input raster formats supported are: TIFF image.
And many others through GDAL library [29] like GRASS raster format, GeoTIFF or JPEG2000.
It is optional to choose between two rendering libraries [30]: Anti-Grain Geometry and Cairo.This implies possible output formats: PNG raster image (AGG, Cairo).

Osmarender
Osmarender [31] is all in all an XSLT template, an XML styling document and a Perl script that converts the OSM XML format to a SVG vector image.It is used for distributed rendering in a Tiles@home [32] project, but it has not so much features like the Mapnik and for the OpenTrackMap is quite insufficient regarding to the requested features list.

Kosmos
This software [33] is just simple GUI to style and render the OpenStreetMap data on a desktop.It is written in C# for Windows and thus it needs .NET Framework to run and it is not much compatible with Mono implementation of it.This means that the Mapnik is only option for renderer for the OpenTrackMap.

Source of Elevation Data
To display contour lines and hill shading layer in the map, elevation height map data are needed.Well-known and free source of elevation data for nearly whole world is lidar data obtained from NASA's SRTM project [34].Resolution of its grid is 3 arc seconds, which is approximately 90 meters in the area of Czech Republic, while it is claimed that their vertical accuracy is 10 meters.There are many servers run by U. S. government organizations providing this dataset.They differ in quality of a data post-processing, that is gap filling of missing data.For the OpenTrackMap project, just USGS [35] and GLCF [36] data was tried.
In the time of the project creation a new source of elevation data was introduced.It is an ASTER GDEM [37] stereoscopic data from joint project of the NASA and the Japanese Ministry of Economy, Trade, and Industry (METI).Its resolution is 1 arc second and expected vertical accuracy is 7 to 14 meters.Although it is not yet clear whether this data can be used in open source projects like the OpenTrackMap is, some experiments with the data was made to prove that they can be useful.) and the GLCF SRTM (figure 2.) data seem pretty much the same, the ASTER GDEM (figure 3.) data contains disturbing artifacts of artificial edges of a splotch shape.This is probably caused by the fact that the ASTER GDEM data was filled in with the SRTM data in areas of missing data occluded by clouds and that there may be some constant value offset between those sources.There are also some missing data  in the SRTM sources but they are few and does not mind so much.They produce small but dense clusters of contour lines as is shown on the figure 4.
Conclusion of this investigation is that though the ASTER GDEM data was very promising at the beginning, they turned out to be inapplicable in practice while there is no significant difference among foreign sources of the SRTM data.

Infractructure
A complete infrastructure of the OpenStreetMap project is quite extensive as a figure 5.
shows.Fortunately, only its subset, marked by a red border on the image, is needed to be setupped to run a regularly updated tile service.
At the beginning there is a source of whole planet data in a database dump form in a compressed OSM XML format.It is imported to a PostgreSQL [39] database with a PostGIS [40] Figure 4: A detail of a dense cluster of contour lines generated from the SRTM data on a place of the missing data.extension using a osm2pgsql [41] tool.An Apache Web server with a mod tile [42] module is running on the server machine.This module directs tile requests to a tile cache on a server's disk storage if tile is in the cache is present and up-to-date or to a resident daemon which renders the tile using the Mapnik library if it is not.A JavaScript AJAX-based [43] web interface called an OpenLayers [44] inlined in a HTML document is responsible for the tiled Geinformatics FCE CTU 2009 map display to a user though a web browser while mobile or desktop applications are using the tile requests directly using the SlippyMap [7] scheme.

Raster and Vector Data Storage
Vector Data First step to establish the OpenSteetMap based tile server is to prepare the vector data for rendering.Though the Mapnik supports rendering from the OSM XML files directly, it is more efficient and customizable to import the OSM XML data to the PostgreSQL database with the PostGIS extension.The only way how to do this is to use the osm2pgsql because an another tool for the OpenSteetMap data processing an osmosis [45] exports to the PostGIS database using a different schema.
Let us suppose that there is a fresh installation of a PostgreSQL 8.4, a PostGIS 1.4.0 and a SVN version of the osm2pgsql.Then the following steps will prepare the data for the Mapnik rendering.They are relevant to Ubuntu Karmic Koala desktop but they should be more or less the same for other systems.See [46,47] for the details.
First we create a new database with a PL/pgSQL and PostGIS support: The osm2pgsql operates in two different modes.First, the default one, uses a memory efficient database schema but it does not support further changes in data using this tool.Second is a slim mode which adds additional tables that are needed for incremental updates support.Slim mode heavily utilizes array types so we need to include an initarray PostgreSQL module from contrib package: $ psql -d osm -f /usr/share/postgresql/8.4/contrib/_init.sql In the osm2pgsql package is a 900913.sqlscript which has to be imported to the database to include a Google projection: The osmosis command here serves for bounding the data to a region of the Czech Republic.Although the osm2pgsql can bound the data too, this feature is quite slow comparing to the osmosis way.The osmosis can read compressed OSM XML files directly but there is a bug in the implementation.The osm2pgsql has to be run from its directory, the -s option enables the slim mode and the -m option specifies that a Merkaartor projection is used.

Contour Lines
A generation and an import of the contour lines to the database in a vector form goes with several steps.First one, the HGT format conversion, is same as for the hill shading.Then a shapefile with the contours is generated using a gdal contour tool from the GDAL library.Then it's imported with a shp2pgsql which is part of the PostGIS.

Mapnik Stylesheet Design
When designing the Mapnik XML stylesheet for the OpenTrackMap project, a default Open-SteetMap stylesheet from a OpenStreetMap SVN [51] was taken as a template and customizing modifications was applied into it.
Hiking tracks of the CTC are marked with one of four colors: red, green, blue and yellow [18,19,20].They are mostly mapped using relations and tagged with kct red, kct green, kct blue and kct yellow tags respectively.Some of the allowed values are major, local, learning, ski and horse [54] depending on the type of the track.Slovak hiking tracks are tagged with marked trail <color> counterparts of CTC-style tags and they are taken into account too.Since the hiking tracks of different colors can go in parallel on the same road, special care has to be taken to display them correctly and legibly.
The main problem about this is that road segments that are part of multiple hiking track relations of a different color has to be detected and rendered appropriately.The OpenTrackMap face this problem by copying segments of relations with kct <color> tags to a new database table of lines a using Python script [55].Disadvantage of this solution is that it is not much friendly to regularly performed incremental updates since whole table has to be recreated each time the data are updated.Better option is to modify osm2pgsql tool to do this automatically with each update and it is a space for further advancement.
The tracks are then rendered from the custom table with the copied segments and a default table without the relations with following styles and layers using dashed lines of intermittent colors: This is the style and layer just for a red-green combination of the hiking track colors and only for higher zoom levels.In the full stylesheet [52,53], there are styles and layers for all combinations of four colors and for three ranges of the zoom levels, that is fifteen different styles with three rules each.
Track type is visualized using shields with symbols placed on the lines: Bartoň R.: Custom OpenStreetMap Rendering -OpenTrackMap Experience An international hiking tracks rendering is easier to set up comparing to the local ones:

Static Tile Rendering
For the static tiles rendering, a custom multithread version of a OpenStreetMap's generate tiles.pyPython script was written [56] because in early times of the OpenTrackMap project existence, no such script was available.The multithreading was introduced to this script as far as with changeset 17484 [57].Rendering times for different zoom levels with the OpenTrackMap Mapnik stylesheet are discussed in a section "Experimental Results".

On-Demand Tile Rendering
A mod tile [25] is an Apache module that allows us to serve, cache and render tiles with a Mapnik in several layers using different XML stylesheets.It communicates with standalone renderd daemon, that performs actual rendering, via sockets.
An Apache virtual host configuration for the Web server with the mod tile and statically served tiles could look like this: The mod tile configuration comprehends a specification of a renderd configuration file and several options for tuning caching behavior.Next is a root location where an index.htmlfile with the OpenLayers JavaScript interface is coded.The last location is directory with the statically rendered tiles.
The renderd configuration file with a single on-demand rendered layer accessible under a http://opentrackmap.no-ip.org/default/URI is following: Here, the socketname option must match the ModTileRenderdSocketName option in the Apache virtual host configuration.To enable another layer, just the [default] section is needed to be copied.
Since the mod tile is still quite a fresh project with its flaws, it is necessary to register a cron job that looks after renderd daemon state:

Automatic Server Updates
The automatic server updates are necessary for a quick feedback during track editing.Mappers would like to see their edits in real render as soon as possible.Concerning a technical restrictions, a database updated once per hour seems to be a good compromise.The main problem with this is a question how to effectively import full and incremental data files of a bounded area of Czech Republic.Also remember that the relations2lines.pyscript has to be re-run each time the data are changed.

Full Update
There are three different possibilities for the full updates.A first is to import a planet file [48] with an osm2pgsql tool and use its -b option to specify the bounding box limits.See a full update osm2pgsql.pyscript [58] for details.A second is to bound data with an osmosis tool and then import the result with the osm2pgsql.This is placed in a full update osmosis.pyscript [59].Finally, the third option is to use an already bounded OSM data from a publicly available service such as [60,61] are.
First solution turned out to be too slow comparing to the osmosis way -16:13 h vs. 1:50 h.This is probably caused by a fact that a bounding algorithm implementation using memory operations written in Java is much faster than a PostGIS implementation of it.Using the data bounded by another developers has its drawbacks in higher delay to an official OpenSteetMap database and in a need of knowing an exact timestamp and bounding area parameters of the data.For this reasons, the second way is currently used in the OpenTrackMap project.

Incremental Updates
After a first full update is performed, another updates started with a regular interval can be incremental to the full one.The osm2pgsql supports this feature but data has to be imported in a so called slim mode which stores overhead information to additional tables.To bound changeset data more ways are also possible.Using the osmosis tool to bound a changeset of a larger area is not much efficient because the changeset is applied to an old full data first, a new full data are bounded and then a bounded changeset is determined by comparison with the old full data.On the other hand, this can be more optimal for smaller areas.An incremental updates osm2pgsql.pyscript [62], that is utilized in the OpenTrackMap project, uses direct changeset import with the osm2pgsql tool and with the -b option.Figure 6.shows an OpenTrackMap full-featured statically served layer in a web browser on a web page with an OpenLayers interface [44] while figure 7 shows a default OpenSteetMap layer as a base layer and a layer with just hiking tracks rendered on-demand using the mod tile as an overlay.

Practical Results
Figure 8. shows the OpenTrackMap full and tracks-only layers in a TangoGPS application which can be run on an Openmoko Neo Freerunner mobile smartphone.
Finally, figure 9. presents the OpenTrackMap on a Sony Ericsson G705 cell phone with a Java Mobile support in a Mobile GMaps application [63].For this application simple wrapper script that transforms Google URL scheme to SlippyMap scheme has to be written for the web server: from mod_python import apache from mod_python.util import FieldStorage

Experimental Results
Few experiments with the generate tiles.pyscript [56]     All tests was done on an Intel Core 2 Duo 3 GHz machine with a 2 GB of RAM and two processing cores using one and two rendering threads.A main finding about them is that a parallel speedup of the two-thread rendering is not approaching 2 but it is approximately 1.56.This is because the one-thread rendering is already partially parallel since the Mapnik rendering and a PostgreSQL database look-ups run in separate processes.It would be interesting to try how would the parallel rendering scale on computers with more cores or in a

Figure 1 :
Figure 1: Hill shading map of central Bohemia near capital city Prague generated using the USGS SRTM data in WGS84 projection.The red rectangular area in the left part of the figure is an extent of the cut-out right part.The completely white area is caused by missing data.

Figure 2 :
Figure 2: Hill shading map of central Bohemia near capital city Prague generated using the GLCF SRTM data in WGS84 projection.The red rectangular area in the left part of the figure is an extent of the cut-out right part.The completely white area is caused by missing data.

Figure 3 :
Figure 3: Hill shading map of central Bohemia near capital city Prague generated using the ASTER GDEM data in WGS84 projection.There are many visible artifacts of a splotch shape.The red rectangular area in the left part of the figure is an extent of the cut-out right part and shows those artifacts in detail.

Figure 5 :
Figure 5: Scheme of the OpenStreetMap infrastructure.Source of image [38]

Figure 6 :
Figure 6: An OpenLayers based OpenTrackMap web interface.A baked layer with all features displayed.

Figure 7 :
Figure 7: An OpenLayers based OpenTrackMap web interface.A layer served from OpenStreetMap displayed with a hiking tracks layer overlay rendered on-demand.

Figure 8 :
Figure 8: The full statically served layer (left) and the hiking tracks on-demand rendered layer in a TangoGPS application.

Figure 9 :
Figure 9: The OpenTrackMap on a mobile device Sony Ericsson G705 in a Mobile GMaps application.
was performed.Their results are on the figures 10., 11. and 12.While the first two show summarized rendering times (Y axis) in minutes for rendering of tiles up to certain zoom level (X axis) in a linear and a logarithmic scale, last figure displays average rendering time per tile in seconds depending on the maximal zoom level rendered.

Figure 10 :
Figure 10: Rendering time in minutes dependence on a maximal zoom level rendered -a linear scale.

Figure 11 :
Figure 11: Rendering time in minutes dependence on a maximal zoom level rendered -a logarithmic scale.

Figure 12 :
Figure 12: Average tile rendering time in seconds dependence on a maximal zoom level rendered.