Rendering with Mapnik

Mapnik Overview

Mapnik is the rendering library and engine behind the familiar cartography of OSM. It can be used to render tiles or serve WMS and is often found powering desktop and web applications.

As Mapnik has great python bindings, we will simply be interacting with the Mapnik library through a variety of python tools in examples below.

Note

This guide focuses on Mapnik 0.7.1 (not yet Mapnik2)

Bounding Boxes

The next few sections are all going to depend on you understanding the basics of bounding boxes. For details we recommend giving a read to: http://weait.com/content/map-tiles-and-bounding-boxes

Basically, Mapnik and most other GIS software accepts bounding boxes in any coordinate system but always expects them to be composed of 4 numbers, made up of two pairs of x,y coordinates. The X is longitude, the Y is latitude, and a bbox is made of up (minx,miny,maxx,maxy).

An example bounding box for the whole globe in WGS 84 is:

-180, -90, 180, 90

And an example bounding box for the whole globe in Google Mercator is:

-20037508.34, -20037508.34, 20037508.34, 20037508.34

Note

You will see the above global mercator bbox appear in the code of the Stylesheets and in the Openlayers Map Projection.

Using generate_xml.py

Now its time to set up the stylesheet and a script called generate_xml.py can help with this. Basically you need to pass your database settings (database name and user/password) to be placed in the Mapnik XML stylesheets.

First head into the stylesheets directory (created when you uzipped the package in Accessing the OSM Mapnik Cartography):

cd projects/osm-rendering/stylesheets/

Now check out the options (and defaults) the script knows about:

./generate_xml.py -h

You should receive output like:

Usage: generate_xml.py <template xml> <output xml> <parameters>

Full help:

$ generate_xml.py -h (or –help for possible options)

Read ‘osm.xml’ and print resulting xml to stdout:

$ generate_xml.py osm.xml

Read template, save output xml, and pass variables as options

$ generate_xml.py osm.xml my_osm.xml –dbname spain –user postgres –host ‘’

Options:

--version	show program’s version number and exit
-h, --help	show this help message and exit
--inc=INC	Includes dir (default: ‘inc’)
--accept-none	Interpret lacking value as unneeded
--symbols=SYMBOLS
	Set value of ‘symbols’ (default: ‘symbols’)
--epsg=EPSG	Set value of ‘epsg’ (default: ‘900913’)
--world_boundaries=WORLD_BOUNDARIES
	Set value of ‘world_boundaries’ (default: ‘world_boundaries’)
--prefix=PREFIX
	Set value of ‘prefix’ (default: ‘planet_osm’)
--password=PASSWORD
	Set value of ‘password’
--host=HOST	Set value of ‘host’
--port=PORT	Set value of ‘port’
--user=USER	Set value of ‘user’
--dbname=DBNAME
	Set value of ‘dbname’
--estimate_extent=ESTIMATE_EXTENT
	Set value of ‘estimate_extent’ (default: ‘false’)
--extent=EXTENT
	Set value of ‘extent’ (default: ‘-20037508,-19929239,20037508,19929239’)

Because generate_xml.py already knows the right directory structure for the symbols and shapefiles all we need to do is issue this command:

./generate_xml.py --dbname osm --user postgres --accept-none

On Windows all postgres users must have a password so do:

./generate_xml.py --dbname osm --user postgres --password osm --accept-none

Stylesheets in EPSG:4326

If you imported data into PostGIS using the –latlong flag with osm2pgsql then you’ll need to do:

./generate_xml.py --dbname osm --extent '-179,-89,179,89' --epsg 4326 --accept-none

And you will also need to customize the stylesheet which contains the references to the shapefiles which we reprojected ( see Reprojecting Shapefiles):

Index: osm.xml
===================================================================
--- osm.xml     (revision 22361)
+++ osm.xml     (working copy)
@@ -6,7 +6,7 @@
    <!-- This stylesheet uses features only available in mapnik builds with
        libxml2 as the XML parser. Furthermore, there are other features
        and behaviour that necessitate an upgrade to mapnik 0.7.1 -->
-<Map bgcolor="#b5d0d0" srs="&srs900913;" minimum_version="0.7.1">
+<Map bgcolor="#b5d0d0" srs="&srs4326;" minimum_version="0.7.1">
    &fontset-settings;
    <Style name="turning_circle-casing">
        <!-- Render turning circle casings.
Index: inc/layer-shapefiles.xml.inc
===================================================================
--- inc/layer-shapefiles.xml.inc        (revision 22361)
+++ inc/layer-shapefiles.xml.inc        (working copy)
@@ -43,25 +43,25 @@
        </PolygonSymbolizer>
        </Rule>
    </Style>
-<Layer name="world" status="on" srs="&srs900913;">
+<Layer name="world" status="on" srs="&srs4326;">
        <StyleName>world</StyleName>
        <Datasource>
        <Parameter name="type">shape</Parameter>
-      <Parameter name="file">&world_boundaries;/shoreline_300</Parameter>
+      <Parameter name="file">&world_boundaries;/shoreline_300_ll</Parameter>
        </Datasource>
    </Layer>
-<Layer name="coast-poly" status="on" srs="&srs900913;">
+<Layer name="coast-poly" status="on" srs="&srs4326;">
        <StyleName>coast-poly</StyleName>
        <Datasource>
        <Parameter name="type">shape</Parameter>
-      <Parameter name="file">&world_boundaries;/processed_p</Parameter>
+      <Parameter name="file">&world_boundaries;/processed_p_ll</Parameter>
        </Datasource>
    </Layer>
-<Layer name="builtup" status="on" srs="&srsmercator;">
+<Layer name="builtup" status="on" srs="&srs4326;">
        <StyleName>builtup</StyleName>
        <Datasource>
        <Parameter name="type">shape</Parameter>
-      <Parameter name="file">&world_boundaries;/builtup_area</Parameter>
+      <Parameter name="file">&world_boundaries;/builtup_area_ll</Parameter>
        </Datasource>
    </Layer>
    <Layer name="necountries" status="on" srs="&srs4326;">

Unicode Fallback texts

One last customization you’ll likely want if you are mapping areas with unicode text is to enable fallback fonts:

Index: inc/fontset-settings.xml.inc.template
===================================================================
--- inc/fontset-settings.xml.inc.template       (revision 22361)
+++ inc/fontset-settings.xml.inc.template       (working copy)
@@ -7,13 +7,13 @@

 <FontSet name="book-fonts">
   <Font face_name="DejaVu Sans Book" />
-  <!--Font face_name="unifont Medium" /-->
+  <Font face_name="unifont Medium" />
 </FontSet>
 <FontSet name="bold-fonts">
   <Font face_name="DejaVu Sans Bold" />
-  <!--Font face_name="unifont Medium" /-->
+  <Font face_name="unifont Medium" />
 </FontSet>
 <FontSet name="oblique-fonts">
   <Font face_name="DejaVu Sans Oblique" />
-  <!--Font face_name="unifont Medium" /-->
+  <Font face_name="unifont Medium" />
 </FontSet>

nik2img.py

Now that your osm.xml is set up, we can test a very simple rendering with nik2img.

This should create a blue and white world map:

nik2img.py osm.xml world.png

Warning

If you get an error starting with “inc/entities.xml.inc:2: I/O warning : failed” see: Troubleshooting osm.xml rendering errors

Note

You might wonder why the last image was for the entire world+. Well, nik2img is designed to zoom to the full extent of the data referenced by the Stylesheets. While we have currently only imported a tiny chunk of Barcelona into our PostGIS database, the stylesheets also reference global shapefiles, so these expand the cumulative extent to the globe.

And this should create an image of barcelona:

nik2img.py osm.xml barcelona.png -e 216471.34 5052350.67 266818.65 5096245.2

Note

You might ask: How did we get those extents to pass to nik2img?

We grabbed them from the postgres tables using the command (just remove the middle comma):

$ psql barcelona -c "Select ST_Extent(way) from planet_osm_roads;"
                   st_extent
-----------------------------------------------
 BOX(216471.34 5052350.67,266818.65 5096245.2)

TileLite

Now head over and serve some tiles from this stylesheet:

cd ../tilelite

Note

If you don’t yet have a tilelite directory, go ahead an use mercurial to check one out:

hg clone https://springmeyer@bitbucket.org/springmeyer/tilelite
cd tilelite
sudo python setup.py install

You can also download the code as a zip archive

Then launch the TileLite development server with the command:

liteserv.py ../stylesheets/osm.xml

Now you’ve got the server ready to respond to tile request on localhost. Open up firefox and go to:

http://localhost:8000 # you should see the TileLite welcome page

Finally to check your tiles on a map open the OpenLayers demo file at:

tilelite/demo/openlayers.html

# note: you can open this file from the command line with

xdg-open tilelite/demo/openlayers.html

You should be able to zoom into Barcelona and once you get close you should see your OSM data appearing.

Note

If you want more data to show up then you’ll need to import a bigger extract. Header over to Geofabrik for extracts.

generate_tiles.py

For fast tile serving its best to cache tiles. TileLite can do this by passing the –caching flag to liteserv.py, but an even easier way is to pre-generate the tiles and then just drop them on a webserver. This way you won’t even need to install mapnik on your web host (or PostGIS).

Warning

This works for small areas, but for large areas it won’t because the time and storage space needed to cache (even a countries worth of tiles) is enormous. So, for large areas mod_tile is the best solution, because it can cache on-demand only those tiles requested and clear out infrequently viewed tiles to save space.

First head into the stylesheets directory:

cd projects/osm-rendering/stylesheets

Then open up generate_tiles.py for editing:

gedit generate_tiles.py

This script is just a bit of python, meant to customize to your needs.

1. On line 13, change NUM_THREADS to 1.

2. On line 199 edit your bounding box in the script to:

   bbox = (1.94459513295834,41.2706895358557,2.39687271383137,41.5663815729642)
   

3. Then delete all code below line 201

Note

How did we get the bounding box for generate_tiles.py?

Well, generate_tiles.py excepts the bbox in EPSG:4326 (aka WGS 84), so we used postgres to fetch the extents of barcelona and then translate them to that coordinate system:

$ psql barcelona -c "select ST_Extent(ST_Transform(way,4326)) from planet_osm_roads;"
                                st_extent
--------------------------------------------------------------------------
 BOX(1.94459513295834 41.2706895358557,2.39687271383137 41.5663815729642)

Now, lets create a directory for tiles:

mkdir tiles

And set a few environment variables the script expects, then run it:

export MAPNIK_MAP_FILE=osm.xml
export MAPNIK_TILE_DIR=tiles
./generate_tiles.py

Yahoo, you should now have tiles being created by the hundred!

Almost Done

At this point we are almost done. Depending on your interests you can now can now point OpenLayers at these tiles instead of at TileLite (port 8000). For a hint about how to do that skip ahead to Making a Slippy Map with OpenLayers.

Or for more detail about on-demand rendering take a look at Serving Map Tiles.