Tilemill was the original tool for the development of the openstreetmap-carto style. It moved out of the Mapbox profile and shifted to an open source community-driven organization, with its own organization and contributor model: tilemill-project. At the moment, the suggested tool for the autohoring of OpenStreetMap stylesheets is Kosmtik. TileMill is not officially supported.
A PostGIS database is needed and can be installed locally (suggested) or remotely (might be slow, depending on the network).
Other distributions like Debian might require changes and are not tested.
This procedure is updated to the version of OpenStreetMap Carto available at the time of writing. To get the correct installation procedure, the INSTALL history should be checked, considering that the OpenStreetMap Carto maintainers use to keep the INSTALL page updated. Check also the README changelog.
This install procedure also supports WSL - Windows Subsystem for Linux.
Make sure your Ubuntu system is fully up-to-date:
Previous command returns the Ubuntu version.
To update the system:
sudo apt-get update sudo apt-get -y upgrade
If on a brand new system you also want to do
sudo apt-get dist-upgrade && sudo shutdown -r now.
sudo apt-get -y install curl unzip gdal-bin tar wget bzip2 build-essential clang
For the subsequent installation steps, we suppose that
cd defaults to your home directory.
Importing and managing map data takes a lot of RAM and a swap is generally needed.
To check whether a swap partition is already configured on your system, use one of the following two commands:
Reports the swap usage summary (no output means missing swap):
Display amount of free and used memory in the system (check the line specifying Swap):
If you do not have an active swap partition, especially if your physical memory is small, you should add a swap file. First we use
fallocate command to create a file. For example, create a file named swapfile with 2G capacity in root file system:
sudo fallocate -l 2G /swapfile
Then make sure only root can read and write to it.
sudo chmod 600 /swapfile
Format it to swap:
sudo mkswap /swapfile
Enable the swap file
sudo swapon /swapfile
The Operating System tuning adopted by the OpenStreetMap tile servers can be found in the related Chef configuration.
Run locale to list what locales are currently defined for the current user account:
To set the en_GB locale:
export LANGUAGE=en_GB.UTF-8 export LANG=en_GB.UTF-8 export LC_ALL=en_GB.UTF-8
The exported variables can be put to the file
New locales can also be generated by issuing:
sudo locale-gen en_GB en_GB.UTF-8 sudo dpkg-reconfigure locales
Git should be already installed on Ubuntu 16.04.
git --version # to verify whether git is already installed sudo apt-get install -y git
To verify whether Node.js is already installed:
nodejs-legacy might be needed for TileMill (at least required with Ubuntu 16.04 at the time of writing).
sudo apt-get install -y nodejs-legacy npm
The following error when running TileMill is related to compatibility issues with nodejs and should be fixed by installing nodejs-legacy.
npm ERR! Failed at the email@example.com install script 'node-pre-gyp install --fallback-to-build'. npm ERR! Make sure you have the latest version of node.js and npm installed.`
Installing the latest version of nodejs is not suggested at the moment. If you really want to do it:
sudo apt-get install -y nodejs npm # this package might not support TileMill, see notes in this paragraph
Read nodejs for further information.
Optional elements (needed for the topcube module related to the client user interface, not needed if TileMill will only be run in server mode):
# Install the topcube module, UI prerequisite to TileMill: sudo apt-get install -y libgtk2.0-dev libwebkit-dev libwebkitgtk-dev
Installation of TileMill:
test -d ~/src || mkdir ~/src ; cd ~/src git clone https://github.com/tilemill-project/tilemill.git cd ~/src/tilemill npm install
cd ~/src/tilemill npm test # you can also run TileMill to test: see "Start TileMill" below.
See Start TileMill.
Some test might not pass (this does not mean that the installation is necessarily failed)
Notice that, when running
npm test, an error like the following indicates that your system does not have a modern enough libstdc++/gcc-base toolchain:
Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version GLIBCXX_3.4.20 not found (required by /node_modules/osrm/lib/binding/osrm.node)
If you are running Ubuntu older than 16.04 you can easily upgrade your libstdc++ version like:
sudo add-apt-repository ppa:ubuntu-toolchain-r/test sudo apt-get update -y sudo apt-get install -y libstdc++-5-dev
Read node-mapnik for further information.
Check that Python is installed:
python -V python3 -V
Otherwise Python needs to be installed.
This is necessary in order to run OpenStreetMap-Carto scripts/indexes.
sudo apt-get install -y python-yaml pip -V # to verify whether pip is already installed sudo apt-get install -y python-pip
The Mapnik Utilities package includes shapeindex.
sudo apt-get install -y mapnik-utils
cd /home/ubuntu/Documents/MapBox/project # if this directory is missing, start TileMill to create it git clone https://github.com/gravitystorm/openstreetmap-carto.git cd openstreetmap-carto
Read installation notes for further information.
Currently Noto fonts are used.
To install them (except Noto Emoji Regular and Noto Sans Arabic UI Regular/Bold):
sudo apt-get install -y fonts-noto-cjk fonts-noto-hinted fonts-noto-unhinted fonts-hanazono ttf-unifont
Installation of Noto Emoji Regular and Noto Sans Arabic UI Regular/Bold:
cd ~/src git clone https://github.com/googlei18n/noto-emoji.git git clone https://github.com/googlei18n/noto-fonts.git sudo cp noto-emoji/fonts/NotoColorEmoji.ttf noto-emoji/fonts/NotoEmoji-Regular.ttf /usr/share/fonts/truetype/noto sudo cp noto-fonts/hinted/NotoSansArabicUI-Regular.ttf noto-fonts/hinted/NotoNaskhArabicUI-Regular.ttf noto-fonts/hinted/NotoSansArabicUI-Bold.ttf noto-fonts/hinted/NotoNaskhArabicUI-Bold.ttf /usr/share/fonts/truetype/noto sudo fc-cache -fv sudo apt install fontconfig fc-list fc-list | grep Emoji cd openstreetmap-carto
DejaVu Sans is used as an optional fallback font for systems without Noto Sans. If all the Noto fonts are installed, it should never be used.
sudo apt-get install -y fonts-dejavu-core
Read font notes for further information.
cd /home/ubuntu/Documents/MapBox/project # if this directory is missing, start TileMill to create it cd openstreetmap-carto scripts/get-shapefiles.py
The actual shapefiles loaded by the OpenStreetMap tile servers are reported in the related Chef configuration.
Read scripted download for further information.
export PGHOST=localhost export PGPORT=5432 export PGUSER=postgres export PGPASSWORD=postgres_007%
If you are preparing a remote virtual machine, configure the firewall to allow remote access to the local port 20009.
If you run a cloud based VM, also the VM itself shall be set to open this port.
PostgreSQL is a relational database, and PostGIS is its spatial extender, which allows you to store geographic objects like map data in it; it serves a similar function to ESRI’s SDE or Oracle’s Spatial extension. PostgreSQL + PostGIS are used for a wide variety of features such as rendering maps, geocoding, and analysis.
Currently the tested versions for OpenstreetMap Carto are PostgreSQL 9.5 and PostGIS 2.2:
Also older PostgreSQL version should be suitable.
On Ubuntu there are pre-packaged versions of both postgis and postgresql, so these can simply be installed via the Ubuntu package manager.
sudo apt-get update sudo apt-get install -y postgresql postgis pgadmin3 postgresql-contrib
With WSL, you need to start the db:
sudo service postgresql start
Note: used PostgreSQL port is 5432 (default).
A user named postgres will be created during the installation process.
sudo -u postgres psql postgres \password postgres
Alternative procedure (useful if you get authentication issues with the previous one):
sudo su - sudo -i -u postgres psql postgres \password postgres
Enter the following password twice:
This is just an example of password, you can use the one you prefer.
After entering the password, exit from psql with:
With the second procedure, also isssue:
exit # from 'sudo -i -u postgres' exit # from 'sudo su -'
Now you need to create a PostGIS database. The defaults of various programs including openstreetmap-carto (ref. project.mml) assume the database is called gis. You need to create a PostgreSQL database and set up a PostGIS extension on it.
The character encoding scheme to be used in the database is UTF8 and the adopted collation is en_GB.utf8. (The
U&"..." escaped Unicode syntax used in project.mml should work only when the server encoding is UTF8. This is also in line with what reported in the PostgreSQL Chef configuration code.)
export PGPASSWORD=postgres_007% HOSTNAME=localhost # set it to the actual ip address or host name psql -U postgres -h $HOSTNAME -c "CREATE DATABASE gis ENCODING 'UTF-8' LC_COLLATE 'en_GB.utf8' LC_CTYPE 'en_GB.utf8'" # alternative command: createdb -E UTF8 -l en_GB.UTF8 -O postgres gis
If you get the following error:
ERROR: invalid locale name: "en_GB.utf8"
then you need to add ‘en_GB.utf8’ locale using the following command:
sudo dpkg-reconfigure locales
And select “en_GB.UTF-8 UTF-8” in the first screen (“Locales to be generated”). Subsequently, restarting the db would be suggested:
sudo service postgresql restart
If you get the following error:
ERROR: new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII) HINT: Use the same encoding as in the template database, or use template0 as template.
(error generally happening with Ubuntu on Windows with WSL), then add also
TEMPLATE template0; e.g., use the following command:
psql -U postgres -h $HOSTNAME -c "CREATE DATABASE gis ENCODING 'UTF-8' LC_COLLATE 'en_GB.utf8' LC_CTYPE 'en_GB.utf8' TEMPLATE template0" # alternative command: createdb -E UTF8 -l en_GB.utf8 -O postgres -T template0 gis
Create the postgis and hstore extensions:
psql -U postgres -h $HOSTNAME -c "\connect gis" psql -U postgres -h $HOSTNAME -d gis -c "CREATE EXTENSION postgis" psql -U postgres -h $HOSTNAME -d gis -c "CREATE EXTENSION hstore"
If you get the following error
ERROR: could not open extension control file "/usr/share/postgresql/9.3/extension/postgis.control": No such file or directory
then you might be installing PostgreSQL 9.3 (instead of 9.5), for which you should also need:
sudo apt-get install postgis postgresql-9.3-postgis-scripts
Install it and repeat the create extension commands.
If you need to use a different tablespace than the default one, execute the following commands instead of the previous ones (example: the tablespace has location
export PGPASSWORD=postgres_007% HOSTNAME=localhost # set it to the actual ip address or host name sudo mkdir /mnt/db # Suppose this is the tablespace location sudo chown postgres:postgres /mnt/db psql -U postgres -h $HOSTNAME -c "CREATE TABLESPACE gists LOCATION '/mnt/db'" psql -U postgres -h $HOSTNAME -c "CREATE DATABASE gis ENCODING 'UTF-8' LC_COLLATE 'en_GB.utf8' LC_CTYPE 'en_GB.utf8' TABLESPACE gists" psql -U postgres -h $HOSTNAME -c "\connect gis" psql -U postgres -h $HOSTNAME -d gis -c "CREATE EXTENSION postgis" psql -U postgres -h $HOSTNAME -d gis -c "CREATE EXTENSION hstore"
In order for the application to access the gis database, a DB user with the same name of your UNIX user is needed. Let’s suppose your UNIX ue is tileserver.
sudo su - sudo -i -u postgres createuser tileserver psql grant all privileges on database gis to tileserver; \q exit exit
To remotely access PostgreSQL, you need to edit pg_hba.conf:
sudo vi /etc/postgresql/9.5/main/pg_hba.conf
and add the following line:
host all all <your IP set>/<your netmask> md5
host all all 0.0.0.0/0 md5 is an access control rule that let anybody login in from any address if providing a valid password (md5 keyword).
Then edit postgresql.conf:
sudo vi /etc/postgresql/9.5/main/postgresql.conf
listen_addresses = '*'
Finally, the DB shall be restarted:
sudo /etc/init.d/postgresql restart
The default PostgreSQL settings aren’t great for very large databases like OSM databases. Proper tuning can just about double the performance.
The PostgreSQL wiki has a page on database tuning.
Paul Norman’s Blog has an interesting note on optimizing the database, which is used here below.
work_mem are probably enough on a development or testing machine.2: both parameters should be increased for faster data loading and faster queries while rendering respectively.
Conservative settings for a 2GB VM are
maintenance_work_mem=128MB. On a machine with enough memory you could set them as high as
Besides, important settings are
shared_buffers and the write-ahead-log (wal). There are also some other settings you might want to change specifically for the import.
To edit the PostgreSQL configuration file with vi editor:
sudo vi /etc/postgresql/9.5/main/postgresql.conf
and if you are running PostgreSQL 9.3:
sudo vi /etc/postgresql/9.3/main/postgresql.conf
shared_buffers = 128MB min_wal_size = 1GB max_wal_size = 2GB maintenance_work_mem = 256MB autovacuum = off fsync = off
The latter two ones allow a faster import: the first turns off auto-vacuum during the import and allows you to run a vacuum at the end; the second introduces data corruption in case of a power outage and is dangerous. If you have a power outage while importing the data, you will have to drop the data from the database and re-import, but it’s faster. Just remember to change these settings back after importing. fsync has no effect on query times once the data is loaded.
The PostgreSQL tuning adopted by OpenStreetMap can be found in the PostgreSQL Chef Cookbook: the specific PostgreSQL tuning for the OpenStreetMap tile servers is reported in the related Tileserver Chef configuration.
For a dev&test installation on a system with 16GB of RAM, the suggested settings are the following:
shared_buffers = 2GB work_mem = 128MB maintenance_work_mem = 1GB wal_level = minimal synchronous_commit = off min_wal_size = 1GB max_wal_size = 2GB checkpoint_timeout = 15min checkpoint_completion_target = 0.9 default_statistics_target = 1000 autovacuum = off fsync = off
To stop and start the database:
sudo /etc/init.d/postgresql stop sudo /etc/init.d/postgresql start
You may get an error and need to increase the shared memory size. Edit /etc/sysctl.d/30-postgresql-shm.conf and run
sudo sysctl -p /etc/sysctl.d/30-postgresql-shm.conf. A parameter like
kernel.shmall=4194304 could be appropriate for a 16GB segment size.3
To manage and maintain the configuration of the servers run by OpenStreetMap, the Chef configuration management tool is used.
The configuration adopted for PostgreSQL is postgresql/attributes/default.rb.
Osm2pgsql is an OpenStreetMap specific software used to load the OSM data into the PostGIS database.
To install osm2pgsql:
sudo apt-get install -y osm2pgsql
This alternative installation procedure generates the most updated executable by compiling the sources.
# Needed dependencies sudo apt-get install -y make cmake g++ libboost-dev libboost-system-dev \ libboost-filesystem-dev libexpat1-dev zlib1g-dev \ libbz2-dev libpq-dev libgeos-dev libgeos++-dev libproj-dev lua5.2 \ liblua5.2-dev # Download osm2pgsql cd /tmp git clone git://github.com/openstreetmap/osm2pgsql.git # Prepare for compiling cd osm2pgsql mkdir build && cd build cmake .. # Compile make # Install sudo make install # Clean-out temporary files cd ../.. rm -rf osm2pgsql
You need to download an appropriate .osm or .pbf file to be subsequently loaded into the previously created PostGIS instance via
There are many ways to download the OSM data.
The reference is Planet OSM.
It’s probably easiest to grab an PBF of OSM data from geofabrik.
Also, BBBike.org provides extracts of more than 200 cities and regions world-wide in different formats.
Map data of the whole planet (32G):
wget -c http://planet.openstreetmap.org/pbf/planet-latest.osm.pbf
Map data of Great Britain (847M):
wget -c http://download.geofabrik.de/europe/great-britain-latest.osm.pbf
Map data of Lombardy (279M):
wget -c http://osm-estratti.wmflabs.org/estratti/regioni/pbf/03---Lombardia.pbf
For just Liechtenstein:
wget http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf.md5 wget http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf md5sum -c liechtenstein-latest.osm.pbf.md5 # Check that the download wasn't corrupted
Another method to download data is directly with your browser. Check this page.
Alternatively, JOSM can be used (Select the area to download the OSM data: JOSM menu, File, Download From OSM; tab Slippy map; drag the map with the right mouse button, zoom with the mouse wheel or Ctrl + arrow keys; drag a box with the left mouse button to select an area to download. The Continuous Download plugin is also suggested. When the desired region is locally available, select File, Save As,
<filename>.osm. Give it a valid file name and check also the appropriate directory where this file is saved.
In all cases, avoid using too small areas.
OpenStreetMap is open data. OSM’s license is Open Database License.
osm2pgsql uses overcommit like many scientific and large data applications, which requires adjusting a kernel setting:
sudo sysctl -w vm.overcommit_memory=1
To load data from an .osm or .pbf file to PostGIS, issue the following:
cd /home/ubuntu/Documents/MapBox/project # if this directory is missing, start TileMill to create it cd openstreetmap-carto HOSTNAME=localhost # set it to the actual ip address or host name osm2pgsql -s -C 300 -c -G --hstore --style openstreetmap-carto.style --tag-transform-script openstreetmap-carto.lua -d gis -H $HOSTNAME -U postgres [.osm or .pbf file]
Notice that the suggested process adopts the
--slim option), which uses temporary tables, so running it takes more diskspace, while less RAM memory is used. You might add
--drop option with
--slim), to also drop temporary tables after import.
If everything is ok, you can go to Create indexes.
Notice that the following elements are used:
Depending on the input file size, the osm2pgsql command might take very long.
Note: if you get the following error:
` node_changed_mark failed: ERROR: prepared statement “node_changed_mark” does not exist
do the following command on your original.osm:
sed "s/action='modify' //" < original.osm | > fixedfile.osm
Then process fixedfile.osm.
If you get errors like this one:
Error reading style file line 79 (fields=4) flag 'phstore' is invalid in non-hstore mode Error occurred, cleaning up
or this one:
Postgis Plugin: ERROR: column "tags" does not exist LINE 8: ...ASE WHEN "natural" IN ('mud') THEN "natural" ELSE tags->'wet...
then you need to enable hstore extension to the db with
CREATE EXTENSION hstore; and also add the –hstore flag to osm2pgsql.
Enabling hstore extension and using it with osm2pgsql will fix those errors.
Add the indexes indicated by openstreetmap-carto:
HOSTNAME=localhost # set it to the actual ip address or host name cd /home/ubuntu/Documents/MapBox/project # if this directory is missing, start TileMill to create it cd openstreetmap-carto scripts/indexes.py | psql -U postgres -h $HOSTNAME -d gis
Read custom indexes for further information.
cd ~/src/tilemill ./index.js --server=true --listenHost=0.0.0.0
--server=true is only needed in case of server based startup, to avoid the local user interface. Besides,
--listenHost=0.0.0.0 is needed to access TileMill installed on a remote server (not necessary when doing http://localhost:20009).
An error message like role … does not exist… might be possibly related to missing environment variables PGUSER, PGPASSWORD, etc. Set these variables and restart TileMill.
The application needs a few seconds to start, so be patient.
Select project Openstreetmap Carto
Give TileMill the time to render the map: accessing the database and rendering images is often a slow process (mainly depending on the amount of data to be managed, but also on the server performance and on the network), so give many seconds to TileMill to output or refresh the map.
Zoom out to the entire world shape (zoom level 1 or to some close zooms like zoom 4), then progressively zoom into the region where you downloaded the map data. You might use the double click and wait for the next zoom level to appear.
You shouldn’t use the text editor built-in to TileMill. Instead, hide the right pane and use an external text editor.
TileMill automatically refreshes the rendering upon any file change, including all .mss and project.mml.
With your browser, access the map through http://localhost:20009
To directly access the project: http://127.0.0.1:20009/#/project/openstreetmap-carto
Notice that Https will not work (use http instead).