[lug] Speed up osm2pgsql

Lee Woodworth blug-mail at duboulder.com
Thu Feb 8 12:08:35 MST 2018


Populating a postgresql database for open street map uses 230GB
of virtual address space. The swap device is very active with
lots of writes at an average rate of 20MB/s.

The source parsing phase was going to take 2+ weeks using a
regular disk for swap. Changing to an ssd speeds things but
writes 1.4TB (actual device writes) out of a 75TB TBW. The
wear level reported by smartctl went from 0 to 10 just for
for this phase.

The post-processing phase is going very slow during a database
sort where the db is on a raid1. Does anybody know of knobs to
tweak to speed up larger-than-memory table sorts (100GB)? Or do we
just need to use an ssd and bite the bullet on having to replace
them frequently?


More information about the LUG mailing list