Finding Differences in Roads

Finding differences between OpenStreetMap and county road data is hard if done manually. Fortunately, Open Source Software tools eases the process. This tutorial will provide you with one means of finding differences. The following software is needed:

QGIS
PostGIS
PGAdmin3
pgShapeLoader
org2osm.py

QGIS, PostGIS, PGAdmin3 and pgShapeLoader installation is beyond the scope of this tutorial. Boundless Open Geo Suite is the perfect solution to install PostGIS and related tools.

I’m comparing Skagit County road center line shapefile data with OSM road ways. Fortunately, Skagit County’s road names are spelled out instead of abbreviated as most road data is presented. Skagit County’s GIS data is open and freely available. Since most road data comes abbreviated, you’ll need to expand the road names to compare with OSM data. Postgresql functions to expand road names can be found in my github repository. Most likely you will need to modify the function for your data.

Download OSM Data

Overpass-Turbo has an easy method to obtain current data. Modify and run the following script:

<osm-script output="json" timeout="900" 
element-limit="1073741824">
<id-query              into="area"/>
<!-- gather results -->
<union>
 	<query type="node">
  		<has-kv k="highway"/>
  		<area-query from="area"/>
    </query>
    <query type="way">
        <has-kv k="highway" />
        <area-query from="area"/>
    </query>
	<query type="relation">
  		<has-kv k="highway"/>
  		<area-query from="area"/>
	</query>      
</union>
<!-- print results -->
<print mode="body"/>
<recurse type="down"/>
<print mode="skeleton" order="quadtile"/>
</osm-script>

Remember, change the nominatim search area from “Skagit County” to your county.

Once the query is complete, select Export from the menu. Select GeoJSON for the data type. It will take a few minutes before the download starts. Once complete move the file export.geojson to the data directory of your counties data.

QGIS

Open the geojson vector file vector icon in QGIS. Select the LineString feature.

Select the OpenStreetMap in OpenLayers plugins menu item. Once open, drag the export to the top position, otherwise the OSM layer will obscure the geojson data.

QGIS Layers

The last step with QGIS is to select the county roads using the Select Features by Rectangle tool. Rectangle Selection and save the layer as a vector file, in the Vector menu.

Save Vector Layer

Vector File

Import into PostGIS

Use pgShapeFile to import the saved shapefiles into PostGIS. Set connections settings. Set the Username, password, Server Host and Database.

Connection Settings

Add the shapefile and import into PostGIS. Make sure to set the srid, in this case 4326 for WGS84. pgShapefile

Comparing OSM data to County Data

Time for some good old sql right joins to find missing roads.

SELECT s.gid, s.geom, s.fullname AS "Name"
FROM skagit_osm o RIGHT JOIN skagit_roads s ON o.name = s.fullname
WHERE o.gid IS NULL

Missing

As viewed in QGIS with the OpenStreetMap background from OpenLayers.The portion being shown is in Sedro-Woolley. The county has the streets spelled out, for example Fourth Street while OSM has 4th Street. (The street signs have it as 4th ST.)

Viewed in QGIS

Convert to .osm

Finally to fix problems, either by correcting the road name or add in missing roads into OSM, the missing roads need to be converted into a format that JOSM can understand. The steps involved are:

convert the layer to a shapefile
convert the shapefile into a .osm file readable by JOSM

Create shapefile

Their are two methods of creating the shapefile. From QGIS, select the missing roads data and save it as a shapefile. It is the same process we used above to convert the geojson file to a shapefile.

The second is to use a utility program to convert the data directly from PostGIS to a shapefile. The command line is:

pgsql2shp -f missing mygis "SELECT s.gid, s.geom, s.fullname AS "Name" FROM skagit_osm o RIGHT JOIN skagit_roads s ON o.name = s.fullname WHERE o.gid IS NULL"

Create .OSM

To create an .osm file, ogr2osm.py is needed along with a translations file. Since I’m not importing the data, only a simple translations file is needed. In this case I opted to skip the translation file completely. To run ogr2osm.py use:

python ogr2osm.py [-t translation.py] shapefile.shp

Load Missing Layer in JOSM

Use JOSM to compare the county data with OSM data. Even turn on the TIGER 2013 overlay. Some missing ways will be easy, for instance, new roads or OSM roads with no name. Others will be difficult. TIGER, OSM and county data can all be different. In those cases, plan for an outing or at least leave a note asking for an in person visit.

JOSM with Missing Layer

Future Goals

The next logical step would be to build tiles from the counties data to use as background images in JOSM. Similar to the TIGER 2012/13 background images. Look for a future article on how to create the tiles and use GeoServer as a WMS server.

Clifford Snow