Frequently asked questions

Frequently asked questions#

Answers to questions that come up often (click a question to expand it). If yours isn’t here, search the issue tracker or open a new issue.

Reading data#

How do I read a large scale extract without running out of memory?

Pyrosm reads with its out-of-core engine by default, which decodes the file in a single streaming pass with bounded peak memory, spilling intermediate data to disk instead of holding the whole file at once, so large files covering e.g. whole countries can be read as they are:

osm = OSM("country.osm.pbf")
buildings = osm.get_buildings()

Reads, crops and merges choose how many threads to use by the size of the data (workers="auto", the default): one for less than about 20 MB, more for larger files. Pass workers=N for a fixed count, for example workers=1 to stay on one core when several pyrosm jobs already run side by side. The threads need no if __name__ == "__main__": guard. Setting the environment variable PYROSM_DECODE_BACKEND=processes switches to worker processes, as earlier versions used; with it the default is one process, and on macOS and Windows a parallel read from a script needs the guard, and without it falls back to a single process with a warning. The processes setting is deprecated: it warns, and it will be removed in a future release. If you need it, please tell us why in an issue.

Other ways to lower memory use: read only the area you need with a bounding_box (or crop the file first with to_pbf()), keep fewer tag columns with tags_to_keep=[...] on the feature call, and drop the element metadata columns with keep_metadata=False.

How do I get OSM data for a specific place or bounding box?

Download an extract by place name (geocoded) or by bounding box, then read it — and/or read only a sub-area of a file with bounding_box:

from pyrosm import OSM, get_data, get_data_by_area, get_data_by_bbox

fp = get_data("Helsinki")                              # an extract by place name
# fp = get_data_by_bbox(bbox=[minx, miny, maxx, maxy]) # an extract for a bounding box
# fp = get_data_by_area(area)                          # the smallest extract containing an area

osm = OSM(fp, bounding_box=[minx, miny, maxx, maxy])   # read only this area of the file

For an area on a national border, get_data_by_area() finds the smallest single extract that contains it, or with strategy="smallest_total" a set of extracts merged into one file when their total download is smaller, and merge_pbf() combines overlapping extracts yourself. See Downloading data and Saving, cropping and merging data for more.

How does bounding_box keep features that are at the edge of the box (“intersects” or “within”)?

It is intersects, and the features are kept whole, not clipped. A way, building or other area is included as long as at least one of its nodes is inside the box, and it is returned with its complete geometry — so a building that is half inside and half outside the box comes back as the whole building, with vertices beyond the box edge. (Internally pyrosm selects “complete ways”, like osmconvert --complete-ways.)

A relation (a multipolygon or boundary) that straddles the box is the exception: by default it is assembled from only the member ways that fall inside the box, so it has a partial geometry. Pass complete_relations=True to fetch each such relation’s full member set so its geometry is complete:

from pyrosm import OSM

osm = OSM("city.osm.pbf", bounding_box=bbox, complete_relations=True)
buildings = osm.get_buildings()
What coordinate reference system are the results in, and how do I reproject?

pyrosm returns GeoDataFrames in WGS84 (EPSG:4326) — longitude/latitude in degrees, matching the OSM source. Reproject with GeoPandas, e.g. to a metric CRS before measuring distances or areas:

buildings = osm.get_buildings().to_crs(epsg=3067)   # a metric CRS (here ETRS-TM35FIN)

A CRSError when importing or first using pyrosm almost always means a broken pyproj/PROJ install in your environment, not a pyrosm problem — reinstall pyproj (conda-forge recommended).

Filtering#

How does custom_filter combine multiple keys and values — AND or OR?

custom_filter is OR: an element is kept if it matches any of the given criteria. Several values for one key match any of them, and several keys are combined with OR as well. True means “any value for this key”.

# kept if it is a building OR carries any amenity tag:
osm.get_data_by_custom_criteria(custom_filter={"building": True, "amenity": True})

# kept if highway is residential OR primary:
osm.get_data_by_custom_criteria(custom_filter={"highway": ["residential", "primary"]})

Pass filter_type="exclude" to drop the matching elements instead of keeping them. See Custom filters.

get_boundaries() returns only administrative areas — how do I get other boundary types?

get_boundaries() returns only boundary=administrative areas by default. For other types, pass boundary_type:

osm.get_boundaries(boundary_type="all")          # every boundary type
osm.get_boundaries(boundary_type="postal_code")  # e.g. postal-code areas

Two things related to filtering that are good to understand:

  • name= filters by the area’s name, not by its boundary type — e.g. get_boundaries(name="Helsinki") searches names, it does not select a type.

  • A neighbourhood/suburb mapped as an area is usually tagged boundary=place (e.g. place=suburb), so it is not an administrative boundary. Use boundary_type="all", or query the place tag directly:

    osm.get_data_by_custom_criteria(custom_filter={"place": ["suburb", "neighbourhood"]})
    

Networks and graphs#

Are the network types the same as in OSMnx?

Mostly. The network_type filters follow the way filters of OSMnx, so pyrosm selects nearly the same streets as the equivalent OSMnx query. The OSMnx names work as network types as well:

pyrosm

OSMnx

driving

drive

driving+service

drive_service

walking

walk

cycling

bike

all

all

all_public

all_public

driving leaves out the service roads (alleys, driveways, parking aisles). Use driving+service when those matter for reachability, for example when routing to individual buildings. The driving network also leaves out busways, ladders and trails, which OSMnx keeps in drive.

Unlike OSMnx, pyrosm follows OSM’s access hierarchy. It uses the most specific access tag on a way to decide whether to leave that way out. For example, access=no leaves a way out of the walking network unless a more specific tag such as foot=yes allows walking.

OSMnx leaves every cycle way out of walk and every foot way out of bike. In pyrosm, a cycle way with a foot tag, such as foot=designated, can stay in the walking network unless the tag is no, private or use_sidepath (see below for use_sidepath). A cycle way without a foot tag follows the default of the country it lies in. It stays in countries such as Finland, where pedestrians may use cycle ways by default, and is left out in other countries and where pyrosm finds no country. The cycling network treats a foot way and its bicycle tag in the same way. With shared_paths="tagged", get_network and get_graph leave out every cycle way without a foot tag and every foot way without a bicycle tag, as OSM’s worldwide defaults do. With shared_paths="all", they keep all of them, also where the country’s rules do not allow it, for a network of the ways people walk and cycle on in practice. A tag such as foot=no or bicycle=no leaves the way out with every setting. foot=use_sidepath says that pedestrians must use a sidewalk mapped beside the way. It leaves the way out of the walking network with "tagged" and "local", and shared_paths="all" keeps it.

The cycling network keeps the ways tagged bicycle=use_sidepath. The tag sends cyclists to a cycle track mapped beside the road. Driveways, yards and side streets on the side of the road without the track often join only the road: cyclists cross the road to reach the track, and OSM maps no way for that crossing. Without the road, these places would be cut off from the network. To make routes follow the cycle track where there is one, give these roads a higher cost and route with it, for example with weight="cost" in NetworkX:

nodes, edges = osm.get_network("cycling", nodes=True)
sidepath = edges["bicycle"].fillna("").str.contains("use_sidepath")
edges["cost"] = edges["length"].where(~sidepath, edges["length"] * 3)
G = osm.to_graph(
    nodes, edges, graph_type="networkx", simplify_kwargs={"length_cols": ["length", "cost"]}
)

The cost does not mean that riding on the road is slower. It expresses how much cyclists prefer the cycle track to riding in the traffic beside it. The factor 3 is at the low end of the costs that the router Valhalla gives these roads with its default settings. Valhalla prices how stressful a road is to ride on: compared with a cycle track, it makes riding along a road tagged bicycle=use_sidepath about 2.5 times as costly on a residential street at 30 km/h, and about 6 to 7 times on a primary road at 60 km/h with two lanes in each direction. To weigh the roads as Valhalla does, choose the factor by the highway or maxspeed column, higher for primary and secondary roads than for residential streets. A much higher factor treats the road almost as closed, so that routes use it only to reach the places on it.

Choose the weight by what you want to find: route with cost to find routes that favour cycle tracks over the roads beside them, and with length to find the shortest routes. Either way, sum length along the route for its distance, not cost, which overstates it, and compute the travel time from that distance at a cycling speed you choose.

Listing cost in length_cols sums it along each simplified edge, as length is summed. str.contains also finds use_sidepath among several values, such as use_sidepath;destination. To leave these ways out instead, keep edges[~sidepath]. Neither looks at bicycle:forward=use_sidepath or bicycle:backward=use_sidepath, which apply in one direction only and are in the tags column.

A network exported with to_graph(..., graph_type="networkx") is OSMnx-compatible as well: the node and edge attributes are named as OSMnx expects, so its functions work on the graph directly.

How do I find street intersections and their coordinates?

Build a graph with to_graph(..., simplify_kwargs={"edge_attrs_differ": None}). This simplification reduces the network to its topological nodes — every node is a real intersection or a dead-end (the default one also keeps the nodes where the speed limit or road class changes) — and each node carries x/y (longitude/latitude in EPSG:4326), a Point geometry, and a street_count (how many streets meet there). Keep the nodes where three or more streets meet:

import geopandas as gpd
from pyrosm import OSM, get_data

osm = OSM(get_data("Helsinki"))
nodes, edges = osm.get_network(network_type="driving", nodes=True)
G = osm.to_graph(
    nodes, edges, graph_type="networkx", simplify_kwargs={"edge_attrs_differ": None}
)

intersections = gpd.GeoDataFrame(
    [
        {"osmid": n, "lon": d["x"], "lat": d["y"],
         "street_count": d["street_count"], "geometry": d["geometry"]}
        for n, d in G.nodes(data=True)
        if d["street_count"] >= 3
    ],
    crs="EPSG:4326",
)

street_count == 1 marks dead-ends. Use network_type="walking" or "cycling" to count crossings for those modes instead.

Installation#

How should I install pyrosm, and what if installation fails?

Install it with pip, which installs prebuilt wheels, so nothing is compiled:

pip install pyrosm

or from conda-forge (with mamba/micromamba or conda):

mamba install -c conda-forge pyrosm

Wheels are published for CPython 3.11–3.15 on glibc-based Linux (x86_64 and aarch64), macOS (Intel and Apple silicon) and Windows on x86_64. Elsewhere pip builds pyrosm from source, which needs a C compiler; see Installation for musl-based Linux (Alpine) and Windows on ARM, where GeoPandas’ dependency pyogrio needs building too.