pyrosm.OSM

Contents

pyrosm.OSM#

class pyrosm.OSM(filepath, bounding_box=None, keep_metadata=True, complete_relations=False, keep_node_info=False, engine='out_of_core', workers=None, progress=None)#

OpenStreetMap PBF reader object.

Parameters:

  • filepath (str) – Filepath to input OSM dataset (*.osm.pbf)

  • bounding_box (list | tuple | numpy.ndarray | shapely geometry | GeoDataFrame | GeoSeries) – Filtering OSM data spatially is allowed by passing a bounding box in lon/lat: four numbers [minx, miny, maxx, maxy] (a list, tuple or array, such as a GeoDataFrame’s total_bounds), a Shapely Polygon/MultiPolygon or closed LineString/LinearRing, or a GeoDataFrame/GeoSeries (its geometries together, reprojected to lon/lat). Every function that takes a bounding box takes these.

  • keep_metadata (bool (default: True)) – Whether to keep the OSM element metadata columns (timestamp, version, changeset) in the returned GeoDataFrames. Set to False to drop them and reduce memory use when the metadata is not needed; the per-node metadata is then also skipped while parsing, which lowers peak memory on node-heavy files. History (.osh.pbf) parsing keeps the metadata it requires regardless of this flag.

  • complete_relations (bool (default: False)) – When reading with a bounding_box, a relation (e.g. a multipolygon or boundary) is normally assembled from only the member ways that fall inside the box, so a relation straddling the edge of the box comes out with a partial geometry. Set this to True to fetch each such relation’s full member set (member ways and their nodes, even outside the box) so the geometry is complete. This adds two extra streaming passes over the file (only when a relation actually has missing members), so it is opt-in. It has no effect on a whole-file read (no bounding_box), which already holds every member. Only member ways are completed; relations whose members are themselves relations (super-relations) are not.

  • engine (str (default: 'out_of_core')) –

    Which reader backend to use. ‘out_of_core’ (the default) decodes the file in a single streaming pass with bounded peak memory, spilling intermediate data to disk, and – when the optional pyarrow is installed – caches each read’s result in the temporary folder, so an identical read returns it without decoding the file again (OSM.clear_cache() removes the cached results). ‘in_memory’ parses the whole file into memory once and serves every read of the OSM object from it.

    The out-of-core backend chooses by the file’s size how many threads decode it (workers=”auto”, the default; see below). A parallel read runs on threads and needs no if __name__ == “__main__”: guard. With the environment variable PYROSM_DECODE_BACKEND=processes (deprecated: it warns, and it will be removed in a future release) it runs on one core by default and in worker processes for workers=”auto” or workers=N; those start with spawn on macOS and Windows, which re-imports the program’s entry point, so a parallel read then must run under the guard:

    if __name__ == "__main__":
    osm = OSM(fp, engine="out_of_core", workers="auto")
    buildings = osm.get_buildings()

    Without the guard such a read still completes – it falls back to a single process and warns – but it is not parallel.

    History reads – an .osh.pbf file, or any feature call with a timestamp – are served by the in-memory reader with either engine, which chooses the latest version of each element at/before the timestamp from all its versions. write_pbf also reads the file in memory with either engine.

  • keep_node_info (bool (default: False)) – Whether to keep the nodes column – the ordered member node ids of each way – in the returned GeoDataFrames. It is dropped by default because a list-valued column cannot be written to most file formats. Set to True to reshape ways with write_pbf(…, apply_geometry=True), which takes a way’s topology from this column.

  • workers (int | str (default: None)) – Number of threads the ‘out_of_core’ engine uses to decode the file. By default (None, as “auto”) pyrosm chooses the count: a single core for files below about 20 MB; for larger files the CPU cores this process may use less two, at most 4 below about 70 MB and at most one per data blob. Pass workers=N for an explicit count, e.g. workers=1 for one core (a count above the cores this process may use is reduced to that count, with a warning). With PYROSM_DECODE_BACKEND=processes they are worker processes, one by default and at most eight with “auto”, and a parallel read needs the if __name__ == “__main__”: guard on macOS/Windows. Has no effect on the ‘in_memory’ engine.

  • progress (bool | callable | None (default: None)) – None follows pyrosm.enable_logging(progress=…), which is True unless set. True shows a bar on stderr while the file is read, once the read has run 2 seconds, from 0 to 100 % over the whole read with its current step in the label (e.g. “Reading Paris.osm.pbf: building the geometries”), and clears it when the read finishes; in Jupyter it is a widget when ipywidgets is installed (pip install “pyrosm[notebook]”). Without a terminal, a read that took 2 seconds or more prints one line with the time it took instead. False shows nothing. A callable is called as progress(done, 100) with the percentage of the read done, from 0 to 100 and never back.

Contents