pyrosm.merge_pbf

Contents

pyrosm.merge_pbf#

pyrosm.merge_pbf(inputs, output_path=None, bounding_box=None, keep_relations=True, workers=None, polygon=None, progress=None)#

Merge overlapping *.osm.pbf extracts into one PBF, optionally cropped to a bounding box or a polygon.

The inputs may come from different providers, have snapshots days apart and carry element metadata or not. An element found in several inputs is written once: the copy with the highest version wins, then the one with the latest timestamp, then the one from the earliest input. The output is sorted by type then id (header feature Sort.Type_then_ID) and written in densely packed blocks. The inputs are streamed block by block; only id sets are held in memory.

Without bounding_box or polygon the output holds everything the inputs hold. With bounding_box, the crop rule of OSM.to_pbf() is applied to the union of the inputs: a node is kept when a copy of it lies inside the box, a way when its winning copy has a node inside the box, and a relation when its winning copy references a kept node or way. A kept way keeps its full node list, whichever input holds those nodes. With polygon the same rule applies with the polygon in place of the box. A single input is cropped the same way.

Parameters:
  • inputs (list of str or path-like) – The PBF files to merge, each sorted by type then id (osmium sort sorts a file that is not). History files and files storing node locations on ways are not supported.

  • output_path (str or path-like, optional) – Where to write the merged PBF. When None (default) a temporary file is created in the system temp directory and its path returned.

  • bounding_box (list | tuple | numpy.ndarray | shapely geometry | GeoDataFrame | GeoSeries, optional) – The box in lon/lat, in any form pyrosm.OSM takes: four numbers [minx, miny, maxx, maxy], or a geometry or GeoDataFrame, which is cropped by its envelope. The output’s header bounding box is this box (the polygon’s envelope with polygon), or else the union of the inputs’ header boxes. A box that OSM refuses is refused, and a merge that keeps no node warns.

  • keep_relations (bool) – When True (default) relations are written (with bounding_box or polygon, those referencing a kept node or way); when False none are written.

  • workers (int | str) – Number of threads for the merge: they select the elements inside bounding_box or polygon, read and decode the inputs’ blocks (cropped first to the kept elements with a region) and encode the merged blocks, which are written in order. By default (None, as "auto") the count is chosen as an out-of-core read does for the inputs’ total size; 1 runs sequentially. With PYROSM_DECODE_BACKEND=processes they are worker processes, one by default, which only select the elements; when those cannot run (on macOS and Windows: a script without an if __name__ == "__main__": guard, or one read from stdin), the merge runs in one process and warns.

  • polygon (shapely Polygon or MultiPolygon | GeoDataFrame | GeoSeries, optional) – The area to crop to, in lon/lat, in place of bounding_box: a node is kept when it lies inside the polygon or on its boundary. The output’s header bounding box is the polygon’s envelope.

  • progress (bool | callable | None (default: None)) – None follows pyrosm.enable_logging(), True shows a bar on stderr once the merge has run 2 seconds, from 0 to 100 % over the whole merge with its current pass in the label, False shows nothing, and a callable is called as progress(done, 100) with the percentage of the merge done, from 0 to 100 and never back.

Returns:

The path of the written PBF file.

Return type:

str or path-like

Raises:

ValueError – When an input cannot be read as a PBF, is not supported, or is not sorted by type then id, or when an input’s size or modification time changes (or the file is replaced) while the merge runs; the message names the file. When both bounding_box and polygon are given, or polygon is not a non-empty Polygon or MultiPolygon.

Examples

>>> import pyrosm
>>> out = pyrosm.merge_pbf(
...     ["switzerland-latest.osm.pbf", "france-latest.osm.pbf"],
...     "basel.osm.pbf",
...     bounding_box=[7.52, 47.51, 7.66, 47.60],
... )