Saving and cropping data

Saving and cropping data#

Pyrosm can write .osm.pbf files, not only read them. This lets you crop a large extract down to a smaller area to share or re-read faster (OSM.to_pbf), write modified OSM data back to a valid PBF after editing tags or attributes in pandas, or export only selected layers (e.g. just the buildings or the road network) to a new PBF (OSM.write_pbf).

How to?

Crop a PBF to a bounding box#

to_pbf() crops the source file by the OSM object’s bounding_box and writes a valid, re-readable PBF. The crop is complete-ways (a way is kept whole when any of its nodes is inside the box), it streams the file blob-by-blob (so it stays low on memory), and coordinates round-trip exactly.

import os
from pyrosm import OSM, get_data

fp = get_data("helsinki")
orig_size_mb = os.path.getsize(fp) / (1024 * 1024)

# Crop to a bounding box [minx, miny, maxx, maxy] in lon/lat
bbox = [24.93, 60.16, 24.96, 60.18]
cropped = OSM(fp, bounding_box=bbox).to_pbf()

size_mb = os.path.getsize(cropped) / (1024 * 1024)
print(f"Cropped size: {os.path.basename(cropped)} {size_mb:.1f} MiB.")
print(f"Original size: {os.path.basename(fp)} {orig_size_mb:.1f} MiB.")

Cropped size: pyrosm_crop_ekhj6h5z.osm.pbf 6.9 MiB.
Original size: Helsinki.osm.pbf 60.0 MiB.

# The cropped file reads back like any other PBF
OSM(cropped).get_buildings().shape

(1486, 36)

Smaller output: compact and repack#

By default the crop keeps each source block’s string table, which is the fastest option. Two flags trade a little speed for a smaller file:

  • compact=True prunes each block’s string table to only the strings its kept elements reference.

  • repack=True re-packs the kept elements into densely filled blocks (as osmium/Osmosis produce) for the smallest output; it supersedes compact.

The written OSM data is identical for every option.

default = OSM(fp, bounding_box=bbox).to_pbf()
compact = OSM(fp, bounding_box=bbox).to_pbf(compact=True)
repack = OSM(fp, bounding_box=bbox).to_pbf(repack=True)

for label, path in [("default", default), ("compact", compact), ("repack", repack)]:
    size_mb = os.path.getsize(path) / (1024 * 1024)
    print(f"{label:8s} {size_mb:.1f} MiB")

default  6.9 MiB
compact  2.6 MiB
repack   2.1 MiB

As we can see, the compact and repack options drop the size of the output file significantly, but in return, the cropping process is slower. The default behavior is the fastest and recommended approach if the larger output file size is not an issue.

Write modified OSM data back to a PBF#

write_pbf() writes the data you read back to a valid PBF after you edit it in pandas — for example to fill a missing maxspeed or add a computed travel_time tag. Each row updates the matching element (by osm_type + id); rows whose id is not in the source are added as new elements synthesised from their geometry.

import tempfile

osm = OSM(get_data("helsinki"))
edges = osm.get_network("driving")

# Edit/add tags in pandas — here we tag every edge with a travel_time (static 15 kmph)
speed_kmph = 15
edges["travel_time"] = 3.6 * edges["length"] / speed_kmph

out_path = os.path.join(tempfile.gettempdir(), "modified.osm.pbf")
osm.write_pbf(edges, out_path)

size_mb = os.path.getsize(out_path) / (1024 * 1024)
print("wrote", os.path.basename(out_path), size_mb, "MiB.")

wrote modified.osm.pbf 54.837538719177246 MiB.

# Read it back, requesting the new tag as a column
reread = OSM(out_path).get_network("driving", extra_attributes=["travel_time"])
reread["travel_time"].astype(float).mean()

np.float64(16.667588739971247)

Export only selected layers#

By default write_pbf() writes the whole dataset it read, so untouched layers survive. Pass subset_only=True to instead write a PBF containing only the features you give it – for example parse the buildings and export just those, or just the road network. The references each feature needs to stay valid are pulled in automatically (a way’s nodes, a relation’s member ways and nodes), so the output is a valid, re-readable PBF.

You can also combine several layers by passing a list ([buildings, network]); the union of their elements is written.

osm = OSM(get_data("helsinki"))
buildings = osm.get_buildings()
network = osm.get_network()

# A buildings-only subset is much smaller than the whole dataset
whole = os.path.join(tempfile.gettempdir(), "whole.osm.pbf")
osm.write_pbf(buildings, whole)  # default: the whole dataset is written
b_only = os.path.join(tempfile.gettempdir(), "buildings_only.osm.pbf")
osm.write_pbf(buildings, b_only, subset_only=True)  # only the buildings (+ refs)
for label, p in [("whole dataset", whole), ("buildings only", b_only)]:
    print(f"{label:15s} {os.path.getsize(p) / 1024 / 1024:.1f} MiB")

# Combine several layers by passing a list -> the union is written
combined = os.path.join(tempfile.gettempdir(), "buildings_and_network.osm.pbf")
osm.write_pbf([buildings, network], combined, subset_only=True)
both = OSM(combined)
print("combined:", len(both.get_buildings()), "buildings,",
      len(both.get_network()), "network edges")

whole dataset   54.4 MiB
buildings only  12.8 MiB
combined: 176900 buildings, 255130 network edges