If you need to create custom dictionaries or wordlists, there are many options, in this case we were looking for UK specific street / road names.

You do not need to ‘scrape’ open street map for data, it is not ideal from their point of view or yours. As the data is open, they do actively provide downloadable exports (i.e an ‘.osm.pbf’ file) for different data sets.

Looking at a page of export data such as (http://download.geofabrik.de/europe.html) will enable you to find and download the correct file for your needs, and it can then be processed offline.

On Kali (or similar) you should have the osmosis package available which can be used to process Open Street Map data exports, filtering out only highways. Then with a little grep, these can be used to create steet / road name lists.

The same idea can be used for counties, towns, citys etc.

#Install tools
apt-get install osmosis

# Download a '.osm.pbf' file for your area: http://download.geofabrik.de/europe.html
wget http://download.geofabrik.de/europe/great-britain-latest.osm.pbf

# use a program like Osmosis to filter out only highways
osmosis --read-pbf file.osm.pbf --tf accept-ways highway=\* --write-xml myfile.osm

# From the resulting XML file, extract all names
grep 'k="name"' myfile.osm | cut -d\" -f4

You will likely find some slightly poor data within the file (at the time of writing), this is essentially down to user supplied data to the open street map project.

This however does not worry me overly for this usage case. You will find that some extracted data files are quite large, so remember to clean up when your done.