Author Topic: OpenStreetMap  (Read 1299 times)

Offline OS923

  • Platinum Member (500+ Posts)
  • *****
  • Posts: 718
  • Liked:
  • Likes Given: 3
OpenStreetMap
« on: May 10, 2021, 09:34:54 AM »
I looked into the source code of osmconvert. Then I wrote my own programs:
  • Count OSM makes some statistics.
  • Filter OSM removes unnecessary attributes.
  • Renumber OSM renumbers the IDs sequentially starting from zero.

After the IDs are renumbered some data can be stored in arrays and the IDs can be used as indexes. This saves memory and it's faster than hash tables or trees. Calculating bounding boxes will be fast for example.

Now the data have to be turned into a quad tree. Then it has to be saved into one file so that everything that you want to draw can be retreived as fast as possible. I have never done this before. I have no idea where to start. Any advice?

Then I have to draw it. I'll use QuickDraw GX for the text and QuickDraw for the rest. I draw it at 200% and then scale it 50% using QuickTime. Then I'll have some antialiasing. If I define this in a plugin then you can write your own plugin.

And it has to be able to export PNG tiles.

Offline IIO

  • Platinum Member (500+ Posts)
  • *****
  • Posts: 3336
  • just a number
  • Liked:
  • Likes Given: 118
Re: OpenStreetMap
« Reply #1 on: May 11, 2021, 12:30:55 AM »
OSM is a very good idea.
insert arbitrary signature here

Offline OS923

  • Platinum Member (500+ Posts)
  • *****
  • Posts: 718
  • Liked:
  • Likes Given: 3
Re: OpenStreetMap
« Reply #2 on: May 12, 2021, 08:13:46 AM »
Im close to bringing all OSM maps to OS 9. The main problem is the limited memory. The maps are now twice as big compared to 4 years ago. I have to anticipate that the size will increase further so all conversions have to be file-based. This means that for large maps a conversion will require multiple passes.

Offline IIO

  • Platinum Member (500+ Posts)
  • *****
  • Posts: 3336
  • just a number
  • Liked:
  • Likes Given: 118
Re: OpenStreetMap
« Reply #3 on: May 15, 2021, 03:54:17 AM »
what memory size are we talking about?
insert arbitrary signature here

Offline OS923

  • Platinum Member (500+ Posts)
  • *****
  • Posts: 718
  • Liked:
  • Likes Given: 3
Re: OpenStreetMap
« Reply #4 on: May 17, 2021, 11:13:06 AM »
400 MB was enough to renumber the map of Belgium.

Count OSM needed 16 minutes to process Belgium.osm (FW). Filter OSM needed 39 minutes to remove the unnecessary attributes from Belgium.osm (FW). Renumber OSM needed 34 minutes to renumber the IDs in Belgium.osm (ATA).

Offline OS923

  • Platinum Member (500+ Posts)
  • *****
  • Posts: 718
  • Liked:
  • Likes Given: 3
Re: OpenStreetMap
« Reply #5 on: May 28, 2021, 12:11:49 PM »
Now I convert it first to a binary format. This is 4 to 5 times smaller. This is processed about 7 times faster.

Offline IIO

  • Platinum Member (500+ Posts)
  • *****
  • Posts: 3336
  • just a number
  • Liked:
  • Likes Given: 118
Re: OpenStreetMap
« Reply #6 on: May 28, 2021, 07:04:51 PM »
why is your macos so huge in RAM?
insert arbitrary signature here

Offline OS923

  • Platinum Member (500+ Posts)
  • *****
  • Posts: 718
  • Liked:
  • Likes Given: 3
Re: OpenStreetMap
« Reply #7 on: May 31, 2021, 08:27:54 AM »
The IDs are 64-bit numbers. If I renumber them starting from 0 then I can use 32-bit numbers. This is faster and saves space. Then I can use arrays for the coordinates and the bounding boxes. This is faster and saves space.

The renumbering requires that it builds a tree with unused ID intervals. I have around 1 interval for every 3.5 objects. That uses all this memory.