{"payload":{"allShortcutsEnabled":false,"path":"applications/utils/planetdiff","repo":{"id":511498073,"defaultBranch":"main","name":"svn-archive","ownerLogin":"openstreetmap","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2022-07-07T11:20:59.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/261431?v=4","public":true,"private":false,"isOrgOwned":true},"currentUser":null,"refInfo":{"name":"main","listCacheKey":"v0:1657837147.4115899","canEdit":false,"refType":"branch","currentOid":"dfa2acd1f6f48cdc4c12f366c0833e35a7e27bdc"},"tree":{"items":[{"name":"Makefile","path":"applications/utils/planetdiff/Makefile","contentType":"file"},{"name":"UTF8sanitizer.c","path":"applications/utils/planetdiff/UTF8sanitizer.c","contentType":"file"},{"name":"example-diff.xml","path":"applications/utils/planetdiff/example-diff.xml","contentType":"file"},{"name":"input.c","path":"applications/utils/planetdiff/input.c","contentType":"file"},{"name":"input.h","path":"applications/utils/planetdiff/input.h","contentType":"file"},{"name":"keyvals.c","path":"applications/utils/planetdiff/keyvals.c","contentType":"file"},{"name":"keyvals.h","path":"applications/utils/planetdiff/keyvals.h","contentType":"file"},{"name":"planetdiff.c","path":"applications/utils/planetdiff/planetdiff.c","contentType":"file"},{"name":"readme.txt","path":"applications/utils/planetdiff/readme.txt","contentType":"file"},{"name":"sanitizer.h","path":"applications/utils/planetdiff/sanitizer.h","contentType":"file"}],"templateDirectorySuggestionUrl":null,"readme":{"displayName":"readme.txt","richText":"
planetdiff\n==========\nGenerates a file containing the differences between two planet.osm dumps.\n\nThe program supports .gz and .bz2 compressed files transparently.\nIt also runs an internal version of UTF8sanitizer on the input data so\nthat it can be used on a file downloaded from \nhttp://planet.openstreeetmap.org without needing any other manipulation.\n\nNote that the algorithm used relies on the strict ordering of data in the \nplanet.osm file to operate correctly. Data produced by other OSM tools\nnormally do not follow these rules and can not be manipulated using this\nprogram. \n\nBuild requirements\n------------------\nThe code relies on the libraries below:\n\n libxml2\n bzip2\n zlib\n\nTo compile this code on Fedora you need at least the following packages \ninstalled:\n\n libxml2-devel\n bzip2-devel\n zlib-devel\n\n\nCompiling\n---------\nOn a Linux or other Unix-like system:\n\n $ make\n\nThis will produce both planetdiff and planetpatch (as below).\n\nData ordering rules\n-------------------\nThe input OSM file must obey the following rules to work with the current\nalgorithms. The planet.osm export script used to generate the planet.osm\ndumps does conform to these rules (whether by accident or design).\n\n- The OSM file must be generated in node, segment, way order.\n- The ID of each object of a given type (e.g. nodes) must be increasing.\n\n\nDiff file format\n----------------\nThe diff format is an XML file containing OSM objects to delete and add.\nObjects which are modified have both a delete and add section. The format\nof each section is a copy of the OSM object from the planet.osm file.\n\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<planetdiff version=\"0.1\" generator=\"OpenStreetMap planetdiff\" from=\"a.osm\" to=\"b.osm\">\n <add>\n <node id=\"10310557\" timestamp=\"2006-07-10 23:17:35\" lat=\"51.7670078090236\" lon=\"-0.471281873153888\">\n <tag k=\"created_by\" v=\"JOSM\"/>\n </node>\n </add>\n <add>\n <node id=\"13602100\" timestamp=\"2006-08-16 00:02:13\" lat=\"51.778541285096\" lon=\"-0.448173637230418\"/>\n </add>\n <delete>\n <node id=\"26983956\" lat=\"51.77874880458334\" lon=\"-0.450481106821043\">\n <tag k=\"created_by\" v=\"JOSM\"/>\n </node>\n </delete>\n <add>\n <node id=\"26983956\" lat=\"51.77874880458334\" lon=\"-0.450481106821043\">\n <tag k=\"created_by\" v=\"JOSMXX\"/>\n </node>\n </add>\n...\n</planetdiff>\n\n\nSee example-diff.xml for an example file.\n\n\nExample usage:\n--------------\nThis example shows how the tool can be used to extract the differences\nbetween two planet.osm dumps. The errors below are from the UTF8sanitizer \ncode and can be ignored.\n\n\n$ planetdiff planet-070307.osm.bz2 planet-070321.osm.bz2 > delta2.xml\n\nProcessing: node(8420k)\nProcessing: segment(0k)Error at line 29333138\nError at line 29334932\nError at line 29334990\nError at line 29334990\nError at line 29334994\nError at line 29334994\nError at line 29336882\nError at line 29337279\nError at line 29337338\nError at line 29337351\nProcessing: segment(8830k)\nProcessing: way(370k)Error at line 72505269\nError at line 73944573\nProcessing: way(380k)Error at line 74022760\nError at line 72583739\nProcessing: way(430k) \n\n$ bzip2 -c delta2.xml > delta2.xml.bz2\n$ ls -l \n-rw-rw-r-- 1 jburgess jburgess 10732308 Apr 6 11:41 delta2.xml.bz2\n-rw-rw-r-- 1 jburgess jburgess 147704026 Apr 6 03:50 delta2.xml\n-rw-rw-r-- 1 jburgess jburgess 186168637 Mar 7 20:21 planet-070307.osm.bz2\n-rw-rw-r-- 1 jburgess jburgess 193761852 Mar 22 19:24 planet-070321.osm.bz2\n\nThe planet.osm file can be regenerated using planetpatch below. The compressed \ndiff file is only 10MB which is a much smaller download than a whole new planet.osm \ndump.\n\n\n\nplanetpatch\n===========\nGenerates a new planet.osm file by applying a differences file created by \nplanetdiff to an existing file.\n\n\nExample usage:\n--------------\nThe patch file generated by the planetdiff example above is used to\nregenerate the planet-070321.osm file:\n\n$ time planetpatch planet-070307.osm.bz2 delta2.xml > regen.xml\n\nProcessing: node(8420k)\nProcessing: segment(8830k)\nProcessing: way(370k)Error at line 72505269\nProcessing: way(380k)Error at line 72583739\nProcessing: way(430k)\n\nreal 19m54.654s\nuser 12m59.771s\nsys 3m35.929s\n\nThe output file, in this case 'regen.xml' should now be the same as an\nuncompressed and UTF8sanitized version of planet-070321.osm.bz2\n\n\n\nVerification\n------------\nTo verify that this is equal to the new planet.osm file we can compare it\nto a previously generated UTF8sanitized version of the same file. \n'cmp -l' reports every single byte of difference between the files (in octal)\n\n$ cmp -l planet-070321a.osm regen.xml\n1403627544 11 40\n3457266276 11 40\n\nIt seems that the process has converted the tab character (ASCII 9) to space (32).\nOther than these two character differences the generated output is identical to the\noriginal version of the new planet.osm. This seems close enough to be useable right\nnow.\n\n\n\n