Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use CLI11 command line parser library #2115

Merged
merged 1 commit into from
Jan 7, 2024

Conversation

joto
Copy link
Collaborator

@joto joto commented Dec 22, 2023

This switches parsing of command line arguments from getopt to the CLI11 library.

Its adds CLI11 as a new dependency. With this commit CLI11 is vendored in in the contrib directory, because it is not available in Ubuntu 20.04 and Debian before Bookworm. It is available in Ubuntu 22.04 und Debian Bookworm and it is available in Homebrew and as vcpkg. So a bit down the line we can switch to the versions available from the OS distribution.

For Windows we don't need the getopt version from alex85k any more and this commit also removes that code from the Github action setup.

This commit is quite large, because we have to change everything at the same time. Changes are for osm2pgsql and osm2pgsql-gen which now share part of the option parsing code (database and logging options), in the new src/command-line-app.[ch]pp files.

The code tries to keep the functionality the same as much as possible. So no extra checks on command line options or so, these can come later.

There are some unavoidable changes:

  • Some error messages have changed
  • Help output is now generated by the CLI11 library so it looks very different. There is no verbose help version any more. All command line options are shown with --help, but some with a bit less detail than before. But all the detail is in the man page anyway.
  • CLI11 can not parse an empty option parameter, so something like "--foo=" does not work, it tries to use the next option as value for the option. This is not very relevant in the real world, but there might be some corner cases where this changes behaviour.

See #142

This switches parsing of command line arguments from getopt to the CLI11
library.

Its adds CLI11 as a new dependency. With this commit CLI11 is vendored
in in the contrib directory, because it is not available in Ubuntu 20.04
and Debian before Bookworm. It is available in Ubuntu 22.04 und Debian
Bookworm and it is available in Homebrew and as vcpkg. So a bit down the
line we can switch to the versions available from the OS distribution.

For Windows we don't need the getopt version from alex85k any more and
this commit also removes that code from the Github action setup.

This commit is quite large, because we have to change everything at the
same time. Changes are for osm2pgsql and osm2pgsql-gen which now share
part of the option parsing code (database and logging options), in the
new src/command-line-app.[ch]pp files.

The code tries to keep the functionality the same as much as possible.
So no extra checks on command line options or so, these can come later.

There are some unavoidable changes:
* Some error messages have changed
* Help output is now generated by the CLI11 library so it looks very
  different. There is no verbose help version any more. All command
  line options are shown with --help, but some with a bit less detail
  than before. But all the detail is in the man page anyway.
* CLI11 can not parse an empty option parameter, so something like
  "--foo=" does not work, it tries to use the next option as value
  for the option. This is not very relevant in the real world, but
  there might be some corner cases where this changes behaviour.

See osm2pgsql-dev#142
@joto
Copy link
Collaborator Author

joto commented Dec 22, 2023

Here is the output of osm2pgsql --help:

osm2pgsql -- Import OpenStreetMap data into a PostgreSQL/PostGIS database

Usage: ./osm2pgsql [OPTIONS] [OSMFILE...]

Positionals:
  OSMFILE ...                         OSM input file(s). Read manual before using multiple files!

Options:
  -h,--help                           Print this help message and exit.
  -V,--version                        Show version and exit.
  -a,--append                         Update existing osm2pgsql database (needs --slim).
  -c,--create                         Import OSM data from file into database. This is the default if --append is not used.
  -s,--slim                           Store raw OSM data in the database. Required if you want to update with --append later.


Database options:
  -d,--database DB                    Database name or PostgreSQL conninfo string.
  -U,--user USERNAME                  Database user.
  -W,--password                       Force password prompt.
  -H,--host HOST                      Database server hostname or unix domain socket location.
  -P,--port PORT                      Database server port.
  -p,--prefix PREFIX                  Prefix for table names (default: 'planet_osm').
  --schema SCHEMA                     Database schema (default: 'public').


Logging options:
  --log-level LEVEL                   Set log level ('debug', 'info' (default), 'warn', or 'error').
  --log-progress PROGRESS             Log progress to console ('true', 'false', 'auto' (default)).
  --log-sql                           Enable logging of SQL commands for debugging.
  --log-sql-data                      Enable logging of all data added to the database.
  -v,--verbose                        Enable debug logging.


Output options:
  -O,--output OUTPUT                  Set output ('pgsql' (default), 'flex', 'gazetteer' (deprecated), 'null').
  -S,--style FILE:FILE                Location of the style file. (Default: '/usr/local/share/osm2pgsql/default.style').


Pgsql output options:
  -k,--hstore Excludes: --hstore-all  Add tags without column to an additional hstore (key/value) column.
  --hstore-add-index                  Add index to hstore (key/value) column.
  -j,--hstore-all Excludes: --hstore  Add all tags to an additional hstore (key/value) column.
  -z,--hstore-column NAME ...         Add additional hstore (key/value) column.
  --hstore-match-only                 Only keep objects that have a non-NULL value in one of the columns.
  -K,--keep-coastlines                Keep coastline data (default: discard objects tagged natural=coastline).
  -l,--latlong                        Store data in degrees of latitude & longitude (WGS84).
  -m,--merc                           Store data in Web Mercator [EPSG 3857]. This is the default if --latlong or --proj are not used.
  -G,--multi-geometry                 Generate multi-geometry features in database tables.
  --output-pgsql-schema SCHEMA        Database schema for pgsql output tables (default: setting of --schema).
  -E,--proj SRID                      Use projection EPSG:SRID.
  --reproject-area                    Compute area column using Web Mercator coordinates.
  --tag-transform-script SCRIPT       Specify a Lua script to handle tag filtering and normalisation.


Expire options:
  --expire-bbox-size SIZE             Max size for a polygon to expire the whole polygon, not just the boundary (default: 20000).
  -o,--expire-output FILE             Output filename for expired tiles list.
  -e,--expire-tiles [MINZOOM-]MAXZOOM Create a tile expiry list. Zoom levels must be larger than 0 and smaller than 32.


Middle options:
  -C,--cache SIZE                     Use up to SIZE MB for caching nodes (default: 800).
  --drop                              Drop middle tables after import (needs --slim).
  -x,--extra-attributes               Include attributes (version, timestamp, changeset id, user id, and user name) for each OSM object.
  -F,--flat-nodes FILE                File for storing node locations (needs --slim, default: store in database).
  --middle-schema SCHEMA              Database schema for middle tables (default: setting of --schema).
  --middle-database-format FORMAT     Set middle db format ('legacy' (default), 'new').
  --middle-with-nodes                 Store tagged nodes in db (new middle db format only).


Input options:
  -b,--bbox MINX,MINY,MAXX,MAXY       Apply a bounding box filter on the imported data, e.g. '--bbox -0.5,51.25,0.5,51.75'.
  -r,--input-reader FORMAT            Input format ('xml', 'pbf', 'o5m', 'opl', 'auto' - autodetect format (default)).


Advanced options:
  -I,--disable-parallel-indexing      Disable concurrent index creation.
  --middle-way-node-index-id-shift N  Set ID shift for bucket index.
  --number-processes NUM:INT bounded to [1 - 32]
                                      Specifies the number of parallel processes used for certain operations (default: number of CPUs).


Tablespace options:
  -i,--tablespace-index TBLSPC        Tablespace for indexes (sets default for --tablespace-(main|slim)-index).
  --tablespace-main-data TBLSPC       Tablespace for main tables.
  --tablespace-main-index TBLSPC      Tablespace for main indexes.
  --tablespace-slim-data TBLSPC       Tablespace for slim mode tables.
  --tablespace-slim-index TBLSPC      Tablespace for slim mode indexes.


Deprecated options:
  --cache-strategy TYPE               The option has been removed and only a warning is printed.
  --with-forward-dependencies BOOL    Propagate changes from nodes to ways and node/way members to relations (default: true).

And this is the output of osm2pgsq-gen --help:

osm2pgsql-gen -- Generalize OpenStreetMap data

Usage: ./osm2pgsql-gen [OPTIONS]

Options:
  -h,--help                   Print this help message and exit.
  -V,--version                Show version and exit.
  -a,--append                 Run in append mode.
  -S,--style FILE             The Lua config/style file (same as for osm2pgsql).
  -j,--jobs NUM:INT in [1 - 256]
                              Number of parallel jobs (default: 1, max 256).


Database options:
  -d,--database DB            Database name or PostgreSQL conninfo string.
  -U,--user USERNAME          Database user.
  -W,--password               Force password prompt.
  -H,--host HOST              Database server hostname or unix domain socket location.
  -P,--port PORT              Database server port.
  --middle-schema SCHEMA      Database schema for middle tables (default: setting of --schema).
  --schema SCHEMA             Database schema (default: 'public').


Logging options:
  --log-level LEVEL           Set log level ('debug', 'info' (default), 'warn', or 'error').
  --log-progress PROGRESS     Log progress to console ('true', 'false', 'auto' (default)).
  --log-sql                   Enable logging of SQL commands for debugging.
  --log-sql-data              Enable logging of all data added to the database.

@pnorman
Copy link
Collaborator

pnorman commented Dec 22, 2023

Huge 👍 to moving from getopt to something that handles more of the work for us.

@lonvia lonvia merged commit 59bb561 into osm2pgsql-dev:master Jan 7, 2024
26 checks passed
@joto joto deleted the cmdline-parser branch January 29, 2024 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants