gdal-vector-partition - Man Page

--output <OUTPUT-DIRECTORY>

Root of the output directory. [required]

--field <FIELD-NAME>

Fields(s) on which to partition. [required]

Only fields of type String, Integer and Integer64 are allowed. The order into which fields are specified matter to determine the directory hierarchy.

-f,  --of,  --format,  --output-format <OUTPUT-FORMAT>

Which output vector format to use. Allowed values may be given by gdal --formats | grep vector | grep rw | sort

--co <NAME>=<VALUE>

Many formats have one or more optional dataset creation options that can be used to control particulars about the file created. For instance, the GeoPackage driver supports creation options to control the version.

May be repeated.

The dataset creation options available vary by format driver, and some simple formats have no creation options at all. A list of options supported for a format can be listed with the --formats command line option but the documentation for the format is the definitive source of information on driver creation options. See Vector drivers format specific documentation for legal creation options for each format.

Note that dataset creation options are different from layer creation options.

--layer-creation-option <NAME>=<VALUE>

Many formats have one or more optional layer creation options that can be used to control particulars about the layer created. For instance, the GeoPackage driver supports layer creation options to control the feature identifier or geometry column name, setting the identifier or description, etc.

May be repeated.

The layer creation options available vary by format driver, and some simple formats have no layer creation options at all. A list of options supported for a format can be listed with the --formats command line option but the documentation for the format is the definitive source of information on driver creation options. See Vector drivers format specific documentation for legal creation options for each format.

Note that layer creation options are different from dataset creation options.

--overwrite

Allow program to overwrite existing target file or dataset. Otherwise, by default, gdal errors out if the target file or dataset already exists.

--append

Whether the output directory must be opened in append mode. Implies that it already exists and that the output format supports appending.

This mode is useful when adding new features to an already an existing partitioned dataset.

--scheme hive|flat

Partitioning scheme. Defaults to hive.

--pattern <PATTERN>

Filename pattern. User chosen string, with substitutions for:

• {LAYER_NAME}, when found, is substituted with the layer name (percent encoded where needed).
• {FIELD_VALUE}, when found, is substituted with the partitioning field value (percent encoded where needed). If several partitioning fields are used, each value is separated by underscore (_). Empty strings are substituted with __EMPTY__ and null fields with __NULL__.
• %[0?][0-9]?[0]?d: C-style integer formatter for the part number. Valid values are for example %d or %05d. One and only one part number specifier must be present in the pattern.

Default values for the pattern are part_%010d for the hive scheme, and {LAYER_NAME}_{FIELD_VALUE}_%010d for the flat scheme.`

--feature-limit <FEATURE-LIMIT>

Maximum number of features per file. By default, unlimited. If the limit is exceeded, several parts are created.

--max-file-size <MAX-FILE-SIZE>

Maximum file size (MB or GB suffix can be used). By default, unlimited. If the limit is exceeded, several parts are created.

Note that the maximum file size is used as a hint, and might not be strictly respected, because the evaluation of the file size corresponding to a feature is based on a heuristics, as the file size itself cannot be reliably used when it is under writing. In particular, the heuristics does not assume any compression, so for compressed formats, the actual size of a part can be significantly smaller than the specified limit.

--omit-partitioned-field

Whether to omit partitioned fields from the target layer definition. Automatically set for Parquet output format and Hive partitioning.

--skip-errors

Whether failures to write feature(s) should be ignored. Note that this option sets the size of the transaction unit to one feature at a time, which may cause severe slowdown when inserting into databases.

--oo <NAME>=<VALUE>

Dataset open option (format specific).

May be repeated.

--if <format>

Format/driver name to be attempted to open the input file(s). It is generally not necessary to specify it, but it can be used to skip automatic driver detection, when it fails to select the appropriate driver. This option can be repeated several times to specify several candidate drivers. Note that it does not force those drivers to open the dataset. In particular, some drivers have requirements on file extensions.

May be repeated.

$ gdal vector partition world_cities.gpkg out_directory --field continent,country --format Parquet

$ gdal pipeline ! read world_cities.gpkg ! filter --where "pop > 1e6" ! partition out_directory --field country --format GPKG --scheme flat