Configuration options

gdalcubes heavily relies on GDAL to read image data. GDAL itself comes with a lot of configuration options that can be set to improve computation times and I/O or network transfer performance. Below, you find the most important options for gdalcubes with a brief description. Please refer to the original documentation for details.

Configuration option	Description
`GDAL_CACHEMAX`	Size of the default block cache, can be set in byte, MB, or as a percentage of available main, memory. Defaults to 256 MB
`GDAL_DISABLE_READDIR_ON_OPEN`	Set to `TRUE` or `EMPTY_DIR` to avoid listing all files in the directory once a single file is opened (this is highly* recommended*). Defaults to `TRUE`
`GDAL_NUM_THREADS`	Number of threads GDAL can use for block reads and (de)compression, set to `ALL_CPUS` to use all available cores. Defaults to 1.
`VSI_CACHE`	Enable / disable per-file caching by setting to `TRUE` or `FALSE`. Default is `FALSE`.
`VSI_CACHE_SIZE`	Per-file cache size in bytes, defaults to 25 MB.
`CPL_VSIL_CURL_CACHE_SIZE`	Global cache size for downloads in bytes, defaults to 16 MB.

Please consider that settings apply for each gdalcubes worker process and you should calculate memory limits such as cache sizes accordingly. To set a configuration option, you can use gdalcubes_set_gdal_config() as in the following examples.

gdalcubes_set_gdal_config("GDAL_NUM_THREADS", "ALL_CPUS")
gdalcubes_set_gdal_config("VSI_CACHE", "TRUE")
gdalcubes_set_gdal_config("VSI_CACHE_SIZE", as.character(5*1000*1000)) # 5 MB per file
gdalcubes_set_gdal_config("GDAL_CACHEMAX", "2%")

Recommended settings for cloud access

Similar to the recommended settings for TiTiler (see here), the following options are recommended as a starting points to improve the performance of cloud data access.

gdalcubes_set_gdal_config("VSI_CACHE", "TRUE")
gdalcubes_set_gdal_config("GDAL_CACHEMAX","30%")
gdalcubes_set_gdal_config("VSI_CACHE_SIZE","10000000")
gdalcubes_set_gdal_config("GDAL_HTTP_MULTIPLEX","YES")
gdalcubes_set_gdal_config("GDAL_INGESTED_BYTES_AT_OPEN","32000")
gdalcubes_set_gdal_config("GDAL_DISABLE_READDIR_ON_OPEN","EMPTY_DIR")
gdalcubes_set_gdal_config("GDAL_HTTP_VERSION","2")
gdalcubes_set_gdal_config("GDAL_HTTP_MERGE_CONSECUTIVE_RANGES","YES")
gdalcubes_set_gdal_config("GDAL_NUM_THREADS", "ALL_CPUS")

To access imagery from requester pays buckets, you additionally may want to set:

gdalcubes_set_gdal_config("AWS_ACCESS_KEY_ID", "xxxxxxxxxxxxxx")
gdalcubes_set_gdal_config("AWS_SECRET_ACCESS_KEY", "xxxxxxxxxxxxxx")
gdalcubes_set_gdal_config("AWS_REQUEST_PAYER", "requester")

For more details, please refer to the GDAL documentation of virtual file systems for cloud providers.

# Configuration options {#gdaloptions} gdalcubes heavily relies on GDAL to read image data. GDAL itself comes with a lot of [configuration options](https://gdal.org/user/configoptions.html) that can be set to improve computation times and I/O or network transfer performance. Below, you find the most important options for gdalcubes with a brief description. Please refer to the [original documentation](https://gdal.org/user/configoptions.html) for details. | Configuration option | Description | |:-------------------- |:-------------------------------------------------- | | `GDAL_CACHEMAX` | Size of the default block cache, can be set in byte, MB, or as a percentage of available main, memory. Defaults to 256 MB | | `GDAL_DISABLE_READDIR_ON_OPEN` | Set to `TRUE` or `EMPTY_DIR` to avoid listing all files in the directory once a single file is opened (_this is **highly** recommended_). Defaults to `TRUE` | | | `GDAL_NUM_THREADS` | Number of threads GDAL can use for block reads and (de)compression, set to `ALL_CPUS` to use all available cores. Defaults to 1. | | `VSI_CACHE` | Enable / disable _per-file_ caching by setting to `TRUE` or `FALSE`. Default is `FALSE`.| | `VSI_CACHE_SIZE` | _Per-file_ cache size in bytes, defaults to 25 MB. | | `CPL_VSIL_CURL_CACHE_SIZE` | Global cache size for downloads in bytes, defaults to 16 MB. |   Please consider that settings apply for each gdalcubes worker process and you should calculate memory limits such as cache sizes accordingly. To set a configuration option, you can use `gdalcubes_set_gdal_config()` as in the following examples. ```{.r code-line-numbers="false"} gdalcubes_set_gdal_config("GDAL_NUM_THREADS", "ALL_CPUS") gdalcubes_set_gdal_config("VSI_CACHE", "TRUE") gdalcubes_set_gdal_config("VSI_CACHE_SIZE", as.character(5*1000*1000)) # 5 MB per file gdalcubes_set_gdal_config("GDAL_CACHEMAX", "2%") ``` ## Recommended settings for cloud access Similar to the recommended settings for TiTiler [(see here)](https://developmentseed.org/titiler/advanced/performance_tuning/), the following options are recommended as a starting points to improve the performance of cloud data access. ```{.r code-line-numbers="false"} gdalcubes_set_gdal_config("VSI_CACHE", "TRUE") gdalcubes_set_gdal_config("GDAL_CACHEMAX","30%") gdalcubes_set_gdal_config("VSI_CACHE_SIZE","10000000") gdalcubes_set_gdal_config("GDAL_HTTP_MULTIPLEX","YES") gdalcubes_set_gdal_config("GDAL_INGESTED_BYTES_AT_OPEN","32000") gdalcubes_set_gdal_config("GDAL_DISABLE_READDIR_ON_OPEN","EMPTY_DIR") gdalcubes_set_gdal_config("GDAL_HTTP_VERSION","2") gdalcubes_set_gdal_config("GDAL_HTTP_MERGE_CONSECUTIVE_RANGES","YES") gdalcubes_set_gdal_config("GDAL_NUM_THREADS", "ALL_CPUS") ``` To access imagery from requester pays buckets, you additionally may want to set: ```{.r code-line-numbers="false"} gdalcubes_set_gdal_config("AWS_ACCESS_KEY_ID", "xxxxxxxxxxxxxx") gdalcubes_set_gdal_config("AWS_SECRET_ACCESS_KEY", "xxxxxxxxxxxxxx") gdalcubes_set_gdal_config("AWS_REQUEST_PAYER", "requester") ``` For more details, please refer to the GDAL documentation of [virtual file systems for cloud providers](https://gdal.org/user/virtual_file_systems.html).