gdalcubes_options

Set or read global options of the gdalcubes package

Description

Set global package options to change the default behavior of gdalcubes. These include how many parallel processes are used to process data cubes, how created netCDF files are compressed, and whether or not debug messages should be printed.

Usage

gdalcubes_options(
  ...,
  parallel,
  ncdf_compression_level,
  debug,
  cache,
  ncdf_write_bounds,
  use_overview_images,
  show_progress,
  default_chunksize,
  streaming_dir,
  log_file,
  threads
)

Arguments

Argument	Description
…	not used
parallel	number of parallel workers used to process data cubes or TRUE to use the number of available cores automatically
ncdf_compression_level	integer; compression level for created netCDF files, 0=no compression, 1=fast compression, 9=small compression
debug	logical; print debug messages
cache	logical; TRUE if temporary data cubes should be cached to support fast reprocessing of the same cubes
ncdf_write_bounds	logical; write dimension bounds as additional variables in netCDF files
use_overview_images	logical; if FALSE, all images are read on original resolution and existing overviews will be ignored
show_progress	logical; if TRUE, a progress bar will be shown for actual computations
default_chunksize	length-three vector with chunk size in t, y, x directions or a function taking a data cube size and returning a suggested chunk size
streaming_dir	directory where temporary binary files for process streaming will be written to
log_file	character, if empty string or NULL, diagnostic messages will be printed to the console, otherwise to the provided file
threads	number of threads used to process data cubes (deprecated)

Details

Data cubes can be processed in parallel where the number of chunks in a cube is distributed among parallel worker processes. The actual number of used workers can be lower if a data cube as less chunks. If parallel is TRUE, the number of available cores is used. Setting parallel = FALSE can be used to disable parallel processing. Notice that since version 0.6.0, separate processes are being used instead of parallel threads to avoid possible R session crashes due to some multithreading issues.

Caching has no effect on disk or memory consumption, it simply tries to reuse existing temporary files where possible. For example, changing only parameters to plot will void reprocessing the same data cube if cache is TRUE.

The streaming directory can be used to control the performance of user-defined functions, if disk IO is a bottleneck. Ideally, this can be set to a directory on a shared memory device.

Passing no arguments will return the current options as a list.

Examples

gdalcubes_options(parallel=4) # set the number 
gdalcubes_options() # print current options

$parallel
[1] 4

$ncdf_compression_level
[1] 1

$debug
[1] FALSE

$cache
[1] TRUE

$ncdf_write_bounds
[1] TRUE

$use_overview_images
[1] TRUE

$show_progress
[1] FALSE

$default_chunksize
function(nt, ny, nx) {
  nparallel = .pkgenv$parallel
  target_nchunks_space = ceiling(2*nparallel)
  cx = sqrt(nx*ny / target_nchunks_space)
  cx = ceiling(cx / 64)*64 # use multiples of 64 only
  cy = cx
  
  # apply limits
  cx = min(cx, 1024)
  cy = min(cy, 1024)
  cx = max(cx, 64)
  cy = max(cy, 64)
  cx = min(nx, cx)
  cy = min(ny, cy)
  
  return(c(1, ceiling(cy), ceiling(cx)))
}
<bytecode: 0x562ddf6181a0>
<environment: namespace:gdalcubes>

$streaming_dir
[1] "/tmp/Rtmp6li43F"

gdalcubes_options(parallel=FALSE) # reset

# gdalcubes_options Set or read global options of the gdalcubes package ```{r include=FALSE} library(gdalcubes) ``` ## Description Set global package options to change the default behavior of gdalcubes. These include how many parallel processes are used to process data cubes, how created netCDF files are compressed, and whether or not debug messages should be printed. ## Usage ```r gdalcubes_options( ..., parallel, ncdf_compression_level, debug, cache, ncdf_write_bounds, use_overview_images, show_progress, default_chunksize, streaming_dir, log_file, threads ) ``` ## Arguments | Argument | Description | |:------------|:----------------------------------| | ... | not used | | parallel | number of parallel workers used to process data cubes or TRUE to use the number of available cores automatically | | ncdf_compression_level | integer; compression level for created netCDF files, 0=no compression, 1=fast compression, 9=small compression | | debug | logical; print debug messages | | cache | logical; TRUE if temporary data cubes should be cached to support fast reprocessing of the same cubes | | ncdf_write_bounds | logical; write dimension bounds as additional variables in netCDF files | | use_overview_images | logical; if FALSE, all images are read on original resolution and existing overviews will be ignored | | show_progress | logical; if TRUE, a progress bar will be shown for actual computations | | default_chunksize | length-three vector with chunk size in t, y, x directions or a function taking a data cube size and returning a suggested chunk size | | streaming_dir | directory where temporary binary files for process streaming will be written to | | log_file | character, if empty string or NULL, diagnostic messages will be printed to the console, otherwise to the provided file | | threads | number of threads used to process data cubes (deprecated) | ## Details Data cubes can be processed in parallel where the number of chunks in a cube is distributed among parallel worker processes. The actual number of used workers can be lower if a data cube as less chunks. If parallel is TRUE, the number of available cores is used. Setting parallel = FALSE can be used to disable parallel processing. Notice that since version 0.6.0, separate processes are being used instead of parallel threads to avoid possible R session crashes due to some multithreading issues. Caching has no effect on disk or memory consumption, it simply tries to reuse existing temporary files where possible. For example, changing only parameters to `plot` will void reprocessing the same data cube if cache is TRUE. The streaming directory can be used to control the performance of user-defined functions, if disk IO is a bottleneck. Ideally, this can be set to a directory on a shared memory device. Passing no arguments will return the current options as a list. ## Examples ```{r} gdalcubes_options(parallel=4) # set the number gdalcubes_options() # print current options gdalcubes_options(parallel=FALSE) # reset ```