4.3. Command-Line Interface
4.3.1. Overview
The CCI Toolbox comprises a single command-line executable, which is called cate
and is available after installing
the CCI Toolbox on your computer. See section Local Setup for more information. The command-line
interface allows for accessing local and remote datasets as well as running virtually all CCI Toolbox
operations on them.
Developers only: If you build and install the CCI Toolbox from Python sources into your current Python environment,
cate
will be registered as an executable script. It can be found as $PYTHON_PREFIX/bin/cate.sh
on Unix systems
and as %PYTHON_PREFIX%\\Scripts\\cate.exe
on Windows systems where PYTHON_PREFIX
is the path to the current
Python environment.
In the console / terminal window type:
cate -h
This should output the following usage help::
usage: cate [-h] [--version] [--traceback] [--license] [--docs] COMMAND ...
ESA CCI Toolbox (Cate) command-line interface, version 2.0
positional arguments:
COMMAND One of the following commands. Type "COMMAND -h" to get
command-specific help.
ds Manage data sources.
op Manage data operations.
ws Manage workspaces.
res Manage workspace resources.
run Run an operation or Workflow file.
io Manage supported data and file formats.
upd Update an existing cate environment to a specific or to the
latest cate version
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
--traceback show (Python) stack traceback for the last error
--license show software license and exit
--docs show software documentation in a browser window
cate
uses up to two sub-command levels. Each sub-command has its own set of options and arguments and can display
help when used with the option `--help
or -h
. The first sub-command level comprises the following list of
commands:
The following first level sub-commands are used to work interactively with datasets and operations:
When you encounter any error while using cate
and you want to report the problem
to the development team, we kindly ask you to rerun the command with option --traceback
and include the Python stack
traceback with a short description of your problem.
4.3.2. Examples
The following examples shall help you understand the basic concepts behind the various cate
commands.
4.3.2.1. Manage datasets
To list all available data sources, type:
cate ds list
To query all data sources that have ozone
in their name, type:
cate ds list -n ozone
To get more detailed information on a specific data source, e.g. esacci.OZONE.mon.L3...
, type:
cate ds info esacci.OZONE.mon.L3.NP.multi-sensor.multi-platform.MERGED.fv0002.r1
To add a local data source from all NetCDF files in e.g. data/sst_v3
and name it e.g. SSTV3
, type:
cate ds def SSTV3 data/sst_v3/*.nc
Make sure it is there:
cate ds list -n SSTV3
To make a temporal subset ECV data source locally available, i.e. avoid remote data access during its usage:
cate ds copy esacci.OZONE.mon.L3.NP.multi-sensor.multi-platform.MERGED.fv0002.r1 -t 2006-01-01,2007-12-31
The section Configuration in Local Setup describes, how to configure the location of directory in which Cate stores such synchronised data.
4.3.2.2. Inspect available operations
To list all available operations, type:
cate op list
To display more details about a particular operation, e.g. tseries_point
, type:
cate op info tseries_point
4.3.2.3. Run an operation
The cate run
command is used to execute single operations. The open
and read
options are used to
ingest datasets which can then be referenced by name. A write
option allows to write the operation result into a
file.
To run the tseries_point
operation on a dataset, e.g. the local.SSTV3
(from above), at lat=0 and lon=0, type:
cate run --open ds=local.SSTV3 --write ts2.nc tseries_point ds=ds lat=0 lon=0
To run the tseries_point
operation on a netCDF file, e.g. test/ui/precip_and_temp.nc
at lat=0 and lon=0, type:
cate run --read ds=test/ui/precip_and_temp.nc --write ts2.nc tseries_point ds=ds lat=0 lon=0
4.3.2.4. Interactive session
The following command sequence is a simple example for an interactive session using the Cate command-line:
cate ws new
cate res open sst local.SSTV3
cate res set sst_ts tseries_point ds=@sst lat=0 lon=0
cate res plot sst_ts
cate res write sst_ts sst_ts.nc
cate ws status
The steps above explained:
cate ws new
is used to create a new in-memory workspace. A workspace can hold any number of named workspace resources which may refer to opened datasets or any other ingested or computed objects.cate res open
is used to open a dataset from the available data stores and assign the opened dataset to the workspace resourcesst
. Accordingly,cate res read
could have been used to read from a local netCDF file.cate res set
assigns the result of thetseries_point
operation to workspace resourcesst_ts
. Note the at-character “@” used as prefix for the inputds
. This indicates that value for inputds
of steptseries_point
will be retrieved “at” theopen
step namedsst
. It establishes a connection between stepopen
andtseries_point
. In fact, this is the way processing graphs are constructed using the CLI.cate res plot
plots the workspace resourcesst_ts
.cate res write
writes the workspace resourcesst_ts
to a netCDF file./sst_ts.nc
.cate ws status
shows the current workspace status and lists all workspace resource assignments.
We could now save the current workspace state and close it:
cate ws save
cate ws close
cate ws save
creates a hidden sub-directory .cate-workspace
and herewith makes the current directory a
workspace directory. cate
uses this hidden directory to persist the workspace state information.
At a later point in time, you could cd
into any of your workspace directories, and:
cate ws open
cate ws status
in order to reopen it, display its status, and continue interactively working with its resources.
The following subsections provide detailed information about the cate
commands.
4.3.3. cate ds
- Dataset Management
Provides a set of sub-commands used to manage climate data sources. Data sources are used to open local and remote datasets which are input to various analysis and processing operations. Type “cate op -h” to find out more about available operations.
usage: cate ds [-h] COMMAND ...
4.3.7.1. Positional Arguments
- COMMAND
Possible choices: list, info, add, del, copy
One of the following commands. Type “COMMAND -h” for help.
4.3.7.2. Sub-commands
4.3.6.2.10. list
List all available data sources
cate ds list [-h] [--name NAME] [--all] [--coverage]
4.3.7.2.1.2. Named Arguments
- --name, -n
List only data sources named NAME or that have NAME in their name. The comparison is case insensitive.
- --all, -a
Show also data sources that can not be opened in Cate
Default: True
- --coverage, -c
Also display temporal coverage
Default: False
4.3.4.2.2. info
Display information about a data source.
cate ds info [-h] [--var] [--local] DS
4.3.7.2.1.1. Positional Arguments
- DS
A data source name. Type “cate ds list” to show all possible data source names.
4.3.7.2.2.2. Named Arguments
- --var, -v
Also display information about contained dataset variables.
Default: False
- --local, -l
Also display temporal coverage of cached datasets.
Default: False
4.3.3.2.3. add
Add a new file data source using a file path pattern.
cate ds add [-h] DS FILE [FILE ...]
4.3.7.2.2.1. Positional Arguments
- DS
A name for the data source.
- FILE
A list of files comprising this data source. The files can contain the wildcard characters “*” and “?”.
4.3.7.2.6. del
Removes a data source from file data store.
cate ds del [-h] [-k] [-y] DS
4.3.7.2.3.1. Positional Arguments
- DS
A name for the data source.
4.3.7.2.3.2. Named Arguments
- -k, --keep_files
Do not ask for confirmation.
Default: False
- -y, --yes
Do not ask for confirmation.
Default: False
4.3.3.2.5. copy
Makes a file copy of any other data source. The copy may be limited to a subset by optional constraints.
cate ds copy [-h] [--name NAME] [--time TIME] [--region REG] [--vars VARS]
REF_DS
4.3.7.2.4.1. Positional Arguments
- REF_DS
A name of origin data source.
4.3.7.2.4.2. Named Arguments
- --name, -n
A name for new data source.
- --time, -t
Time range constraint. Use format “YYYY-MM-DD,YYYY-MM-DD”.
- --region, -r
Region constraint. Use format: “min_lon,min_lat,max_lon,max_lat”.
- --vars, -v
Names of variables to be included. Use format “pattern1,pattern2,…”
4.3.4. cate op
- Operation Management
Provides a set of commands to inquire the available operations used to analyse and process climate datasets.
usage: cate op [-h] COMMAND ...
4.3.7.1. Positional Arguments
- COMMAND
Possible choices: list, info
One of the following commands. Type “COMMAND -h” for help.
4.3.7.2. Sub-commands
4.3.6.2.10. list
List operations.
cate op list [-h] [--name NAME] [--tag TAG] [--deprecated] [--internal]
4.3.7.2.1.2. Named Arguments
- --name, -n
List only operations with name NAME or that have NAME in their name. The comparison is case insensitive.
- --tag, -t
List only operations tagged by TAG or that have TAG in one of their tags. The comparison is case insensitive.
- --deprecated, -d
List deprecated operations.
Default: False
- --internal, -i
List operations tagged “internal”.
Default: False
4.3.5. cate run
- Running Operations and Workflows
Runs the given operation or Workflow file with the specified operation arguments. Argument values may be constant values or the names of data loaded by the –open or –read options. Type “cate op list” to list all available operations. Type “cate op info” to find out which arguments are supported by a given operation.
usage: cate run [-h] [-m] [-o DS_EXPR] [-r FILE_EXPR] [-w FILE_EXPR] OP ...
4.3.7.1. Positional Arguments
- OP
Fully qualified operation name or Workflow file. Type “cate op list” to list available operators.
- ...
Operation arguments given as KEY=VALUE. KEY is any supported input by OP. VALUE depends on the expected data type of an OP input. It can be a True, False, a string, a numeric constant, one of the names specified by the –open and –read options, or a Python expression. Type “cate op info OP” to print information about the supported OP input names to be used as KEY and their data types to be used as VALUE.
4.3.7.2.1.2. Named Arguments
- -m, --monitor
Display progress information during execution.
Default: False
- -o, --open
Open a dataset from DS_EXPR. The DS_EXPR syntax is NAME=DS[,START[,END]]. DS must be a valid data source name. Type “cate ds list” to show all known data source names. START and END are dates and may be used to create temporal data subsets. The dataset loaded will be assigned to the arbitrary name NAME which is used to pass the datasets or its variables as an OP argument. To pass a variable use syntax NAME.VAR_NAME.
- -r, --read
Read object from FILE_EXPR. The FILE_EXPR syntax is NAME=PATH[,FORMAT]. Type “cate io list -r” to see which formats are supported.If FORMAT is not provided, file format is derived from the PATH’s filename extensions or file content. NAME may be passed as an OP argument that receives a dataset, dataset variable or any other data type. To pass a variable of a dataset use syntax NAME.VAR_NAME
- -w, --write
Write result to FILE_EXPR. The FILE_EXPR syntax is [NAME=]PATH[,FORMAT]. Type “cate io list -w” to see which formats are supported.If FORMAT is not provided, file format is derived from the object type and the PATH’s filename extensions. If OP returns multiple named output values, NAME is used to identify them. Multiple -w options may be used in this case.
4.3.6. cate ws
: Workspace Management
Used to create, open, save, modify, and delete workspaces. Workspaces contain named workflow resources, which can be datasets read from data stores, or any other data objects originating from applying operations to datasets and other data objects. The origin of every resource is stored in the workspace’s workflow description. Type “cate res -h” for more information about workspace resource commands.
usage: cate ws [-h] COMMAND ...
4.3.7.1. Positional Arguments
- COMMAND
Possible choices: init, new, open, close, save, run, del, clean, status, list, exit
One of the following commands. Type “COMMAND -h” for help.
4.3.7.2. Sub-commands
4.3.6.2.1. init
Initialize workspace.
cate ws init [-h] [-d DIR] [--desc DESCRIPTION]
4.3.7.2.1.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- --desc
Workspace description.
4.3.6.2.2. new
Create new in-memory workspace.
cate ws new [-h] [-d DIR] [--desc DESCRIPTION]
4.3.7.2.2.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- --desc
Workspace description.
4.3.7.2.1. open
Open workspace.
cate ws open [-h] [-d DIR]
4.3.7.2.3.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
4.3.6.2.4. close
Close workspace.
cate ws close [-h] [-d DIR] [-a] [-s]
4.3.7.2.4.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- -a, --all
Close all workspaces. Ignores DIR option.
Default: False
- -s, --save
Save modified workspace before closing.
Default: False
4.3.6.2.5. save
Save workspace.
cate ws save [-h] [-d DIR] [-a]
4.3.7.2.5.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- -a, --all
Save all workspaces. Ignores DIR option.
Default: False
4.3.6.2.6. run
Run operation.
cate ws run [-h] [-d DIR] OP ...
4.3.7.2.1.1. Positional Arguments
- OP
Operation name or Workflow file path. Type “cate op list” to list available operations.
- ...
Operation arguments given as KEY=VALUE. KEY is any supported input by OP. VALUE depends on the expected data type of an OP input. It can be either a value or a reference an existing resource prefixed by the add character “@”. The latter connects to operation steps with each other. To provide a (constant) value you can use boolean literals True and False, strings, or numeric values. Type “cate op info OP” to print information about the supported OP input names to be used as KEY and their data types to be used as VALUE.
4.3.7.2.6.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
4.3.7.2.6. del
Delete workspace.
cate ws del [-h] [-d DIR] [-y]
4.3.7.2.7.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- -y, --yes
Do not ask for confirmation.
Default: False
4.3.6.2.8. clean
Clean workspace (removes all resources).
cate ws clean [-h] [-d DIR] [-y]
4.3.7.2.8.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- -y, --yes
Do not ask for confirmation.
Default: False
4.3.7. cate res
- Workspace Resources Management
Used to set, run, open, read, write, plot, etc. workspace resources. All commands expect an opened workspace. Type “cate ws -h” for more information about workspace commands.
usage: cate res [-h] COMMAND ...
4.3.7.1. Positional Arguments
- COMMAND
Possible choices: open, read, write, set, rename, del, print, plot
One of the following commands. Type “COMMAND -h” for help.
4.3.7.2. Sub-commands
4.3.7.2.1. open
Open a dataset from a data source and set a resource.
cate res open [-h] [-d DIR] NAME DS [START] [END] [REGION] [VAR_NAMES]
4.3.7.2.1.1. Positional Arguments
- NAME
Name of the new target resource.
- DS
A data source named DS. Type “cate ds list” to list valid data source names.
- START
Start date. Use format “YYYY[-MM[-DD]]”.
- END
End date. Use format “YYYY[-MM[-DD]]”.
- REGION
Region constraint. Use format “min_lon,min_lat,max_lon,max_lat”.
- VAR_NAMES
Names of variables to be included. Use format “pattern1,pattern2,pattern3”.
4.3.7.2.1.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
4.3.7.2.2. read
Read an object from a file and set a resource.
cate res read [-h] [-d DIR] [-f FORMAT] NAME FILE
4.3.7.2.2.1. Positional Arguments
- NAME
Name of the new target resource.
- FILE
File path.
4.3.7.2.2.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- -f, --format
File format. Type “cate io list -r” to see which formats are supported.
4.3.7.2.3. write
Write a resource to a file.
cate res write [-h] [-d DIR] [-f FORMAT] NAME FILE
4.3.7.2.3.1. Positional Arguments
- NAME
Name of an existing resource.
- FILE
File path.
4.3.7.2.3.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- -f, --format
File format. Type “cate io list -w” to see which formats are supported.
4.3.7.2.4. set
Set a resource from the result of an operation.
cate res set [-h] [-d DIR] [-o] NAME OP ...
4.3.7.2.4.1. Positional Arguments
- NAME
Name of the target resource to be set. Use -o to overwrite an existing NAME.
- OP
Operation name. Type “cate op list” to list available operation names.
- ...
Operation arguments given as KEY=VALUE. KEY is any supported input by OP. VALUE depends on the expected data type of an OP input. It can be either a value or a reference an existing resource prefixed by the add character “@”. The latter connects to operation steps with each other. To provide a (constant) value you can use boolean literals True and False, strings, or numeric values. Type “cate op info OP” to print information about the supported OP input names to be used as KEY and their data types to be used as VALUE.
4.3.7.2.4.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- -o, --overwrite
Overwrite an existing workflow step / target resource with same NAME.
Default: False
4.3.7.2.5. rename
Rename a resource.
cate res rename [-h] [-d DIR] NAME NEW_NAME
4.3.7.2.5.1. Positional Arguments
- NAME
Resource name.
- NEW_NAME
New resource name.
4.3.7.2.5.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
4.3.7.2.6. del
Delete a resource.
cate res del [-h] [-d DIR] NAME
4.3.7.2.6.1. Positional Arguments
- NAME
Resource name.
4.3.7.2.6.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
4.3.7.2.7. print
If EXPR is omitted, print value of all current resources. Otherwise, if EXPR identifies a resource, print its value. Else print the value of a (Python) expression evaluated in the context of the current workspace.
cate res print [-h] [-d DIR] [EXPR]
4.3.7.2.7.1. Positional Arguments
- EXPR
Name of an existing resource or a valid (Python) expression.
4.3.7.2.7.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
4.3.7.2.8. plot
Plot a resource or the value of a (Python) expression evaluated in the context of the current workspace.
cate res plot [-h] [-d DIR] [-v [VAR]] [-o [FILE]] EXPR
4.3.7.2.8.1. Positional Arguments
- EXPR
Name of an existing resource or any (Python) expression.
4.3.7.2.8.2. Named Arguments
- -d, --dir
The workspace’s base directory. If not given, the current working directory is used.
Default: “.”
- -v, --var
Name of a variable to plot.
- -o, --out
Output file to write the plot figure to.