Welcome to AQUA’s documentation!

• Introduction
  • Overview of AQUA
  • Purpose
  • Key Features
  • Contributing
• Installation
  • Prerequisites
  • Installation with Miniforge
  • Update of the environment
  • HPC Installation
    • Installation on LUMI HPC
    • Installation on Levante HPC at DKRZ
    • Installation on MareNostrum 5 (MN5) HPC at BSC
    • Installation on ECMWF HPC2020
  • Installation and use of the AQUA container
• Getting Started
  • Basic Concepts
  • Python installation
  • Catalog installation
    • Set up the configuration folder
    • Add a catalog
  • Set up Jupyter kernel
• Example use case
• The Reader class
  • The Reader
  • Input and Output formats
  • Catalog exploration
  • Reader basic usage
  • Dask and streaming capabilities
• The AQUA console
  • What is the AQUA console?
  • aqua install
  • aqua avail
  • aqua add <catalog>
  • aqua remove <catalog>
  • aqua set <catalog>
  • aqua uninstall
  • aqua list
  • aqua update
  • aqua fixes {add,remove} <fixes-file>
  • aqua grids {add,remove} <grid-file>
  • aqua grids set <path>
  • aqua grids build
  • aqua drop -c <config_file> <drop-options>
  • aqua analysis <analysis-options>
• Regrid and interpolation capabilities
  • Basic usage within the Reader()
  • Basic usage of the Regridder()
  • Concept
  • Available target grids
  • Oceanic grid files naming scheme
    • Elements Description
    • Examples
  • Vertical interpolation
• Fixer functionalities
  • Concept
  • Convention file structure
  • Fixer file structure
  • AQUA variables convention
  • Metadata Correction
    • Merge of convention and fixer
    • Variables block structure
  • Data Model and Coordinates/Dimensions Correction
  • Develop your own fix
• Graphic tools
  • Plot styles
  • Single map
  • Single map with differences
  • Projections and custom maps
  • Vertical profiles
  • Vertical lines
  • Time series
  • Seasonal cycle
  • Multiple maps
  • Multiple maps with differences
• Other core functionalities
  • Time Statistics
  • Detrend
  • Spatial Selection
  • Spatial Statistics
  • Histogram
  • Time selection
  • Level selection
  • Streaming of data
    • Basic usage
  • Accessors
    • Basic usage
    • Usage with multiple Reader instances
  • Parallel Processing
  • Data provenance
• Container
  • Download the container image
  • Load the container
    • Load container script
  • Generate a Personal Access Token (PAT)
  • Advanced Topics
    • Running Jupyter Notebook
    • Running Jupyter Notebook within VSCode
    • Temporary Upgrade of Any Package
• State of the art diagnostics
  • List of diagnostics
    • Boxplots Diagnostic
      • Description
      • Classes
      • File structure
      • Input variables and datasets
      • Basic usage
      • CLI usage
      • Output
      • Example plots
      • Available demo notebooks
      • Detailed API
    • Global Biases Diagnostic
      • Description
      • Classes
      • File structure
      • Input variables and datasets
      • Basic usage
      • CLI usage
      • Config file structure
      • Output
      • Observations
      • Example Plots
      • Available demo notebooks
      • Authors and contributors
      • Detailed API
    • ECmean4 Performance Metrics
      • Description
      • Structure
      • Input variables
      • Output
      • Methods and functions used
      • Observations
      • References
      • Example Plot(s)
      • Available demo notebooks
    • Timeseries
      • Description
      • Classes
      • File structure
      • Input variables and datasets
      • CLI usage
      • Config file structure
      • Output
      • Observations
      • Example Plots
      • Available demo notebooks
      • Authors and contributors
      • Detailed API
    • Ocean3D
      • Description
      • Structure
      • Input variables example
      • Output
      • CLI (Command Line Interface)
      • Prepare the o3d request
      • Functions used
      • Methods used
      • Observations
      • References
      • Example Plot(s)
      • Available demo notebooks
      • Detailed API
    • Sea ice
      • Description
      • Classes
      • File structure
      • Input variables and datasets
      • Basic usage
      • CLI usage
      • Config file structure
      • CLI processing
      • Outputs
      • Example Plot(s)
      • Observations (Reference data)
      • Available demo notebooks
      • Authors and contributors
      • Detailed API
    • Teleconnections
      • Description
      • Classes
      • File structure
      • Input variables and datasets
      • CLI usage
      • Config file structure
      • Output
      • Example Plots
      • Available demo notebooks
      • Authors and contributors
      • Detailed API
    • Ensemble Statistics
      • Description
      • Structure
      • Input variables
      • Basic usage
      • Example Plots
      • Detailed API
  • Diagnostics configuration files
  • Diagnostics CLI arguments
  • Running the monitoring diagnostics
    • Minimum Data Requirements
• Frontier diagnostics
  • SSH variability
    • Description
    • Structure
    • Input variables
    • Output
    • Observations
    • Available demo notebooks
    • References
    • Example Plot(s)
    • Detailed API
  • Tropical Cyclones detection, tracking and zoom in diagnostic
    • Description
    • Structure
    • Input variables
    • Output
    • Methods used
    • Functions used
    • Observations
    • References
    • Example Plot(s)
    • Available demo notebooks
    • Detailed API
      • DetectNodes
      • StitchNodes
      • TCs
      • multi_plot()
      • plot_hist_cat()
      • plot_press_wind()
      • plot_trajectories()
  • Tropical rainfall diagnostic
    • Description
    • Structure
    • Input variables
    • Output
    • Examples
      • The histogram calculation
      • List of histograms
      • The histogram plots
      • Seasonal Mean Values
      • Bias between model and observations
    • Available demo notebooks
    • Detailed API
• Creating Custom Diagnostics
  • Core Diagnostic: Diagnostic
  • Diagnostic Classes
  • Comparison and Plot Classes
  • Command-Line Interface (CLI)
  • Configuration file
  • Configuration Files and AQUA console
  • Automatic Standardized File Naming
    • Class Overview
    • Attributes
    • Example Usage
      • Initializing the OutputSaver Class
      • Generating a Filename
      • Generating a Filename with Extra Keys
      • Saving a NetCDF File with Metadata
      • Saving a PDF or PNG Plot with Metadata
      • Opening a PDF File and Displaying Metadata
      • Generating a Filename for Multimodel or Multireference Comparisons
      • Creating a catalog entry
  • Core Functions
    • Diagnostic
      • Diagnostic.retrieve()
      • Diagnostic.save_netcdf()
      • Diagnostic.select_region()
    • OutputSaver
      • OutputSaver.create_metadata()
      • OutputSaver.generate_folder()
      • OutputSaver.generate_name()
      • OutputSaver.generate_path()
      • OutputSaver.save_netcdf()
      • OutputSaver.save_pdf()
      • OutputSaver.save_png()
      • OutputSaver.unpack_list()
    • close_cluster()
    • get_diagnostic_configpath()
    • load_diagnostic_config()
    • merge_config_args()
    • open_cluster()
    • round_enddate()
    • round_startdate()
    • start_end_dates()
    • template_parse_arguments()
• Adding new data
  • File-based sources
  • FDB-based sources
  • Experiment metadata
  • Regridding capabilities
  • Grid definitions
  • Compact catalogs with YAML override
  • Intake capabilities and kwargs data access
  • DE_340 source syntax convention
    • Models (model key)
    • Experiments (exp key)
    • Sources (source key)
• Catalog Generator
  • aqua catgen -c <config_file> -p <portfolio>
    • Basic usage
  • Configuration file
• DROP - Data Reduction OPerator
  • What is DROP?
  • DROP Capabilities
  • The Low Resolution Archive (LRA) Context
  • Source Naming Convention
  • Accessing DROP-generated data
  • Using DROP to process data
    • Usage
    • Parallel DROP tool
• AQUA analysis wrapper
  • Basic usage
  • Additional options
  • Configuration file
    • Job
    • Cluster
    • Diagnostics
• Dashboard
  • General description and main components
  • Automatic uploading of figures and documentation to aqua-web
    • Basic usage
    • Additional options
    • Returns
    • Grouping configuration file
    • AWS credentials file
  • Pushing to LUMI-O or another S3 bucket
    • Basic usage
    • Options
    • Returns
  • Multiple experiment analysis submitter
    • Basic usage
    • Options
• Examples and Tutorials
  • Jupyter Notebooks for Various Use Cases
• Command Line Interface tools
  • Benchmarker
  • Grids management
    • Grids downloader
    • Checksum verifications for obs and grids
    • Generation of grid from data
    • Grids synchronization
    • Grids uploader
    • ORCA grid generator
  • HPC container utilities
  • LUMI conda installation
  • Weights generator
  • Orography generator
• Advanced Topics
  • Set up the configuration file
  • Adding a new catalog
    • Creation of the catalog folder
    • Populating the catalog
    • Adding the catalog to the AQUA package
    • Access to aqua-dvc data for developers
    • Download of grids
  • Dask access to FDB or GSV
  • Polytope access to Destination Earth data
  • Level selection and regridding
  • Reader prepocessing option
  • Enable regrid capabilities in a new machine
    • Set the machine in the catalog machine file
    • Download the grids
  • Developer notes
    • Set up environment variables
    • Add new catalogs as developer
  • ecCodes
    • ecCodes fixer
• API Reference
  • Drop
    • Drop.append_history()
    • Drop.check_integrity()
    • Drop.create_catalog_entry()
    • Drop.create_zarr_entry()
    • Drop.dask
    • Drop.drop_generator()
    • Drop.get_filename()
    • Drop.retrieve()
    • Drop.write_chunk()
  • Fixer
    • Fixer.find_fixes()
    • Fixer.fixer()
    • Fixer.get_fixer_varname()
  • FldStat
    • FldStat.AVAILABLE_FLDSTATS
    • FldStat.align_area_coordinates()
    • FldStat.align_area_dimensions()
    • FldStat.fldstat()
    • FldStat.integrate_over_area()
    • FldStat.select_area()
    • FldStat.sum_area()
  • GridBuilder
    • GridBuilder.GRIDTYPE_REGISTRY
    • GridBuilder.build()
  • Reader
    • Reader.detrend()
    • Reader.fldarea()
    • Reader.fldintg()
    • Reader.fldmax()
    • Reader.fldmean()
    • Reader.fldmin()
    • Reader.fldstat()
    • Reader.fldstd()
    • Reader.fldsum()
    • Reader.histogram()
    • Reader.instance
    • Reader.reader_esm()
    • Reader.reader_fdb()
    • Reader.reader_intake()
    • Reader.regrid()
    • Reader.retrieve()
    • Reader.select_area()
    • Reader.set_default()
    • Reader.timhist()
    • Reader.timmax()
    • Reader.timmean()
    • Reader.timmin()
    • Reader.timstat()
    • Reader.timstd()
    • Reader.timsum()
    • Reader.vertinterp()
  • Regridder
    • Regridder.loglevel
    • Regridder.logger
    • Regridder.cfg_grid_dict
    • Regridder.src_grid_name
    • Regridder.handler
    • Regridder.src_grid_dict
    • Regridder.src_horizontal_dims
    • Regridder.src_vertical_dim
    • Regridder.tgt_horizontal_dims
    • Regridder.error
    • Regridder.cdo
    • Regridder.smmregridder
    • Regridder.src_grid_area
    • Regridder.tgt_grid_area
    • Regridder.masked_attrs
    • Regridder.masked_vars
    • Regridder.extra_dims
    • Regridder.areas()
    • Regridder.configure_masked_fields()
    • Regridder.regrid()
    • Regridder.weights()
  • Streaming
    • Streaming.reset()
    • Streaming.stream()
    • Streaming.stream_chunk()
  • catalog()
  • histogram()
  • inspect_catalog()
  • plot_hovmoller()
  • plot_lat_lon_profiles()
  • plot_maps()
  • plot_seasonal_lat_lon_profiles()
  • plot_single_map()
  • plot_single_map_diff()
  • plot_timeseries()
• Troubleshooting and FAQ
  • I am getting an error when trying to install AQUA on LUMI
    • Resource is busy
    • Pip dependency resolution
    • Permission denied
  • I am getting an error MissingGridDefinitionPathError
  • I am getting TypeError: ‘GeometryCollection’ object is not subscriptable when using plot_single_map function
  • How do I cite AQUA in my research or publications?
• Contributing to AQUA
  • Reporting Bugs
  • Suggesting Features
  • Contributing Code
  • Documentation and Tutorials
  • Testing
    • Running Tests Locally
    • Writing Tests
• References and Acknowledgments
  • Citing AQUA
  • Acknowledgments