Optuna Optimization

VISWIR integrates Optuna to automatically optimize fusion parameters. This page explains how to configure, run, and analyze Optuna experiments.

Configuration

Three configuration files are required:

  • config_viswir.yaml : general control parameters, with mode: "optuna".

  • optuna_config.yaml : optimization settings (trials, jobs, sampler, pruner).

  • optuna_search_space.yaml : definition of the hyperparameter ranges.

Example optuna_config.yaml:

# Configuration spécifique à Optuna
n_trials: 2
n_jobs: 1
sampler: "TPE"          # or "Random", "CMA-ES"…
pruner: "MedianPruner"  # or "SuccessiveHalving"
timeout: 7200           # in seconds
storage: null           # e.g. "sqlite:///results/optuna.db"
sample_size: 50

Example optuna_search_space.yaml:

# Définition de l’espace de recherche pour Optuna
facteur_swir:
  type: float
  min: 0.0
  max: 1.0
  step: 0.01

beta:
  type: float
  min: 1.0
  max: 5.0
  step: 0.01

level:
  type: int
  min: 1
  max: 6
  step: 1

apply_gamma:
  type: categorical
  values: [True]

gamma_value:
  type: float
  min: 0.01
  max: 4.0
  step: 0.01

Example config_viswir.yaml (pilotage):

visible_folder: "../data/VIS"
swir_folder: "../data/SWIR"
output_folder: "../results/optim"
ref_image_path: null
ground_truth_path: "../data/ground_truth"

mode: "optuna"

run_detection: false # true or false
save_output: false # always true for optimization

Important

You do not need include fusion parameters in parameters.json when using Optuna. They are defined and explored via optuna_search_space.yaml.

Execution

Run VISWIR in Optuna mode:

python src/VISWIR_vQuasar.py

Each trial explores a different set of parameters. Results are stored in results/optuna_study.sqlite3 (SQLite).

Tip

Start with a small number of trials (10-20) to validate setup, then scale up for production runs.

Note

Once optimisation is complete, the best parameters are stored in result/best_parameters.json.

Resource Considerations

  • Workers (n_jobs): increasing jobs consumes more CPU/RAM.

  • Detection optimization: not recommended without a powerful machine (risk of crash or extremely long runtimes).

  • Batch size & image size: strongly influence optimization time.

Warning

Running detection optimization on limited hardware can lead to memory exhaustion or absurdly long computation times.

Results and Visualization

  • Trials and metrics are saved in results/optuna_study.sqlite3.

  • The best parameters are stored in result/best_parameters.json.

  • Use tools/optuna_sqlite3_loader_mono_obj.py or tools/optuna_sqlite3_loader_multi_obj.py to explore results.

  • On local machines, graphs are displayed in the browser at the end of optimization.

  • The VS Code extension Optuna Dashboard can be used to visualize trials in real time.

Note

Visualization requires a local environment with a browser. On HPC clusters, results must be exported and analyzed offline.

Advanced Usage

Optuna offers several advanced features that can be leveraged in VISWIR:

  • Multi-objective optimization Instead of optimizing a single metric, Optuna can handle multiple objectives (e.g. maximizing Entropy while minimizing BRISQUE, NIQE and PIQE). Results can be explored with tools/optuna_sqlite3_loader_multi_obj.py and visualized as a Pareto front.

    Important

    To activate multi-objective optimization, simply disable detection in config_viswir.yaml. The optimization will then be performed on NR-IQA metrics instead of the F1 score. This allows Optuna to balance several image quality metrics simultaneously.

  • Custom samplers By default, the TPE sampler is used. You can switch to other samplers such as Random, CMA-ES, or GridSearch by editing optuna_config.yaml. Each sampler has trade-offs: - TPE: efficient for continuous spaces. - Random: baseline, useful for debugging. - CMA-ES: good for continuous, smooth search spaces.

  • Pruners Pruners stop unpromising trials early to save resources. Common options: - MedianPruner: stops trials worse than the median (default). - SuccessiveHalving: aggressively prunes based on performance tiers. Configure in optuna_config.yaml.

  • Custom metrics You can extend VISWIR to optimize on custom metrics (e.g. perceptual quality, task-specific accuracy). This requires modifying the evaluation/objective function inside the pipeline.

  • Storage backends Optuna supports different storage backends for results: - SQLite (default, local file). - MySQL/PostgreSQL (for collaborative or large-scale experiments). Configure via the storage field in optuna_config.yaml.

Tip

Use multi-objective optimisation when you want to optimise the quality of the output images independently of the detection scores.

Best Practices

  • Start small Begin with a limited number of trials (10-20) to validate your setup before scaling up.

  • Control resources The number of workers (n_jobs) directly impacts CPU and memory usage. Avoid setting it too high on limited hardware like laptop.

  • Detection optimization caution Running Optuna with detection enabled requires significant memory and compute. On modest machines, this can lead to crashes or extremely long runtimes.

  • Batch size & image size Larger batches and high-resolution images drastically increase optimization time. Adjust these parameters carefully.

  • Crash and restart If optimisation stops midway, simply restart it. As long as the *.sqlite3 file is still in the output folder, VISWIR and Optuna can pick up where they stopped.

  • Use pruners Enable pruners to stop unpromising trials early and save resources.

  • Visualize progress On local runs, Optuna automatically opens graphs in the browser at the end of optimization. For real-time monitoring, the VS Code extension Optuna Dashboard is recommended but not mendatory.

Warning

Misconfigured search spaces (too wide or unrealistic ranges) can lead to wasted trials and excessive runtimes. Always define sensible bounds.