Fallback Options: Using a Different Solver When the Previous One Failed

Note

This example focuses on how to implement a fallback option in RTC-Tools. It assumes basic exposure to RTC-Tools. If you are a first-time user of RTC-Tools, see Filling a Reservoir.

If a solver fails to find a solution, you can fall back to a different solver. The following example shows how this can be automated by overwriting the optimize and solver_options method of an OptimizationProblem class. After that, an example is given for how to include a fallback option when using goal programming.

Implementing a Basic Fallback Option

A fallback option to fall back to a different solver if the previous one failed, can be implemented in the following way:

  • Overwrite the optimize method of your optimization problem class, to loop over a list of solvers until one of them succeeds. The solver attribute is set at the start of each iteration.

  • Overwrite the solver_options method of your optimization problem class, to select the correct solver options for the current solver.

To iterate over a list of solvers, we can overwrite the optimize method:

57    def optimize(
58        self,
59        preprocessing: bool = True,
60        postprocessing: bool = True,
61        log_solver_failure_as_error: bool = True,
62    ) -> bool:
63        # Overwrite the optimize method to try different solvers if the previous solver fails.
64        if preprocessing:
65            self.pre()
66        solvers = ["ipopt", "highs"]
67        for solver in solvers:
68            self.solver = solver
69            success = super().optimize(
70                preprocessing=False,
71                postprocessing=False,
72                log_solver_failure_as_error=log_solver_failure_as_error,
73            )
74            logger.info(f"Finished running solver {solver} with success = {success}.")
75            if success:
76                break
77        if postprocessing:
78            self.post()
79        return success

Here, we iterate over the solvers ipopt and highs. The solver attribute is set at the start of each iteration. In case the solver succeeds, we break out of the loop.

To select the correct solver options based on the selected solver, we overwrite the solver_options method:

46    def solver_options(self):
47        options = super().solver_options()
48        if self.solver == "ipopt":
49            options["solver"] = "ipopt"
50        elif self.solver == "highs":
51            options["casadi_solver"] = "qpsol"
52            options["solver"] = "highs"
53        else:
54            raise ValueError(f"Solver should be 'ipopt' or 'highs', not {self.solver}.")
55        return options

The script of the entire example is as follows:

 1import logging
 2from pathlib import Path
 3
 4from rtctools.optimization.collocated_integrated_optimization_problem import (
 5    CollocatedIntegratedOptimizationProblem,
 6)
 7from rtctools.optimization.csv_mixin import CSVMixin
 8from rtctools.optimization.modelica_mixin import ModelicaMixin
 9from rtctools.optimization.optimization_problem import OptimizationProblem
10from rtctools.util import run_optimization_problem
11
12logger = logging.getLogger("rtctools")
13
14BASIC_EXAMPLE_FOLDER = Path(__file__).parents[2] / "basic"
15
16
17class DummySolver(OptimizationProblem):
18    """
19    Class for enforcing a solver result.
20
21    This class enforces a solver result
22    and is just used for illustrating how to implement a fallback option.
23    """
24
25    def optimize(
26        self,
27        preprocessing: bool = True,
28        postprocessing: bool = True,
29        log_solver_failure_as_error: bool = True,
30    ) -> bool:
31        # Call the optimize method and pretend it is only successful when using the 'highs' solver.
32        super().optimize(preprocessing, postprocessing, log_solver_failure_as_error)
33        solver = self.solver_options()["solver"]
34        success = solver == "highs"
35        return success
36
37
38class Example(CSVMixin, ModelicaMixin, CollocatedIntegratedOptimizationProblem, DummySolver):
39    """
40    An example of automatically switching to a different solver if the first attempt fails.
41
42    This class inherits from the DummySolver class to enforce a solver result,
43    which is only used to illustrate how the fallback can be implemented.
44    """
45
46    def solver_options(self):
47        options = super().solver_options()
48        if self.solver == "ipopt":
49            options["solver"] = "ipopt"
50        elif self.solver == "highs":
51            options["casadi_solver"] = "qpsol"
52            options["solver"] = "highs"
53        else:
54            raise ValueError(f"Solver should be 'ipopt' or 'highs', not {self.solver}.")
55        return options
56
57    def optimize(
58        self,
59        preprocessing: bool = True,
60        postprocessing: bool = True,
61        log_solver_failure_as_error: bool = True,
62    ) -> bool:
63        # Overwrite the optimize method to try different solvers if the previous solver fails.
64        if preprocessing:
65            self.pre()
66        solvers = ["ipopt", "highs"]
67        for solver in solvers:
68            self.solver = solver
69            success = super().optimize(
70                preprocessing=False,
71                postprocessing=False,
72                log_solver_failure_as_error=log_solver_failure_as_error,
73            )
74            logger.info(f"Finished running solver {solver} with success = {success}.")
75            if success:
76                break
77        if postprocessing:
78            self.post()
79        return success
80
81
82# Try solving the optimization problem.
83problem = run_optimization_problem(Example, base_folder=BASIC_EXAMPLE_FOLDER)

The DummySolver class forces the solver to only succeed if it is highs and is only added for illustration purposes.

Implementing a Fallback Option When Using Goal Programming

When using goal programming, a solver might fail for a specific priority and you might want to fall back to a different solver for just this priority. To implement this, we need the following:

  • Create MultiRunMixin class that inherits from OptimizationProblem and overwrites the optimize method to loop over a list of solvers until one of them succeeds.

  • Let your main optimization problem class also inherit from MultiRunMixin. It is important that MultiRunMixin comes after GoalProgrammingMixin in the inheritance list. This ensures that the optimize method of MultiRunMixin is called within the optimize method of GoalProgrammingMixin and thus that we loop over all solvers for each priority.

  • Overwrite the solver_options method of your main optimization problem class, to select the correct solver options for the current solver.

The MultiRunMixin class, can look something like this:

39class MultiRunMixin(OptimizationProblem):
40    """
41    Enables a workflow to solve an optimization problem with multiple attempts.
42    """
43
44    def optimize(
45        self,
46        preprocessing: bool = True,
47        postprocessing: bool = True,
48        log_solver_failure_as_error: bool = True,
49    ) -> bool:
50        # Overwrite the optimize method to try different solvers if the previous solver fails.
51        if preprocessing:
52            self.pre()
53        solvers = ["ipopt", "highs"]
54        for solver in solvers:
55            self.solver = solver
56            success = super().optimize(
57                preprocessing=False,
58                postprocessing=False,
59                log_solver_failure_as_error=log_solver_failure_as_error,
60            )
61            logger.info(f"Finished running solver {solver} with success = {success}.")
62            if success:
63                break
64        if postprocessing:
65            self.post()
66        return success

It has an attribute solver that keeps track of the current solver. During optimization, it iterates over the solvers ipopt and highs and breaks out of the loop in case a solver succeeds.

The main optimization problem class inherits from GoalProgrammingMixin and MultiRunMixin:

80class Example(
81    GoalProgrammingMixin,
82    CSVMixin,
83    ModelicaMixin,
84    CollocatedIntegratedOptimizationProblem,
85    MultiRunMixin,
86    DummySolver,
87):
88    # initially the solver is set to None
89    solver = None

It is important that MultiRunMixin comes after GoalProgrammingMixin so that the optimize method of MultiRunMixin is called within the optimize method of GoalProgrammingMixin.

As before, the main optimization problem class also overwrites the solver_options method to select the correct solver options:

103    def solver_options(self):
104        options = super().solver_options()
105        if self.solver == "ipopt":
106            options["solver"] = "ipopt"
107        elif self.solver == "highs":
108            options["casadi_solver"] = "qpsol"
109            options["solver"] = "highs"
110            options["highs"] = {"time_limit": 1}
111        elif self.solver:
112            raise ValueError(f"Solver should be 'ipopt' or 'highs', not {self.solver}.")
113        return options

The script of the entire example is as follows:

  1import logging
  2from pathlib import Path
  3
  4from rtctools.optimization.collocated_integrated_optimization_problem import (
  5    CollocatedIntegratedOptimizationProblem,
  6)
  7from rtctools.optimization.csv_mixin import CSVMixin
  8from rtctools.optimization.goal_programming_mixin import Goal, GoalProgrammingMixin
  9from rtctools.optimization.modelica_mixin import ModelicaMixin
 10from rtctools.optimization.optimization_problem import OptimizationProblem
 11from rtctools.util import run_optimization_problem
 12
 13logger = logging.getLogger("rtctools")
 14
 15BASIC_EXAMPLE_FOLDER = Path(__file__).parents[2] / "basic"
 16
 17
 18class DummySolver(OptimizationProblem):
 19    """
 20    Class for enforcing a solver result.
 21
 22    This class enforces a solver result
 23    and is just used for illustrating how to implement a fallback option.
 24    """
 25
 26    def optimize(
 27        self,
 28        preprocessing: bool = True,
 29        postprocessing: bool = True,
 30        log_solver_failure_as_error: bool = True,
 31    ) -> bool:
 32        # Call the optimize method and pretend it is only successful when using the 'highs' solver.
 33        super().optimize(preprocessing, postprocessing, log_solver_failure_as_error)
 34        solver = self.solver_options()["solver"]
 35        success = solver == "highs"
 36        return success
 37
 38
 39class MultiRunMixin(OptimizationProblem):
 40    """
 41    Enables a workflow to solve an optimization problem with multiple attempts.
 42    """
 43
 44    def optimize(
 45        self,
 46        preprocessing: bool = True,
 47        postprocessing: bool = True,
 48        log_solver_failure_as_error: bool = True,
 49    ) -> bool:
 50        # Overwrite the optimize method to try different solvers if the previous solver fails.
 51        if preprocessing:
 52            self.pre()
 53        solvers = ["ipopt", "highs"]
 54        for solver in solvers:
 55            self.solver = solver
 56            success = super().optimize(
 57                preprocessing=False,
 58                postprocessing=False,
 59                log_solver_failure_as_error=log_solver_failure_as_error,
 60            )
 61            logger.info(f"Finished running solver {solver} with success = {success}.")
 62            if success:
 63                break
 64        if postprocessing:
 65            self.post()
 66        return success
 67
 68
 69class DummyGoal(Goal):
 70    """Goal to illustrate the fallback option during goal programming."""
 71
 72    def __init__(self, priority):
 73        super().__init__()
 74        self.priority = priority
 75
 76    def function(self, optimization_problem, ensemble_member):
 77        return optimization_problem.integral("Q_release")
 78
 79
 80class Example(
 81    GoalProgrammingMixin,
 82    CSVMixin,
 83    ModelicaMixin,
 84    CollocatedIntegratedOptimizationProblem,
 85    MultiRunMixin,
 86    DummySolver,
 87):
 88    # initially the solver is set to None
 89    solver = None
 90    """
 91    An example of automatically switching to a different solver during goal programming.
 92
 93    This class inherits from the DummySolver class to enforce a solver result,
 94    which is only used to illustrate how the fallback can be implemented.
 95
 96    The MultiRunMixin class should be inherited after GoalProgrammingMixin.
 97    This way, the solver loop is applied for each priority during goal programming.
 98    """
 99
100    def goals(self):
101        return [DummyGoal(priority=prio) for prio in range(3)]
102
103    def solver_options(self):
104        options = super().solver_options()
105        if self.solver == "ipopt":
106            options["solver"] = "ipopt"
107        elif self.solver == "highs":
108            options["casadi_solver"] = "qpsol"
109            options["solver"] = "highs"
110            options["highs"] = {"time_limit": 1}
111        elif self.solver:
112            raise ValueError(f"Solver should be 'ipopt' or 'highs', not {self.solver}.")
113        return options
114
115
116# Try solving the optimization problem.
117problem = run_optimization_problem(Example, base_folder=BASIC_EXAMPLE_FOLDER)

The DummySolver class forces the solver to only succeed if it is highs and is only added for illustration purposes. The DummyGoal is also only added for illustration purposes.