From 3903d75f853cfea1736879f4b41ea7c3f6624971 Mon Sep 17 00:00:00 2001
From: Yash Raj Singh <yash.raj270998@gmail.com>
Date: Wed, 16 Aug 2023 00:52:48 +0200
Subject: [PATCH 1/2] added rad upt scheme docs
---
docs/src/api/nonlinearsolve.md | 10 ++++++++
src/trustRegion.jl | 44 ++++++++++++++++++++++++++++++++++
2 files changed, 54 insertions(+)
diff --git a/docs/src/api/nonlinearsolve.md b/docs/src/api/nonlinearsolve.md
index ebe9c7203..a8d3a6f2c 100644
--- a/docs/src/api/nonlinearsolve.md
+++ b/docs/src/api/nonlinearsolve.md
@@ -8,3 +8,13 @@ These are the native solvers of NonlinearSolve.jl.
NewtonRaphson
TrustRegion
```
+
+### Radius Update Schemes for Trust Region (RadiusUpdateSchemes)
+
+```@docs
+RadiusUpdateSchemes.Simple
+RadiusUpdateSchemes.Hei
+RadiusUpdateSchemes.Yuan
+RadiusUpdateSchemes.Bastin
+RadiusUpdateSchemes.Fan
+```
diff --git a/src/trustRegion.jl b/src/trustRegion.jl
index 828eeb9b1..c1ab1aab6 100644
--- a/src/trustRegion.jl
+++ b/src/trustRegion.jl
@@ -1,8 +1,52 @@
EnumX.@enumx RadiusUpdateSchemes begin
+ """
+ `RadiusUpdateSchemes.Simple`
+
+ The simple or conventional radius update scheme. This scheme is chosen by default
+ and follows the conventional approach to update the trust region radius, i.e. if the
+ trial step is accepted it increases the radius by a fixed factor (bounded by a maximum radius)
+ and if the trial step is rejected, it shrinks the radius by a fixed factor.
+ """
Simple
+
+ """
+ `RadiusUpdateSchemes.Hei`
+
+ This scheme is proposed by [Hei, L.] (https://www.jstor.org/stable/43693061). The trust region radius
+ depends on the size (norm) of the current step size. The hypothesis is to let the radius converge to zero
+ as the iterations progress, which is more reliable and robust for ill-conditioned as well as degenerate
+ problems.
+ """
Hei
+
+ """
+ `RadiusUpdateSchemes.Yuan`
+
+ This scheme is proposed by [Yuan, Y.] (https://www.researchgate.net/publication/249011466_A_new_trust_region_algorithm_with_trust_region_radius_converging_to_zero).
+ Similar to Hei's scheme, the trust region is updated in a way so that it converges to zero, however here,
+ the radius depends on the size (norm) of the current gradient of the objective (merit) function. The hypothesis
+ is that the step size is bounded by the gradient size, so it makes sense to let the radius depend on the gradient.
+ """
Yuan
+
+ """
+ `RadiusUpdateSchemes.Bastin`
+
+ This scheme is proposed by [Bastin, et al.] (https://www.researchgate.net/publication/225100660_A_retrospective_trust-region_method_for_unconstrained_optimization).
+ The scheme is called a retrospective update scheme as it uses the model function at the current
+ iteration to compute the ratio of the actual reduction and the predicted reduction in the previous
+ trial step, and use this ratio to update the trust region radius. The hypothesis is to exploit the information
+ made available during the optimization process in order to vary the accuracy of the objective function computation.
+ """
Bastin
+
+ """
+ `RadiusUpdateSchemes.Fan`
+
+ This scheme is proposed by [Fan, J.] (https://link.springer.com/article/10.1007/s10589-005-3078-8). It is very much similar to
+ Hei's and Yuan's schemes as it lets the trust region radius depend on the current size (norm) of the objective (merit)
+ function itself. These new update schemes are known to improve local convergence.
+ """
Fan
end
From 24fcd98dfe5a4b0283a4b9899ea25710a64c32ae Mon Sep 17 00:00:00 2001
From: Yash Raj Singh <yash.raj270998@gmail.com>
Date: Thu, 17 Aug 2023 01:53:53 +0200
Subject: [PATCH 2/2] fixes
---
docs/src/api/nonlinearsolve.md | 8 +++++++-
src/trustRegion.jl | 16 ++++++++++++++++
2 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/docs/src/api/nonlinearsolve.md b/docs/src/api/nonlinearsolve.md
index a8d3a6f2c..ff61c4813 100644
--- a/docs/src/api/nonlinearsolve.md
+++ b/docs/src/api/nonlinearsolve.md
@@ -9,7 +9,13 @@ NewtonRaphson
TrustRegion
```
-### Radius Update Schemes for Trust Region (RadiusUpdateSchemes)
+## Radius Update Schemes for Trust Region (RadiusUpdateSchemes)
+
+```@docs
+RadiusUpdateSchemes
+```
+
+### Available Radius Update Schemes
```@docs
RadiusUpdateSchemes.Simple
diff --git a/src/trustRegion.jl b/src/trustRegion.jl
index 026d49219..d070bdfc8 100644
--- a/src/trustRegion.jl
+++ b/src/trustRegion.jl
@@ -1,3 +1,19 @@
+"""
+`RadiusUpdateSchemes`
+
+`RadiusUpdateSchemes` is the standard enum interface for different types of radius update schemes
+implemented in the Trust Region method. These schemes specify how the radius of the so-called trust region
+is updated after each iteration of the algorithm. The specific role and caveats associated with each
+scheme are provided below.
+
+## Using `RadiusUpdateSchemes`
+
+`RadiusUpdateSchemes` uses the standard EnumX interface (https://github.com/fredrikekre/EnumX.jl),
+and hence inherits all properties of being an EnumX, including the type of each constituent enum
+states as `RadiusUpdateSchemes.T`. Simply put the desired scheme as follows:
+`TrustRegion(radius_update_scheme = your desired update scheme)`. For example,
+`sol = solve(prob, alg=TrustRegion(radius_update_scheme = RadiusUpdateSchemes.Hei))`.
+"""
EnumX.@enumx RadiusUpdateSchemes begin
"""
`RadiusUpdateSchemes.Simple`