Vector variable - Matrix variable - Symmetric matrix variable - Perspective functions - Divergences
pip install proxop
from proxop import BoxConstraint
import numpy as np
x = np.array([-5, 1, 10., 0, 3])
BoxConstraint(-2., 2.).prox(x)
# result: array([-2., 1., 2., 0., 2.])
from proxop import AbsValue
x = np.array([ -3., 1., 6.])
AbsValue().prox(x, gamma=2)
#result: array([-1., 0., 4.])
Description |
f(x) (∀ x ∈ ℝN) |
proxγf(x) (∀ γ ∈ ℝ+) |
Matlab | Python | Ref |
---|---|---|---|---|---|
Euclidean norm | $\|x\|_2 = \displaystyle\sqrt{\sum_{n=1}^N x_n^2}$ | $\left(1 - \dfrac{\gamma}{\max\big\{\|x\|_2,\gamma\big\}}\right)x$ |
[Function]
[Prox] |
[L2Norm] | [Combettes et al., 2005] |
Infinity norm | $\|x\|_\infty = \displaystyle\max_{1\le n\le N} |x_n|$ |
$\Big(\operatorname{sign}(x_n)\min\{|x_n|,s\}\Big)_{1\le n\le N}$
where $s\in\mathbb{R}$ is such that $\displaystyle\sum_{n=1}^N \max\{0, |x_n| - s\} = \gamma$ |
[Function]
[Prox] |
[Linf] | |
Maximum | $\displaystyle\max_{1\le n\le N} x_n$ |
$\Big(\min\{x_n,s\}\Big)_{1\le n\le N}$
where $s\in\mathbb{R}$ is such that $\displaystyle\sum_{n=1}^N \max\{0, x_n - s\} = \gamma$ |
[Function]
[Prox] |
[Max] | |
L1Norm | $\|x\|_1 = \sum_{k=1}^{N}|x_k|$ |
$\Big(\operatorname{soft}_{[-\gamma,\gamma]}(x_k)\Big)_{1\le n\le N}$
where $\operatorname{soft}_{[-\gamma,\gamma ]}(x_k) = \left\{\begin{aligned} &x+\gamma &&\textrm{if $x < -\gamma$}\\&0 &&\textrm{if $-\gamma\le x \le \gamma$}\\ &x-\gamma &&\textrm{otherwise}\end{aligned}\right.$ |
[Function]
[Prox] |
[L1Norm] | |
Vapnik |
$\displaystyle\max\big\{\|x\|_2-\varepsilon,0\big\}$
(with $\varepsilon>0$) |
$\left\{\begin{aligned} &x &&\textrm{if $\|x\|_2 - \varepsilon\le 0$}\\ &\frac{\varepsilon}{\|x\|_2}x &&\textrm{if $0<\|x\|_2-\varepsilon\le\gamma$}\\ &\Big(1-\frac{\gamma}{\|x\|_2}\Big) x &&\textrm{otherwise}\\ \end{aligned}\right.$ |
[Function]
[Prox] |
[Vapnik] | [Combettes et al., 2011] |
Euclidean norm + positivity | $\|x\|_2 + \iota_{[0,+\infty)^N}(x)$ | $\left(1 - \dfrac{\gamma}{\max\big\{\|\max(x,0)\|_2,\gamma\big\}}\right)\max(x,0)$ |
[Function]
[Prox] |
[L2Positive] | |
Barrier of affine constraints |
$\left\{\begin{aligned} &-\log(b-a^{\top} x) &&\textrm{if $a^{\top} x < b$}\\&+ \infty &&\textrm{otherwise }\end{aligned}\right.$
(with $b \in \mathbb{R}$ and $a \in \mathbb{R}^N \setminus \{0\}$) |
$x+\frac{b-a^{\top}x-\sqrt{(b-a^{\top}x)^2+4\gamma\|a\|^2}}{2\|a\|^2} a$ |
[Function]
[Prox] |
[AffineBarrier] | [Bertocchi et al., 2018] |
Barrier of hyperslab constraints |
$\left\{\begin{aligned} &-\log(b_{M}a^{\top} x) -\log(a^{\top} x-b_{m})&&
\textrm{if $b_{m} < a^{\top} x < b_{M}$}\\&+ \infty &&\textrm{otherwise }\end{aligned}\right.$
(with $b_m \in \mathbb{R}$, $b_M \in \mathbb{R}$, $b_m < b_M$, and $a \in \mathbb{R}^N \setminus \{0\}$) |
$x+\frac{\kappa(x,\gamma)-a^{\top}x}{\|a\|^2} a$
with $\kappa(x,\gamma)$ unique solution in $]b_{m},b_{M}[$ of: $0=z^3 -(b_{m}+b_{M}+ a^{\top}x) z^2 +(b_{m}b_{M} +a^{\top} x(b_{m}+b_{M})-2\gamma \|a\|^2)z -b_{m}b_{M}a^{\top}x+\gamma (b_{m}+b_{M})||a||^2$ |
[Function]
[Prox] |
[HyperslabBarrier] | [Bertocchi et al., 2018] |
Barrier of $\ell_2$ ball constraint |
$\left\{\begin{aligned} &-\log(\alpha-\|x-c\|^2)&&\textrm{if $\|x-c\|^2< \alpha$}\\&+ \infty &&\textrm{otherwise }\end{aligned}\right.$
(with $\alpha > 0$ and $c \in \mathbb{R}^N$) |
$c+\frac{\alpha -\kappa(x,\gamma)^2}{\alpha -\kappa(x,\gamma)^2+2\gamma} (x-c)$
with $\kappa(x,\gamma)$ unique solution in $[0,\sqrt{\alpha}[$ of: $0=z^3 -\|x-c\| z^2 -(\alpha+2\gamma)z +\alpha \|x-c\|$ |
[Function]
[Prox] |
[L2BallBarrier] | [Bertocchi et al., 2018] |
Softmax activation |
$$f(x)=\phi(x+u)+u$$
where $ \phi(\xi)=\left\{\begin{aligned} & \sum_{k=1}^{N}\left(\xi_k ln(\xi_k) -\frac{\xi_k^2}{2}\right) &&\textrm{if $(\xi_k)_{(1 \le k \le N)} \in [0,1]^N$ and $\sum_{k=1}^{N}\xi_k = 1$}\\&+ \infty &&\textrm{otherwise }\end{aligned}\right.$ and $u=(1,1,...,1)^T \in \mathbb{R}^N$ |
If $\gamma = 1$: $\left( \frac{ exp( \xi_k )}{\sum_{j=1}^{N}exp(\xi_j) }\right)_{(1 \le k \le N)}-u $ where $u=(1,1,...,1)^T \in \mathbb{R}^N$ |
[coming soon] | [SoftmaxActi] | [Combettes and Pesquet, 2018] |
Squashing |
$\left\{\begin{aligned}
&\mu \textrm{arctan}\sqrt{\frac{\|x\|}{\mu-\|x\|}}-\sqrt{\|x\|(\mu-\|x\|)}-\frac{\|x\|^2}{2} &&\textrm{if $\|x\|< \mu$}\\
& \frac{\mu(\pi - \mu)}{2} &&\textrm{if $\|x\|= \mu$}\\
&+\infty &&\textrm{otherwise}
\end{aligned}\right.$
with $\mu = \frac{8}{3 \sqrt{3}}$ |
If $\gamma = 1$: $\frac{\mu \|x\|}{1 + \|x\|^2} x$ |
(coming soon) | [Squashing] | [P.L. Combettes and J.-C. Pesquet 2020] |
Least squares downsampled convolution |
$\frac12 \|\mathbf{S}_m\mathbf{Hx - y} \|_2^2 $
(with $\mathbf{S}_m$ downsampling operator with factor $m$ in each dimension, $\mathbf{H}= \mathbf{F}^\ast \mathbf{\Lambda} \mathbf{F}$ convolution operator, $\mathbf{F}$ discrete Fourier transform, $\mathbf{\Lambda}$ diagonal) |
$ \mathbf{F}^\ast \left(\mathbf{I} - \frac{\gamma}{m^d} \mathbf{\Lambda}^\ast \mathbf{P} \mathbf{D}^{-1} \mathbf{P}^\ast \mathbf{\Lambda} \right) \mathbf{F } \left(\mathbf{x} + \gamma \mathbf{H}^\ast \mathbf{S}_m^\ast \mathbf{y} \right) $
with $\mathbf{P}$ is a periodization operator with $m$ repetitions in each dimension, $d$ is the number of dimensions of $\mathbf{x}$, $\mathbf{D}= \mathbf{I} + \frac{\gamma}{m^d} \mathrm{diag}(\mathbf{P}^\ast \mathbf{\Lambda \Lambda}^\ast \mathbf{1})$ |
[Function]
[Prox] |
[L2DownConv] | [Zhao et al, 2016] [Soubies et al, 2019] |
Description |
f(X) = φ(s)
(∀ X = U Diag(s) V⊤∈ ℝM × N) |
proxγf(X) (∀ γ ∈ ℝ+) |
Matlab | Python | Ref |
---|---|---|---|---|---|
Nuclear norm | $\displaystyle\|X\|_N = \|s\|_1$ | $U\operatorname{Diag}(d)V^\top$ with $d = \operatorname{prox}_{\gamma\|\cdot\|_1}(s) = \operatorname{soft}_{[-\gamma,\gamma]}(s)$ |
[Function]
[Prox] |
[NuclearNorm] | [Bauschke et al., 2017] |
Nuclear norm constraint | $\displaystyle \iota_{\|X\|_N \le \eta}(X) = \iota_{\|s\|_1 \le \eta}(s)$ | $U\operatorname{Diag}(d)V^\top$ with $d = s - \operatorname{prox}_{\eta\|\cdot\|_\infty}(s)$ |
[Function]
[Prox] |
[NuclearBall] | [Bauschke et al., 2017] |
Block nuclear norm | $\displaystyle\sum_{i= 1}^N\displaystyle\sum_{j=1}^M w_{i,j} \|X_{i,j}\|_N = \displaystyle\sum_{i= 1}^N\displaystyle\sum_{j=1}^M w_{i,j} \|s_{i,j}\|_1$ with each $w_{i,j} \ge 0$ acting on a block $X_{i,j}$ of matrix $X$ | ($U_{i,j}\operatorname{Diag}(d_{i,j})V_{i,j}^\top$$)_{i,j}$ with $d_{i,j} = \operatorname{prox}_{\gamma w_{i,j}\|\cdot\|_1}(s_{i,j}) = \operatorname{soft}_{[-\gamma w_{i,j},\gamma w_{i,j}]}(s_{i,j})$ |
[Function]
[Prox] |
[NuclearBlocks] | [Bauschke et al., 2017] |
Spectral norm | $\displaystyle\|X\|_S = \|s\|_\infty$ | $U\operatorname{Diag}\Big(\operatorname{prox}_{\gamma\|\cdot\|_\infty}(s)\Big)V^\top$ |
[Function]
[Prox] |
[Spectral] | [Bauschke et al., 2017] |
Rank | $\displaystyle\operatorname{rank}(X) = \|s\|_0$ | $U\operatorname{Diag}(d)V^\top$ with $d = \operatorname{prox}_{\gamma\|\cdot\|_0}(s) = \operatorname{hard}_{\sqrt{2 \gamma}}(s)$ | (coming soon) | [Rank] | [Benfenati et al., 2018] |
L21 norm (columnwise) | $\displaystyle\sum_{j= 1}^M|\displaystyle\sum_{i=1}^N|X_{i,j}|^2|^{\frac{1}{2}}$ with $X_{i,j} \in \mathbb{R}$ the entry $(i,j)$ of matrix $X$ | $\left((1- \frac{\gamma}{\max(\|X_j\|_2,\gamma)})X_j\right)_{1 \le j \le M}$ with $X_j \in \mathbb{R}^M$ the $j$-th column of $X$ |
[Function]
[Prox] |
[L21columns] | [Gramfort et al., 2012] |
L21 norm (rowwise) | $\displaystyle\sum_{i=1}^N|\displaystyle\sum_{j=1}^M|X_{i,j}|^2|^{\frac{1}{2}}$ with $X_{i,j} \in \mathbb{R}$ the entry $(i,j)$ of matrix $X$ | $\left((1- \frac{\gamma}{\max(\|X_i\|_2,\gamma)})X_i\right)_{1 \le i \le N}$ with $X_i \in \mathbb{R}^N$ the $i$-th row of $X$ |
[Function]
[Prox] |
[L21rows] | [Gramfort et al., 2012] |
L$\infty$1 norm | $\displaystyle\sum_{i=1}^N \displaystyle\sup_{1\le j \le M} |X_{i,j}|$ with $X_{i,j} \in \mathbb{R}$ the entry $(i,j)$ of matrix $X$ | $\Big(\Big(\operatorname{sign}(X_{i,j})\min\{|X_{i,j}|,s_i\}\Big)_{1\le j\le M}\Big)_{1\le i\le N}$
where $s_i\in\mathbb{R}$ is such that $\displaystyle\sum_{j=1}^M \max\{0, |X_{i,j}| - s_i\} = \gamma$ |
[Function]
[Prox] |
[Linf1] | [Gramfort et al., 2012] |
Nuclear norm + Ridge | $\frac{1}{2}\|X\|_F ^2 +\mu \|X\|_N = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_1$ with $\mu \geq 0$ | $U\operatorname{Diag}(d)V^\top$ with $d =\operatorname{soft}_{[-\frac{\mu \gamma}{\gamma+1},\frac{\mu \gamma}{\gamma+1}]}(\frac{s}{1+\gamma})$ |
[Function]
[Prox] |
[NuclearNormRidge] | [Bauschke et al., 2017] |
Rank + Ridge | $\frac{1}{2}\|X\|_F ^2 +\mu \operatorname{rank}(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_0$ with $\mu \geq 0$ | $U\operatorname{Diag}(d)V^\top$ with $d =\operatorname{hard}_{\sqrt{\frac{2 \mu \gamma}{1 + \gamma}}}(\frac{s}{1+\gamma})$ |
(coming soon) | [RankRidge] | [Benfenati et al., 2018] |
Schatten 3-penalty + Ridge | $\frac{1}{2}\|X\|_F ^2 +\mu \mathcal{R}_3^3(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_3^3$ with $\mu \geq 0$ |
$U\operatorname{Diag}(d)V^\top$ with $d =(6\gamma\mu)^{-1}\Bigg(\operatorname{sign}(s)\sqrt{(\gamma+1)^2+12|s|\gamma\mu}-\gamma-1\Bigg)$ |
[Function]
[Prox] |
[Schatten3Penalty] | [Bauschke et al., 2017] |
Schatten 4-penalty + Ridge | $\frac{1}{2}\|X\|_F ^2 +\mu \mathcal{R}_4^4(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_4^4$ with $\mu \geq 0$ |
$U\operatorname{Diag}(d)V^\top$ with $d =(8\gamma\mu)^{-\frac{1}{3}}\Bigg(\sqrt[3]{s+\sqrt{s^2+\zeta}}-\sqrt[3]{\sqrt{s^2+\zeta}-s}\Bigg)$And $\zeta=\frac{(\gamma+1)^3}{27\gamma\mu}$ |
[Function]
[Prox] |
[Schatten4Penalty] | [Bauschke et al., 2017] |
Schatten 4/3-penalty + Ridge | $\frac{1}{2}\|X\|_F ^2 +\mu \mathcal{R}_{4/3}^{4/3}(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_{4/3}^{4/3}$ with $\mu \geq 0$ |
$U\operatorname{Diag}(d)V^\top$ with $d = \frac{1}{1+\gamma}\Bigg(s+\frac{4\gamma\mu}{3\sqrt[3]{2(1+\gamma)}}\Bigg(\sqrt[3]{\sqrt{s^2+\zeta}-s}-\sqrt[3]{s+\sqrt{s^2+\zeta}}\Bigg)\Bigg)$And $\zeta=\frac{256(\gamma\mu)^3}{729(1+\gamma)}$ |
[Function]
[Prox] |
[Schatten4_3Penalty] | [Bauschke et al., 2017] |
Schatten 3/2-penalty + Ridge | $\frac{1}{2}\|X\|_F ^2 +\mu \mathcal{R}_{3/2}^{3/2}(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_{3/2}^{3/2}$ with $\mu \geq 0$ |
$U\operatorname{Diag}(d)V^\top$ with $d =\frac{1}{1+\gamma}\Bigg(s+\frac{9\gamma^2\mu^2}{8(1+\gamma)}\operatorname{sign}(s)\Bigg(1-\sqrt{1+\frac{16(1+\gamma)}{9\gamma^2\mu^2}|s|}\Bigg)\Bigg)$ |
[Function]
[Prox] |
[Schatten3_2Penalty] | [Bauschke et al., 2017] |
Description |
f(X) = φ(s)
(∀ X = U Diag(s) U⊤∈ ℝN × N symmetric) |
proxγf(X) (∀ γ ∈ ℝ+) |
Matlab | Python | Ref |
---|---|---|---|---|---|
Permutation invariant | $\varphi(s)$ | $U\operatorname{Diag}\Big(\operatorname{prox}_{\gamma\varphi}(s)\Big)U^\top$ |
[Function]
[Prox] |
[PermutationInvariant] | [Bauschke et al., 2017] |
logdet |
$f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ | $U\operatorname{Diag}(d)U^\top$ with $d =\frac{1}{2} \Bigg( s+\sqrt{s^2+4\gamma}\Bigg)$ |
[Function]
[Prox] |
[LogDet] | [Bauschke et al., 2017] |
Nuclear norm + logdet |
$f(X) +\mu \|X\|_N $ with $f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ and $\mu \geq 0$ | $U\operatorname{Diag}(d)U^\top$ with $d =\frac{1}{2} \Bigg( s-\gamma\mu+\sqrt{(s-\gamma\mu)^2+4\gamma}\Bigg)$ |
[Function]
[Prox] |
[NuclearLogDet] | [Bauschke et al., 2017] |
Squared Frobenius norm + logdet |
$f(X) +\mu \|X\|_F^2$ with $f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ and $\mu \geq 0$ | $U\operatorname{Diag}(d)U^\top$ with $d =\frac{1}{2(2\mu\gamma+1)} \Bigg( s+\sqrt{s^2+4\gamma(2\gamma\mu+1)}\Bigg)$ |
[Function]
[Prox] |
[SquaredFrobeniusNormLogDet] | [Bauschke et al., 2017] |
Schatten p-penalty + logdet |
$f(X) +\mu \mathcal{R}_p^p(X) $ with $p\le1$ and $f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ and $\mu \geq 0$ | $U\operatorname{Diag}(d)U^\top$ with $d =(d_i)_{1\le i\le n}$ and $(\forall i\in {1,....n}) \quad d_i >0 $ and $\gamma\mu p d_i^p + d_i^2-\lambda_i d_i = \gamma$ |
[Function]
[Prox] |
[SchattenPenaltyLogDet] | [Benfenati et al., 2018] |
Inverse Schatten p-penalty + logdet |
$f(X) +\mu \mathcal{R}_p^p(X^{-1}) $ with $p\le1$ and $f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ and $\mu \geq 0$ | $U\operatorname{Diag}(d)U^\top$ with $d =(d_i)_{1\le i\le n}$ and $(\forall i\in {1,....n}) \quad d_i >0 $ and $ d_i^{p+2} -\lambda_i d_i^{p+1} - \gamma d_i^p= \mu\gamma p $ |
[Function]
[Prox] |
[InvPenaltyLogDet] | [Benfenati et al., 2018] |
Van Neumann entropy |
$f(X) =\left\{ \begin{aligned} &\operatorname{trace}(X \log(X)) &&\textrm{if $X\succeq 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ | $U\operatorname{Diag}(d)U^\top$ with $d = \gamma W \left(\frac{1}{\gamma} \exp(\frac{s}{\gamma}-1) \right)$ |
[Function]
[Prox] |
[NeumannEntropy] | [Bauschke et al., 2017] |
Description |
f(x,ξ)
(∀(x,ξ) ∈ ℝN×ℝ) |
proxγf(x,ξ)
(∀ γ ∈ ℝ+) |
Matlab | Python | Ref |
---|---|---|---|---|---|
Square | $\begin{cases} \dfrac{\|x\|_2^2}{\xi} & \textrm{if $\xi > 0$}\\ 0 & \textrm{if $x = 0$ and $\xi=0$}\\ +\infty & \textrm{otherwise}.\end{cases}$ |
$\left\{\begin{aligned}
&(0,0) &&\textrm{if $\|x\|_2^2\le -4\gamma\xi$}\\
&(0,\xi) &&\textrm{if $x=0$ and $\xi>0$}\\
&\Big(x-\dfrac{\gamma tx}{\|x\|_2}, \xi+\dfrac{\gamma t^2}{4}\Big) &&\textrm{otherwise}
\end{aligned}\right.$
with $t\ge0$ such that $\small \gamma t^3 + 4(\xi+2\gamma) t - 8\|x\|_2=0$ |
[Function]
[Prox] |
[PersSquare] | [Combettes et al., 2017] |
Squared root | $\begin{cases} -\sqrt{\xi^2-\|x\|_2^2} & \textrm{if $\xi > 0$ and $\|x\|_2^2\le\xi$}\\ 0 & \textrm{if $x = 0$ and $\xi=0$}\\ +\infty & \textrm{otherwise}.\end{cases}$ |
$\left\{\begin{aligned}
&\Big(x-\dfrac{\gamma tx}{\|x\|_2}, \xi+\gamma\sqrt{1+ t^2}\Big) &&\textrm{if $\xi+\sqrt{\gamma^2+\|x\|_2^2}>0$}\\
&(0,0) &&\textrm{otherwise}
\end{aligned}\right.$
with $t\ge0$ such that $\small 2\gamma t + \dfrac{\xi t}{\sqrt{1+t^2}} - \|x\|_2^2 = 0$ (assuming that $x/\|x\|_2=0$ if $x=0$) |
[Function]
[Prox] |
[PersSqrt] | [Combettes et al., 2017] |
Huber |
$\begin{cases}
\rho|x|-\dfrac{\xi\rho^2}{2} & \textrm{if $\xi > 0$ and $|x|>\xi\rho$}\\
\dfrac{|x|^2}{2\xi} & \textrm{if $\xi > 0$ and $|x|\le\xi\rho$}\\
\rho|y| & \textrm{if $x = 0$}\\
+\infty & \textrm{if $x < 0$}\end{cases}$
(with $N=1$ and $\rho>0$) |
$\left\{\begin{aligned} &(0,0) &&\textrm{if $2\gamma\xi+|x|^2\le0$ and $|y|\le\gamma\rho$}\\ &\Big(x-\gamma\rho\operatorname{sign}(x),0\Big) &&\textrm{if $\xi\le -\gamma\rho^2/2$ and $|x|>\gamma\rho$}\\ &\Big(x-\gamma\rho\operatorname{sign}(x),\xi+\gamma\rho^2/2\Big) &&\textrm{if $\xi>-\gamma\rho^2/2$ and $|x|>\rho\xi+\gamma\rho(1+\rho^2/2)$}\\ &\operatorname{prox}_{\gamma|\cdot|^2/(\cdot)}(x,\xi) &&\textrm{if $\xi>-\gamma\rho^2/2$ and $|x|\le\rho\xi+\gamma\rho(1+\rho^2/2)$} \end{aligned}\right.$ |
[Function]
[Prox] |
[PersHuber] | [Combettes et al., 2017] |
Vapnik |
$\begin{cases}
\displaystyle\inf_{y\in\mathbb{R}} \; |y| + \iota_{[-\varepsilon\xi,\varepsilon\xi]}(x-y) & \textrm{if $\xi \ge 0$}\\
+\infty & \textrm{if $\xi < 0$}\end{cases}$
(with $N=1$ and $\rho>0$) |
$\left\{\begin{aligned} &(0,0) &&\textrm{if $\xi+\varepsilon|x|\le0$ and $|x|\le\gamma$}\\ &\Big(x-\gamma\operatorname{sign}(x),0\Big) &&\textrm{if $\xi\le-\gamma\varepsilon$ and $|x|>\gamma$}\\ &\Big(x-\gamma\operatorname{sign}(x),\xi+\gamma\varepsilon\Big) &&\textrm{if $\xi>-\gamma\varepsilon$ and $|x|>\varepsilon\xi+\gamma(1+\varepsilon^2)$}\\ &\Big(\varepsilon(\xi+\varepsilon|x|)\operatorname{sign}(x)/(1+\varepsilon^2),(\xi+\varepsilon|x|)/(1+\varepsilon^2)\Big) &&\textrm{if $|x|>-\xi/\varepsilon$ and $\varepsilon\xi\le|x|\le\varepsilon\xi+\gamma(1+\varepsilon^2)$}\\ &(y,\xi) &&\textrm{if $\xi\ge0$ and $|x|\le\varepsilon\xi$} \end{aligned}\right.$ |
[Function]
[Prox] |
[PersVapnik] | [Combettes et al., 2017] |
Description |
f(x,y)
(∀(x,y) ∈ ℝ2) |
proxγf(x,y)
(∀ γ ∈ ℝ+) |
Matlab | Python | Ref |
---|---|---|---|---|---|
Absolute difference |
$\left\{\begin{aligned} &|x-y| && \mbox{if $x\ge0$ and $y\ge0$}\\ &+\infty && \mbox{otherwise}\end{aligned}\right.$ |
$\small\left\{\begin{aligned}
&\frac12\big(x+y+p,x+y-p\big) && \mbox{if $|p| < x+y$}\\
&(x-\gamma,0) && \mbox{if $x > \gamma$ and $y \le -\gamma$}\\
&(0,y-\gamma) && \mbox{if $y > \gamma$ and $x \le -\gamma$}\\
&(0,0) && \mbox{otherwise}
\end{aligned}\right.$
where $\small p=\operatorname{sign}(x-y)\max\big\{|x-y|-2\gamma,0\big\}$ |
[Function]
[Prox] |
[AbsDiffk] | [El Gheche et al., 2018] |
Hellinger | $\left\{\begin{aligned}&(\sqrt{x}-\sqrt{y})^2 && \mbox{if $x\ge0$ and $y\ge0$}\\ &+\infty && \mbox{otherwise.}\end{aligned}\right.$ |
$\small\left\{\begin{aligned}
&\Big(x + \gamma (p-1),y+\gamma \big(p^{-1}-1\big)\Big) &&\mbox{if $x \ge \gamma$ or $\left(1-\dfrac{x}{\gamma}\right) \left(1-\dfrac{y}{\gamma}\right) < 1$}\\[1em]
&(0,0) && \mbox{otherwise}
\end{aligned}\right.$
where $\small\; p>\max\left\{1-\dfrac{x}{\gamma},0\right\}\;$ is such that $\small p^{4}+ \left(\dfrac{x}{\gamma}-1\right)p^3 + \left(1-\dfrac{y}{\gamma}\right)p-1= 0$ |
[Function]
[Prox] |
[Hellinger] | [El Gheche et al., 2018] |
Chi-square | $ \left\{\begin{aligned} &\dfrac{(x-y)^2}{y} &&\textrm{if $x \ge 0$ and $y>0$}\\ &0 &&\textrm{if $x = y = 0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
$\small \left\{\begin{aligned}
&\Big(x + 2 \gamma(1-p),y+\gamma (p^2-1)\Big) &&\mbox{if $x > -2\gamma$ and $y > - \left(x+\frac{1}{4\gamma}x^2\right)$}\\[1em]
&\big(0,\max\{y-\gamma,0\}\big) && \mbox{otherwise}
\end{aligned}\right.$
where $\small\; p \in \left]0,1+\dfrac{x}{2\gamma}\right[ \;$ is such that $\small p^3+\left(1+\dfrac{y}{\gamma}\right)p - \dfrac{x}{\gamma} -2 = 0$ |
[Function]
[Prox] |
[ChiSquare] | [El Gheche et al., 2018] |
Kullback-Leibler | $ \left\{\begin{aligned} &x \log\Big(\dfrac{x}{y}\Big) && \mbox{if $x>0$ and $y>0$}\\ &0 &&\mbox{if $x=0$ and $y \ge 0$}\\ &+\infty && \mbox{otherwise}\end{aligned}\right.$ |
$\small \left\{\begin{aligned}
&\Big(x + \gamma \log(p) -\gamma,y+\gamma p^{-1}\Big) && \mbox{if $\exp\left(\dfrac{x}{\gamma}-1\right) > -\dfrac{y}{\gamma}$}\\[1em]
&(0,0) && \mbox{otherwise}
\end{aligned}\right.$
where $\small\; p>\exp\left(-\dfrac{x}{\gamma}-1\right) \;$ is such that $\small\;p \log(p) + \left(\dfrac{x}{\gamma}-1\right)p - p^{-1} - \dfrac{y}{\gamma} = 0$ |
[Function]
[Prox] |
[Kullback] | [El Gheche et al., 2018] |
Jeffrey | $\small \left\{\begin{aligned}&(x-y) \Big(\log(x)-\log(y)\Big) && \mbox{if $x>0$ and $y>0$}\\ &0 && \mbox{if $x=y=0$}\\ &+\infty && \mbox{otherwise.}\end{aligned}\right.$ |
$\small \begin{cases}
\Big(x + \gamma \big(\log(p) + p-1),y-\gamma \big(\log(p) -p^{-1}+1)\Big) &\mbox{if $W(e^{1-\gamma^{-1}x})W(e^{1 - \gamma^{-1}y}) < 1$}\\[1em]
(0,0) & \mbox{otherwise}
\end{cases}$
where $\small\; p\in\left[W(e^{1-\gamma^{-1}x}),\Big(W(e^{1 - \gamma^{-1}y})\Big)^{-1}\right] \;$ is such that $\small (p+1) \log(p) - p^{-1} + p^2 + \Big(\dfrac{x}{\gamma}- 1\Big) p + 1 - \dfrac{y}{\gamma} = 0$ |
[Function]
[Prox] |
[Jeffrey] | [El Gheche et al., 2018] |
Rényi | $ \left\{\begin{aligned} &\dfrac{x^\alpha}{y^{\alpha-1}} &&\textrm{if $x \ge 0$ and $y>0$}\\ &0 &&\textrm{if $x = y = 0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
$\small \left\{\begin{aligned}
&\left(x - \frac{\gamma \alpha}{ p^{\alpha-1}},y+ \frac{\gamma (\alpha-1)}{p^{\alpha}}\right) &&\mbox{if $x>0$ and $\dfrac{\gamma^{\frac{1}{\alpha-1}} \, y}{1-\alpha} < \left(\dfrac{x}{\alpha}\right)^{\frac{\alpha}{\alpha-1}}$ }\\[1em]
&\big(0,\max\{y,0\}\big) && \mbox{otherwise}
\end{aligned}\right.$
where $\small\; p > \Big(\dfrac{\alpha \gamma}{x}\Big)^{\frac{1}{\alpha-1}} \;$ is such that $\small\dfrac{x}{\gamma}\;p^{\alpha+1} - \dfrac{y}{\gamma}\,p^{\alpha} - \alpha p^{2} + 1-\alpha = 0$ |
[Function]
[Prox] |
[Renyi] | [El Gheche et al., 2018] |
$I_\alpha$ | $ \left\{\begin{aligned} &-\sqrt[\alpha]{x \, y^{\alpha-1}} &&\textrm{if $x \ge 0$ and $y\ge0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
$\small \left\{\begin{aligned}
&\left(x +\frac{\gamma}{\alpha}\big(p^{\alpha-1}-1\big),y + \frac{\gamma(\alpha-1)}{\alpha} \big(p^{-1}-1\big)\right) &&\mbox{if $\alpha x \ge \gamma\;$ or $\;1-\frac{\alpha y}{\gamma(\alpha - 1)}<\left(\frac{\gamma}{\gamma-\alpha x}\right)^{\frac{1}{\alpha-1}}$}\\[1em]
&(0,0) && \mbox{otherwise}
\end{aligned}\right.$
where $\small\; p > \left(\max\Big\{1- \dfrac{\alpha x}{\gamma},0\Big\}\right)^{\frac{1}{\alpha-1}}\;$ is such that $\small p^{2\alpha} + \left(\dfrac{\alpha x}{\gamma}-1\right) p^{\alpha+1} + \left(\alpha-1-\dfrac{\alpha y}{\gamma}\right) p -\alpha+1=0$ |
[Function]
[Prox] |
[Ialpha] | [El Gheche et al., 2018] |