Functions of a multivariate variable

Vector variable - Matrix variable - Symmetric matrix variable - Perspective functions - Divergences

Python users do NOT need to download the files, they can directly use our Python library proxop . To install proxop containing the proximity of all functions in this repository, you can run the following command in the Python terminal:

pip install proxop

The name of a class in the library is the same as the one indicated in the column 'Python' of this following table. For example, to compute the projection onto the constrained box $[\eta_1=-2, \eta_2=2]^N$:

from proxop import BoxConstraint
import numpy as np

x = np.array([-5, 1, 10., 0, 3])
BoxConstraint(-2., 2.).prox(x)
# result: array([-2.,  1.,  2.,  0.,  2.])

For the majority of the functions $f$ in the library, one can compute the proximity operator of their scaled version $\gamma f$ (with $\gamma > 0$) by simply using the parameter 'gamma' in the method prox.(default value: gamma=1).

Example: Compute the proximity operator of the scaled version of the absolute value function:

from proxop import AbsValue

x = np.array([ -3., 1., 6.])

AbsValue().prox(x, gamma=2)
#result: array([-1.,  0.,  4.])

Vector variable

Description f(x)

(∀ x ∈ ℝN)

proxγf(x)

(∀ γ ∈ ℝ+)

Matlab Python Ref
Euclidean norm $\|x\|_2 = \displaystyle\sqrt{\sum_{n=1}^N x_n^2}$ $\left(1 - \dfrac{\gamma}{\max\big\{\|x\|_2,\gamma\big\}}\right)x$ [Function]
[Prox]
[L2Norm] [Combettes et al., 2005]
Infinity norm $\|x\|_\infty = \displaystyle\max_{1\le n\le N} |x_n|$ $\Big(\operatorname{sign}(x_n)\min\{|x_n|,s\}\Big)_{1\le n\le N}$

where $s\in\mathbb{R}$ is such that $\displaystyle\sum_{n=1}^N \max\{0, |x_n| - s\} = \gamma$

[Function]
[Prox]
[Linf]
Maximum $\displaystyle\max_{1\le n\le N} x_n$ $\Big(\min\{x_n,s\}\Big)_{1\le n\le N}$

where $s\in\mathbb{R}$ is such that $\displaystyle\sum_{n=1}^N \max\{0, x_n - s\} = \gamma$

[Function]
[Prox]
[Max]
L1Norm $\|x\|_1 = \sum_{k=1}^{N}|x_k|$ $\Big(\operatorname{soft}_{[-\gamma,\gamma]}(x_k)\Big)_{1\le n\le N}$

where $\operatorname{soft}_{[-\gamma,\gamma ]}(x_k) = \left\{\begin{aligned} &x+\gamma &&\textrm{if $x < -\gamma$}\\&0 &&\textrm{if $-\gamma\le x \le \gamma$}\\ &x-\gamma &&\textrm{otherwise}\end{aligned}\right.$

[Function]
[Prox]
[L1Norm]
Vapnik $\displaystyle\max\big\{\|x\|_2-\varepsilon,0\big\}$

(with $\varepsilon>0$)

$\left\{\begin{aligned} &x &&\textrm{if $\|x\|_2 - \varepsilon\le 0$}\\ &\frac{\varepsilon}{\|x\|_2}x &&\textrm{if $0<\|x\|_2-\varepsilon\le\gamma$}\\ &\Big(1-\frac{\gamma}{\|x\|_2}\Big) x &&\textrm{otherwise}\\ \end{aligned}\right.$ [Function]
[Prox]
[Vapnik] [Combettes et al., 2011]
Euclidean norm + positivity $\|x\|_2 + \iota_{[0,+\infty)^N}(x)$ $\left(1 - \dfrac{\gamma}{\max\big\{\|\max(x,0)\|_2,\gamma\big\}}\right)\max(x,0)$ [Function]
[Prox]
[L2Positive]
Barrier of affine constraints $\left\{\begin{aligned} &-\log(b-a^{\top} x) &&\textrm{if $a^{\top} x < b$}\\&+ \infty &&\textrm{otherwise }\end{aligned}\right.$

(with $b \in \mathbb{R}$ and $a \in \mathbb{R}^N \setminus \{0\}$)

$x+\frac{b-a^{\top}x-\sqrt{(b-a^{\top}x)^2+4\gamma\|a\|^2}}{2\|a\|^2} a$ [Function]
[Prox]
[AffineBarrier] [Bertocchi et al., 2018]
Barrier of hyperslab constraints $\left\{\begin{aligned} &-\log(b_{M}a^{\top} x) -\log(a^{\top} x-b_{m})&& \textrm{if $b_{m} < a^{\top} x < b_{M}$}\\&+ \infty &&\textrm{otherwise }\end{aligned}\right.$

(with $b_m \in \mathbb{R}$, $b_M \in \mathbb{R}$, $b_m < b_M$, and $a \in \mathbb{R}^N \setminus \{0\}$)

$x+\frac{\kappa(x,\gamma)-a^{\top}x}{\|a\|^2} a$

with $\kappa(x,\gamma)$ unique solution in $]b_{m},b_{M}[$ of: $0=z^3 -(b_{m}+b_{M}+ a^{\top}x) z^2 +(b_{m}b_{M} +a^{\top} x(b_{m}+b_{M})-2\gamma \|a\|^2)z -b_{m}b_{M}a^{\top}x+\gamma (b_{m}+b_{M})||a||^2$

[Function]
[Prox]
[HyperslabBarrier] [Bertocchi et al., 2018]
Barrier of $\ell_2$ ball constraint $\left\{\begin{aligned} &-\log(\alpha-\|x-c\|^2)&&\textrm{if $\|x-c\|^2< \alpha$}\\&+ \infty &&\textrm{otherwise }\end{aligned}\right.$

(with $\alpha > 0$ and $c \in \mathbb{R}^N$)

$c+\frac{\alpha -\kappa(x,\gamma)^2}{\alpha -\kappa(x,\gamma)^2+2\gamma} (x-c)$

with $\kappa(x,\gamma)$ unique solution in $[0,\sqrt{\alpha}[$ of: $0=z^3 -\|x-c\| z^2 -(\alpha+2\gamma)z +\alpha \|x-c\|$

[Function]
[Prox]
[L2BallBarrier] [Bertocchi et al., 2018]
Softmax activation $$f(x)=\phi(x+u)+u$$

where


$ \phi(\xi)=\left\{\begin{aligned} & \sum_{k=1}^{N}\left(\xi_k ln(\xi_k) -\frac{\xi_k^2}{2}\right) &&\textrm{if $(\xi_k)_{(1 \le k \le N)} \in [0,1]^N$ and $\sum_{k=1}^{N}\xi_k = 1$}\\&+ \infty &&\textrm{otherwise }\end{aligned}\right.$


and $u=(1,1,...,1)^T \in \mathbb{R}^N$

If $\gamma = 1$:

$\left( \frac{ exp( \xi_k )}{\sum_{j=1}^{N}exp(\xi_j) }\right)_{(1 \le k \le N)}-u $

where $u=(1,1,...,1)^T \in \mathbb{R}^N$

[coming soon] [SoftmaxActi] [Combettes and Pesquet, 2018]
Squashing $\left\{\begin{aligned} &\mu \textrm{arctan}\sqrt{\frac{\|x\|}{\mu-\|x\|}}-\sqrt{\|x\|(\mu-\|x\|)}-\frac{\|x\|^2}{2} &&\textrm{if $\|x\|< \mu$}\\ & \frac{\mu(\pi - \mu)}{2} &&\textrm{if $\|x\|= \mu$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$



with $\mu = \frac{8}{3 \sqrt{3}}$
If $\gamma = 1$:

$\frac{\mu \|x\|}{1 + \|x\|^2} x$
(coming soon) [Squashing] [P.L. Combettes and J.-C. Pesquet 2020]
Least squares downsampled convolution $\frac12 \|\mathbf{S}_m\mathbf{Hx - y} \|_2^2 $

(with $\mathbf{S}_m$ downsampling operator with factor $m$ in each dimension,


$\mathbf{H}= \mathbf{F}^\ast \mathbf{\Lambda} \mathbf{F}$ convolution operator, $\mathbf{F}$ discrete Fourier transform, $\mathbf{\Lambda}$ diagonal)

$ \mathbf{F}^\ast \left(\mathbf{I} - \frac{\gamma}{m^d} \mathbf{\Lambda}^\ast \mathbf{P} \mathbf{D}^{-1} \mathbf{P}^\ast \mathbf{\Lambda} \right) \mathbf{F } \left(\mathbf{x} + \gamma \mathbf{H}^\ast \mathbf{S}_m^\ast \mathbf{y} \right) $

with $\mathbf{P}$ is a periodization operator with $m$ repetitions in each dimension, $d$ is the number of dimensions of $\mathbf{x}$, $\mathbf{D}= \mathbf{I} + \frac{\gamma}{m^d} \mathrm{diag}(\mathbf{P}^\ast \mathbf{\Lambda \Lambda}^\ast \mathbf{1})$

[Function]
[Prox]
[L2DownConv] [Zhao et al, 2016] [Soubies et al, 2019]

Matrix variable

Description f(X) = φ(s)

(∀ X = U Diag(s) V∈ ℝM × N)

proxγf(X)

(∀ γ ∈ ℝ+)

Matlab Python Ref
Nuclear norm $\displaystyle\|X\|_N = \|s\|_1$ $U\operatorname{Diag}(d)V^\top$ with $d = \operatorname{prox}_{\gamma\|\cdot\|_1}(s) = \operatorname{soft}_{[-\gamma,\gamma]}(s)$ [Function]
[Prox]
[NuclearNorm] [Bauschke et al., 2017]
Nuclear norm constraint $\displaystyle \iota_{\|X\|_N \le \eta}(X) = \iota_{\|s\|_1 \le \eta}(s)$ $U\operatorname{Diag}(d)V^\top$ with $d = s - \operatorname{prox}_{\eta\|\cdot\|_\infty}(s)$ [Function]
[Prox]
[NuclearBall] [Bauschke et al., 2017]
Block nuclear norm $\displaystyle\sum_{i= 1}^N\displaystyle\sum_{j=1}^M w_{i,j} \|X_{i,j}\|_N = \displaystyle\sum_{i= 1}^N\displaystyle\sum_{j=1}^M w_{i,j} \|s_{i,j}\|_1$ with each $w_{i,j} \ge 0$ acting on a block $X_{i,j}$ of matrix $X$ ($U_{i,j}\operatorname{Diag}(d_{i,j})V_{i,j}^\top$$)_{i,j}$ with $d_{i,j} = \operatorname{prox}_{\gamma w_{i,j}\|\cdot\|_1}(s_{i,j}) = \operatorname{soft}_{[-\gamma w_{i,j},\gamma w_{i,j}]}(s_{i,j})$ [Function]
[Prox]
[NuclearBlocks] [Bauschke et al., 2017]
Spectral norm $\displaystyle\|X\|_S = \|s\|_\infty$ $U\operatorname{Diag}\Big(\operatorname{prox}_{\gamma\|\cdot\|_\infty}(s)\Big)V^\top$ [Function]
[Prox]
[Spectral] [Bauschke et al., 2017]
Rank $\displaystyle\operatorname{rank}(X) = \|s\|_0$ $U\operatorname{Diag}(d)V^\top$ with $d = \operatorname{prox}_{\gamma\|\cdot\|_0}(s) = \operatorname{hard}_{\sqrt{2 \gamma}}(s)$ (coming soon) [Rank] [Benfenati et al., 2018]
L21 norm (columnwise) $\displaystyle\sum_{j= 1}^M|\displaystyle\sum_{i=1}^N|X_{i,j}|^2|^{\frac{1}{2}}$ with $X_{i,j} \in \mathbb{R}$ the entry $(i,j)$ of matrix $X$

$\left((1- \frac{\gamma}{\max(\|X_j\|_2,\gamma)})X_j\right)_{1 \le j \le M}$

with $X_j \in \mathbb{R}^M$ the $j$-th column of $X$
[Function]
[Prox]
[L21columns] [Gramfort et al., 2012]
L21 norm (rowwise) $\displaystyle\sum_{i=1}^N|\displaystyle\sum_{j=1}^M|X_{i,j}|^2|^{\frac{1}{2}}$ with $X_{i,j} \in \mathbb{R}$ the entry $(i,j)$ of matrix $X$

$\left((1- \frac{\gamma}{\max(\|X_i\|_2,\gamma)})X_i\right)_{1 \le i \le N}$

with $X_i \in \mathbb{R}^N$ the $i$-th row of $X$
[Function]
[Prox]
[L21rows] [Gramfort et al., 2012]
L$\infty$1 norm $\displaystyle\sum_{i=1}^N \displaystyle\sup_{1\le j \le M} |X_{i,j}|$ with $X_{i,j} \in \mathbb{R}$ the entry $(i,j)$ of matrix $X$ $\Big(\Big(\operatorname{sign}(X_{i,j})\min\{|X_{i,j}|,s_i\}\Big)_{1\le j\le M}\Big)_{1\le i\le N}$

where $s_i\in\mathbb{R}$ is such that $\displaystyle\sum_{j=1}^M \max\{0, |X_{i,j}| - s_i\} = \gamma$

[Function]
[Prox]
[Linf1] [Gramfort et al., 2012]
Nuclear norm + Ridge $\frac{1}{2}\|X\|_F ^2 +\mu \|X\|_N = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_1$ with $\mu \geq 0$

$U\operatorname{Diag}(d)V^\top$

with $d =\operatorname{soft}_{[-\frac{\mu \gamma}{\gamma+1},\frac{\mu \gamma}{\gamma+1}]}(\frac{s}{1+\gamma})$
[Function]
[Prox]
[NuclearNormRidge] [Bauschke et al., 2017]
Rank + Ridge $\frac{1}{2}\|X\|_F ^2 +\mu \operatorname{rank}(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_0$ with $\mu \geq 0$

$U\operatorname{Diag}(d)V^\top$

with $d =\operatorname{hard}_{\sqrt{\frac{2 \mu \gamma}{1 + \gamma}}}(\frac{s}{1+\gamma})$
(coming soon) [RankRidge] [Benfenati et al., 2018]
Schatten 3-penalty + Ridge $\frac{1}{2}\|X\|_F ^2 +\mu \mathcal{R}_3^3(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_3^3$ with $\mu \geq 0$

$U\operatorname{Diag}(d)V^\top$

with $d =(6\gamma\mu)^{-1}\Bigg(\operatorname{sign}(s)\sqrt{(\gamma+1)^2+12|s|\gamma\mu}-\gamma-1\Bigg)$
[Function]
[Prox]
[Schatten3Penalty] [Bauschke et al., 2017]
Schatten 4-penalty + Ridge $\frac{1}{2}\|X\|_F ^2 +\mu \mathcal{R}_4^4(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_4^4$ with $\mu \geq 0$

$U\operatorname{Diag}(d)V^\top$

with $d =(8\gamma\mu)^{-\frac{1}{3}}\Bigg(\sqrt[3]{s+\sqrt{s^2+\zeta}}-\sqrt[3]{\sqrt{s^2+\zeta}-s}\Bigg)$

And $\zeta=\frac{(\gamma+1)^3}{27\gamma\mu}$

[Function]
[Prox]
[Schatten4Penalty] [Bauschke et al., 2017]
Schatten 4/3-penalty + Ridge $\frac{1}{2}\|X\|_F ^2 +\mu \mathcal{R}_{4/3}^{4/3}(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_{4/3}^{4/3}$ with $\mu \geq 0$

$U\operatorname{Diag}(d)V^\top$

with $d = \frac{1}{1+\gamma}\Bigg(s+\frac{4\gamma\mu}{3\sqrt[3]{2(1+\gamma)}}\Bigg(\sqrt[3]{\sqrt{s^2+\zeta}-s}-\sqrt[3]{s+\sqrt{s^2+\zeta}}\Bigg)\Bigg)$

And $\zeta=\frac{256(\gamma\mu)^3}{729(1+\gamma)}$

[Function]
[Prox]
[Schatten4_3Penalty] [Bauschke et al., 2017]
Schatten 3/2-penalty + Ridge $\frac{1}{2}\|X\|_F ^2 +\mu \mathcal{R}_{3/2}^{3/2}(X) = \frac{1}{2} \|s\|_2^2 + \mu \| s \|_{3/2}^{3/2}$ with $\mu \geq 0$

$U\operatorname{Diag}(d)V^\top$

with $d =\frac{1}{1+\gamma}\Bigg(s+\frac{9\gamma^2\mu^2}{8(1+\gamma)}\operatorname{sign}(s)\Bigg(1-\sqrt{1+\frac{16(1+\gamma)}{9\gamma^2\mu^2}|s|}\Bigg)\Bigg)$
[Function]
[Prox]
[Schatten3_2Penalty] [Bauschke et al., 2017]

Symmetric matrix variable

Description f(X) = φ(s)

(∀ X = U Diag(s) U∈ ℝN × N symmetric)

proxγf(X)

(∀ γ ∈ ℝ+)

Matlab Python Ref
Permutation invariant $\varphi(s)$ $U\operatorname{Diag}\Big(\operatorname{prox}_{\gamma\varphi}(s)\Big)U^\top$ [Function]
[Prox]
[PermutationInvariant] [Bauschke et al., 2017]
logdet

$f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$

$U\operatorname{Diag}(d)U^\top$

with $d =\frac{1}{2} \Bigg( s+\sqrt{s^2+4\gamma}\Bigg)$
[Function]
[Prox]
[LogDet] [Bauschke et al., 2017]
Nuclear norm + logdet

$f(X) +\mu \|X\|_N $

with $f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ and $\mu \geq 0$

$U\operatorname{Diag}(d)U^\top$

with $d =\frac{1}{2} \Bigg( s-\gamma\mu+\sqrt{(s-\gamma\mu)^2+4\gamma}\Bigg)$
[Function]
[Prox]
[NuclearLogDet] [Bauschke et al., 2017]
Squared Frobenius norm + logdet

$f(X) +\mu \|X\|_F^2$

with $f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ and $\mu \geq 0$

$U\operatorname{Diag}(d)U^\top$

with $d =\frac{1}{2(2\mu\gamma+1)} \Bigg( s+\sqrt{s^2+4\gamma(2\gamma\mu+1)}\Bigg)$
[Function]
[Prox]
[SquaredFrobeniusNormLogDet] [Bauschke et al., 2017]
Schatten p-penalty + logdet

$f(X) +\mu \mathcal{R}_p^p(X) $

with $p\le1$ and $f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ and $\mu \geq 0$

$U\operatorname{Diag}(d)U^\top$

with $d =(d_i)_{1\le i\le n}$ and $(\forall i\in {1,....n}) \quad d_i >0 $ and $\gamma\mu p d_i^p + d_i^2-\lambda_i d_i = \gamma$
[Function]
[Prox]
[SchattenPenaltyLogDet] [Benfenati et al., 2018]
Inverse Schatten p-penalty + logdet

$f(X) +\mu \mathcal{R}_p^p(X^{-1}) $

with $p\le1$ and $f(X) =\left\{ \begin{aligned} &-\log(\det(X)) &&\textrm{if $X\succ 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$ and $\mu \geq 0$

$U\operatorname{Diag}(d)U^\top$

with $d =(d_i)_{1\le i\le n}$ and $(\forall i\in {1,....n}) \quad d_i >0 $ and $ d_i^{p+2} -\lambda_i d_i^{p+1} - \gamma d_i^p= \mu\gamma p $
[Function]
[Prox]
[InvPenaltyLogDet] [Benfenati et al., 2018]
Van Neumann entropy

$f(X) =\left\{ \begin{aligned} &\operatorname{trace}(X \log(X)) &&\textrm{if $X\succeq 0$}\\ &+\infty &&\textrm{otherwise} \end{aligned} \right.$

$U\operatorname{Diag}(d)U^\top$

with $d = \gamma W \left(\frac{1}{\gamma} \exp(\frac{s}{\gamma}-1) \right)$
[Function]
[Prox]
[NeumannEntropy] [Bauschke et al., 2017]

Perspective of convex functions

Description f(x,ξ)

(∀(x,ξ) ∈ ℝN×ℝ)

proxγf(x,ξ)

(∀ γ ∈ ℝ+)

Matlab Python Ref
Square $\begin{cases} \dfrac{\|x\|_2^2}{\xi} & \textrm{if $\xi > 0$}\\ 0 & \textrm{if $x = 0$ and $\xi=0$}\\ +\infty & \textrm{otherwise}.\end{cases}$ $\left\{\begin{aligned} &(0,0) &&\textrm{if $\|x\|_2^2\le -4\gamma\xi$}\\ &(0,\xi) &&\textrm{if $x=0$ and $\xi>0$}\\ &\Big(x-\dfrac{\gamma tx}{\|x\|_2}, \xi+\dfrac{\gamma t^2}{4}\Big) &&\textrm{otherwise} \end{aligned}\right.$

with $t\ge0$ such that

$\small \gamma t^3 + 4(\xi+2\gamma) t - 8\|x\|_2=0$
[Function]
[Prox]
[PersSquare] [Combettes et al., 2017]
Squared root $\begin{cases} -\sqrt{\xi^2-\|x\|_2^2} & \textrm{if $\xi > 0$ and $\|x\|_2^2\le\xi$}\\ 0 & \textrm{if $x = 0$ and $\xi=0$}\\ +\infty & \textrm{otherwise}.\end{cases}$ $\left\{\begin{aligned} &\Big(x-\dfrac{\gamma tx}{\|x\|_2}, \xi+\gamma\sqrt{1+ t^2}\Big) &&\textrm{if $\xi+\sqrt{\gamma^2+\|x\|_2^2}>0$}\\ &(0,0) &&\textrm{otherwise} \end{aligned}\right.$

with $t\ge0$ such that

$\small 2\gamma t + \dfrac{\xi t}{\sqrt{1+t^2}} - \|x\|_2^2 = 0$

(assuming that $x/\|x\|_2=0$ if $x=0$)
[Function]
[Prox]
[PersSqrt] [Combettes et al., 2017]
Huber $\begin{cases} \rho|x|-\dfrac{\xi\rho^2}{2} & \textrm{if $\xi > 0$ and $|x|>\xi\rho$}\\ \dfrac{|x|^2}{2\xi} & \textrm{if $\xi > 0$ and $|x|\le\xi\rho$}\\ \rho|y| & \textrm{if $x = 0$}\\ +\infty & \textrm{if $x < 0$}\end{cases}$

(with $N=1$ and $\rho>0$)

$\left\{\begin{aligned} &(0,0) &&\textrm{if $2\gamma\xi+|x|^2\le0$ and $|y|\le\gamma\rho$}\\ &\Big(x-\gamma\rho\operatorname{sign}(x),0\Big) &&\textrm{if $\xi\le -\gamma\rho^2/2$ and $|x|>\gamma\rho$}\\ &\Big(x-\gamma\rho\operatorname{sign}(x),\xi+\gamma\rho^2/2\Big) &&\textrm{if $\xi>-\gamma\rho^2/2$ and $|x|>\rho\xi+\gamma\rho(1+\rho^2/2)$}\\ &\operatorname{prox}_{\gamma|\cdot|^2/(\cdot)}(x,\xi) &&\textrm{if $\xi>-\gamma\rho^2/2$ and $|x|\le\rho\xi+\gamma\rho(1+\rho^2/2)$} \end{aligned}\right.$ [Function]
[Prox]
[PersHuber] [Combettes et al., 2017]
Vapnik $\begin{cases} \displaystyle\inf_{y\in\mathbb{R}} \; |y| + \iota_{[-\varepsilon\xi,\varepsilon\xi]}(x-y) & \textrm{if $\xi \ge 0$}\\ +\infty & \textrm{if $\xi < 0$}\end{cases}$

(with $N=1$ and $\rho>0$)

$\left\{\begin{aligned} &(0,0) &&\textrm{if $\xi+\varepsilon|x|\le0$ and $|x|\le\gamma$}\\ &\Big(x-\gamma\operatorname{sign}(x),0\Big) &&\textrm{if $\xi\le-\gamma\varepsilon$ and $|x|>\gamma$}\\ &\Big(x-\gamma\operatorname{sign}(x),\xi+\gamma\varepsilon\Big) &&\textrm{if $\xi>-\gamma\varepsilon$ and $|x|>\varepsilon\xi+\gamma(1+\varepsilon^2)$}\\ &\Big(\varepsilon(\xi+\varepsilon|x|)\operatorname{sign}(x)/(1+\varepsilon^2),(\xi+\varepsilon|x|)/(1+\varepsilon^2)\Big) &&\textrm{if $|x|>-\xi/\varepsilon$ and $\varepsilon\xi\le|x|\le\varepsilon\xi+\gamma(1+\varepsilon^2)$}\\ &(y,\xi) &&\textrm{if $\xi\ge0$ and $|x|\le\varepsilon\xi$} \end{aligned}\right.$ [Function]
[Prox]
[PersVapnik] [Combettes et al., 2017]

φ-divergences

Description f(x,y)

(∀(x,y) ∈ ℝ2)

proxγf(x,y)

(∀ γ ∈ ℝ+)

Matlab Python Ref
Absolute
difference
$\left\{\begin{aligned} &|x-y| && \mbox{if $x\ge0$ and $y\ge0$}\\ &+\infty && \mbox{otherwise}\end{aligned}\right.$ $\small\left\{\begin{aligned} &\frac12\big(x+y+p,x+y-p\big) && \mbox{if $|p| < x+y$}\\ &(x-\gamma,0) && \mbox{if $x > \gamma$ and $y \le -\gamma$}\\ &(0,y-\gamma) && \mbox{if $y > \gamma$ and $x \le -\gamma$}\\ &(0,0) && \mbox{otherwise} \end{aligned}\right.$

where

$\small p=\operatorname{sign}(x-y)\max\big\{|x-y|-2\gamma,0\big\}$
[Function]
[Prox]
[AbsDiffk] [El Gheche et al., 2018]
Hellinger $\left\{\begin{aligned}&(\sqrt{x}-\sqrt{y})^2 && \mbox{if $x\ge0$ and $y\ge0$}\\ &+\infty && \mbox{otherwise.}\end{aligned}\right.$ $\small\left\{\begin{aligned} &\Big(x + \gamma (p-1),y+\gamma \big(p^{-1}-1\big)\Big) &&\mbox{if $x \ge \gamma$ or $\left(1-\dfrac{x}{\gamma}\right) \left(1-\dfrac{y}{\gamma}\right) < 1$}\\[1em] &(0,0) && \mbox{otherwise} \end{aligned}\right.$

where

$\small\; p>\max\left\{1-\dfrac{x}{\gamma},0\right\}\;$

is such that

$\small p^{4}+ \left(\dfrac{x}{\gamma}-1\right)p^3 + \left(1-\dfrac{y}{\gamma}\right)p-1= 0$
[Function]
[Prox]
[Hellinger] [El Gheche et al., 2018]
Chi-square $ \left\{\begin{aligned} &\dfrac{(x-y)^2}{y} &&\textrm{if $x \ge 0$ and $y>0$}\\ &0 &&\textrm{if $x = y = 0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ $\small \left\{\begin{aligned} &\Big(x + 2 \gamma(1-p),y+\gamma (p^2-1)\Big) &&\mbox{if $x > -2\gamma$ and $y > - \left(x+\frac{1}{4\gamma}x^2\right)$}\\[1em] &\big(0,\max\{y-\gamma,0\}\big) && \mbox{otherwise} \end{aligned}\right.$

where

$\small\; p \in \left]0,1+\dfrac{x}{2\gamma}\right[ \;$

is such that

$\small p^3+\left(1+\dfrac{y}{\gamma}\right)p - \dfrac{x}{\gamma} -2 = 0$
[Function]
[Prox]
[ChiSquare] [El Gheche et al., 2018]
Kullback-Leibler $ \left\{\begin{aligned} &x \log\Big(\dfrac{x}{y}\Big) && \mbox{if $x>0$ and $y>0$}\\ &0 &&\mbox{if $x=0$ and $y \ge 0$}\\ &+\infty && \mbox{otherwise}\end{aligned}\right.$ $\small \left\{\begin{aligned} &\Big(x + \gamma \log(p) -\gamma,y+\gamma p^{-1}\Big) && \mbox{if $\exp\left(\dfrac{x}{\gamma}-1\right) > -\dfrac{y}{\gamma}$}\\[1em] &(0,0) && \mbox{otherwise} \end{aligned}\right.$

where

$\small\; p>\exp\left(-\dfrac{x}{\gamma}-1\right) \;$

is such that

$\small\;p \log(p) + \left(\dfrac{x}{\gamma}-1\right)p - p^{-1} - \dfrac{y}{\gamma} = 0$
[Function]
[Prox]
[Kullback] [El Gheche et al., 2018]
Jeffrey $\small \left\{\begin{aligned}&(x-y) \Big(\log(x)-\log(y)\Big) && \mbox{if $x>0$ and $y>0$}\\ &0 && \mbox{if $x=y=0$}\\ &+\infty && \mbox{otherwise.}\end{aligned}\right.$ $\small \begin{cases} \Big(x + \gamma \big(\log(p) + p-1),y-\gamma \big(\log(p) -p^{-1}+1)\Big) &\mbox{if $W(e^{1-\gamma^{-1}x})W(e^{1 - \gamma^{-1}y}) < 1$}\\[1em] (0,0) & \mbox{otherwise} \end{cases}$

where

$\small\; p\in\left[W(e^{1-\gamma^{-1}x}),\Big(W(e^{1 - \gamma^{-1}y})\Big)^{-1}\right] \;$

is such that

$\small (p+1) \log(p) - p^{-1} + p^2 + \Big(\dfrac{x}{\gamma}- 1\Big) p + 1 - \dfrac{y}{\gamma} = 0$
[Function]
[Prox]
[Jeffrey] [El Gheche et al., 2018]
Rényi $ \left\{\begin{aligned} &\dfrac{x^\alpha}{y^{\alpha-1}} &&\textrm{if $x \ge 0$ and $y>0$}\\ &0 &&\textrm{if $x = y = 0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ $\small \left\{\begin{aligned} &\left(x - \frac{\gamma \alpha}{ p^{\alpha-1}},y+ \frac{\gamma (\alpha-1)}{p^{\alpha}}\right) &&\mbox{if $x>0$ and $\dfrac{\gamma^{\frac{1}{\alpha-1}} \, y}{1-\alpha} < \left(\dfrac{x}{\alpha}\right)^{\frac{\alpha}{\alpha-1}}$ }\\[1em] &\big(0,\max\{y,0\}\big) && \mbox{otherwise} \end{aligned}\right.$

where

$\small\; p > \Big(\dfrac{\alpha \gamma}{x}\Big)^{\frac{1}{\alpha-1}} \;$

is such that

$\small\dfrac{x}{\gamma}\;p^{\alpha+1} - \dfrac{y}{\gamma}\,p^{\alpha} - \alpha p^{2} + 1-\alpha = 0$
[Function]
[Prox]
[Renyi] [El Gheche et al., 2018]
$I_\alpha$ $ \left\{\begin{aligned} &-\sqrt[\alpha]{x \, y^{\alpha-1}} &&\textrm{if $x \ge 0$ and $y\ge0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ $\small \left\{\begin{aligned} &\left(x +\frac{\gamma}{\alpha}\big(p^{\alpha-1}-1\big),y + \frac{\gamma(\alpha-1)}{\alpha} \big(p^{-1}-1\big)\right) &&\mbox{if $\alpha x \ge \gamma\;$ or $\;1-\frac{\alpha y}{\gamma(\alpha - 1)}<\left(\frac{\gamma}{\gamma-\alpha x}\right)^{\frac{1}{\alpha-1}}$}\\[1em] &(0,0) && \mbox{otherwise} \end{aligned}\right.$

where

$\small\; p > \left(\max\Big\{1- \dfrac{\alpha x}{\gamma},0\Big\}\right)^{\frac{1}{\alpha-1}}\;$

is such that

$\small p^{2\alpha} + \left(\dfrac{\alpha x}{\gamma}-1\right) p^{\alpha+1} + \left(\alpha-1-\dfrac{\alpha y}{\gamma}\right) p -\alpha+1=0$
[Function]
[Prox]
[Ialpha] [El Gheche et al., 2018]