Functions of a scalar variable

Standard - Bregman

Python users do NOT need to download the files, they can directly use our Python library proxop . To install proxop containing the proximity of all functions in this repository, you can run the following command in the Python terminal:

pip install proxop

The name of a class in the library is the same as the one indicated in the column 'Python' of this following table. For example, to compute the proximity operator of the absolute value function:

from proxop import AbsValue
import numpy as np

x = np.array([ -3., 1., 6., 3.])
AbsValue().prox(x)
# result: array([-2.,  0.,  5., 2.])

For the majority of the functions $f$ in the library, one can compute the proximity operator of their scaled version $\gamma f$ (with $\gamma > 0$) by simply using the parameter 'gamma' in the method prox (default value: gamma=1).

Example: Compute the proximity operator of the scaled version of the absolute value function:

from proxop import AbsValue

x = np.array([ -3., 1., 6.])

AbsValue().prox(x, gamma=2)
#result: array([-1.,  0.,  4.])

Standard

Name f(x)

(∀ x ∈ ℝ)

proxγf(x)

(∀ γ ∈ ℝ+)

Matlab Python Ref
Support function $\sigma_{[a,b]}(x)=\left\{\begin{aligned} &a x &&\textrm{if $x < 0$}\\&0 &&\textrm{if $x = 0$}\\ &bx &&\textrm{if $x > 0$}\end{aligned}\right.$

(with a < b)

$\operatorname{soft}_{[a\gamma,b\gamma ]}(x) = \left\{\begin{aligned} &x-a\gamma &&\textrm{if $x < a\gamma$}\\&0 &&\textrm{if $a\gamma\le x \le b\gamma$}\\ &x-b\gamma &&\textrm{otherwise}\end{aligned}\right.$ [Function]
[Prox]
[Thresholder] [Combettes et al., 2008]
Absolute value $\sigma_{[-1,1]}(x) =|x|$ $\operatorname{soft}_{[-\gamma,\gamma]}(x)$ [Function]
[Prox]
[AbsValue] [Donoho, 1995]
Hinge loss $\max\{1-x,0\}$ $\left\{\begin{aligned} &x + \gamma &&\textrm{if $x < 1-\gamma$}\\&1 &&\textrm{if $1 -\gamma \le x \le 1$}\\ &x &&\textrm{if $x > 1$}\end{aligned}\right.$ [Function]
[Prox]
[HingeLoss] [Combettes et al., 2008]
$\omega$-insensitive loss $\max\{|x|-\omega,0\}$

(with ω>0)

$\left\{ \begin{aligned} &x &&\textrm{if $|x| < \omega$}\\ &\omega\operatorname{sign}(x) &&\textrm{if $\omega\le|x| \le \omega+\gamma$}\\ &x - \gamma\operatorname{sign}(x) &&\textrm{otherwise} \end{aligned} \right.$ [Function]
[Prox]
[Bathtub] [Combettes et al., 2011]
Square $\dfrac{1}{2}x^2$ $\dfrac{x}{\gamma+1}$ [Function]
[Prox]
[Square] [Moreau, 1965]
Squared hinge loss $\dfrac{1}{2}\big(\max\{x,0\}\big)^2$ $\left\{ \begin{aligned} &x &&\textrm{if $x \le 0$}\\ &\dfrac{x}{\gamma+1} &&\textrm{otherwise} \end{aligned} \right.$ [Function]
[Prox]
[SquaredHinge]
Squared $\omega$-insensitive loss $\dfrac{1}{2}\big(\max\{|x|-\omega,0\}\big)^2$

(with ω>0)

$\left\{ \begin{aligned} &x &&\textrm{if $|x|\le\omega$}\\ &\dfrac{x+\omega\gamma\operatorname{sign}(x)}{\gamma+1} &&\textrm{otherwise} \end{aligned} \right.$ [Function]
[Prox]
[SquaredBathtub]
Huber $\left\{ \begin{aligned} &\frac{1}{2} x^2 &&\textrm{if $|x| \le \omega$}\\ &\omega |x|-\frac{\omega^2}{2} &&\textrm{otherwise} \end{aligned} \right.$

(with ω>0)

$\left\{ \begin{aligned} &\dfrac{x}{\gamma+1} &&\textrm{if $|x| \le \omega(\gamma+1)$}\\ &x - \omega\gamma \operatorname{sign}(x) &&\textrm{otherwise} \end{aligned} \right.$ [Function]
[Prox]
[Huber] [Combettes et al., 2007]
Berhu $\left\{ \begin{aligned} & \frac{x^2 + \omega^2}{2 \omega} &&\textrm{if $|x| > \omega$}\\ & |\omega|&&\textrm{otherwise} \end{aligned} \right.$

(with ω>0)

If $\gamma = 1$:

$\left\{ \begin{aligned} &\dfrac{\omega x}{\omega+1} &&\textrm{if $|x| > \omega +1$}\\ & x - \textrm{sign}(x) && \textrm{if $1 < |x| \leq \omega +1$}\\ &0 &&\textrm{otherwise} \end{aligned} \right.$
(coming soon) [Berhu] [Bauschke et al., 2017]
Hyperbolic $\sqrt{x^2 + \delta^2}$

(with $\delta$>0)

$\operatorname{sign}(x) p$ with $p$ unique root in $[0,|x|]$ of:



$\psi(p)= p^4 + (-2 |x|)p^3 + (x^2 - \gamma^2 + \delta^2) p^2 + (-2 |x| \delta^2) p + \delta^2 x^2$
[Function]
[Prox]
[Hyperbolic]
Power $|x|^q$

(with q ≥ 1)

$\left\{\begin{aligned} &\operatorname{sign}(x) \max\{0, |x| - \gamma\} &&\textrm{if $q=1$}\\[0.5em] &x + \dfrac{4\gamma}{3\sqrt[3]{2}}\left(\sqrt[3]{\xi-x}-\sqrt[3]{\xi+x}\right) \scriptstyle\quad\textrm{with $\; \xi=\sqrt{x^2+256\gamma^3/729}$} &&\textrm{if $q=\dfrac{4}{3}$}\\[0.5em] &x + \frac{9}{8}\gamma^2\operatorname{sign}(x)\left(1-\sqrt{1+\frac{16|x|}{9\gamma^2}}\right) &&\textrm{if $q=\dfrac{3}{2}$}\\[0.5em] &\dfrac{x}{2\gamma+1} &&\textrm{if $q=2$}\\[0.5em] &\operatorname{sign}(x)\dfrac{\sqrt{1+12\gamma|x|}-1}{6\gamma} &&\textrm{if $q=3$}\\[0.5em] &\sqrt[3]{\dfrac{\xi+x}{8\gamma}}-\sqrt[3]{\dfrac{\xi-x}{8\gamma}} \scriptstyle\qquad\textrm{with $\;\xi=\sqrt{x^2+1/(27\gamma)}$} &&\textrm{if $q=4$}\\[0.5em] &\operatorname{sign}(x) p \scriptstyle\quad\textrm{with $p\ge0$ such that $\quad\displaystyle\gamma \, q \, p^{q-1} + p - |x| = 0$} &&\textrm{otherwise} \end{aligned}\right.$ [Function]
[Prox]
[Power] [Chaux et al., 2007]
Inverse $\left\{\begin{aligned} &\dfrac{1}{x^q} &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with q ≥ 1)

$p > 0$

such that

$p^{q+2}-xp^{q+1} - \gamma \, q = 0$
[Function]
[Prox]
[Inverse] [Combettes et al., 2011]
Negative root $\left\{\begin{aligned} &-\sqrt[q]{x} &&\textrm{if $x\ge0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with q ≥ 1)

$\left\{\begin{aligned} &\max\{0, x+\gamma\} &&\textrm{if $q=1$}\\ &\sqrt[q]{p} \scriptstyle\quad\textrm{with $p >0$ such that $\quad p^{2q-1}-xp^{q-1} - \gamma \, q^{-1} = 0$}&&\textrm{otherwise} \end{aligned}\right.$ [Function]
[Prox]
[NegativeRoot] [Combettes et al., 2011]
Exponential $\exp(x)$ $x - W\Big( \gamma \exp(x) \Big)$

where $W$ denotes the Lambert W-function

[Function]
[Prox]
[Exp] [Combettes et al., 2011]
Logistic loss $\log\Big(1+\exp(x)\Big)$ $x - \operatorname{W}_{\exp(x)} \Big(\gamma \exp \left(x \right) \Big)$

where $\operatorname{W}_{\theta}$ is the generalized Lambert function

[Function]
[Prox]
[LogisticLoss] [Briceno-Arias et al., 2017]
Fermi-Dirac entropy $\left\{\begin{aligned} &x\log(x)+(1-x)\log(1-x) &&\textrm{if $0 < x < 1$}\\ &0 &&\textrm{if $x\in\{0,1\}$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ $\gamma \operatorname{W}_{\exp(\gamma^{-1}x)} \Big(\gamma^{-1} \exp \left(\gamma^{-1}x \right) \Big)$

where $\operatorname{W}_{\theta}$ is the generalized Lambert function

[Function]
[Prox]
[FermiDiracEntropy] [Chierchia et al., 2017]
Smoothed Fermi-Dirac entropy $\left\{\begin{aligned} &x\log(x)+(1-x)\log(1-x) - \frac{1}{2}x^2 &&\textrm{if $0 < x < 1$}\\ &0 &&\textrm{if $x\in\{0,1\}$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ If $\gamma = 1$:

$\dfrac{1}{1 + \exp(-x)}$
[Function]
[Prox]
[SmoothedFermiDiracEntropy] [Bauschke et al., 2017]
Entropy $\left\{\begin{aligned} &x\log(x) &&\textrm{if $x>0$}\\ &0 &&\textrm{if $x=0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ $\gamma W\Big(\gamma^{-1}\exp\big(\gamma^{-1}x-1\big)\Big)$

where $W$ denotes the Lambert W-function

[Function]
[Prox]
[Entropy] [Chaux et al., 2007]
Logarithm $\left\{\begin{aligned} &-\log(x) &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ $\dfrac{x + \sqrt{x^2 + 4\gamma}}{2}$ [Function]
[Prox]
[Log] [Combettes et al., 2005]
Logarithm + power $\left\{\begin{aligned} &-\log(x) + \omega x^q &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with ω>0 and q ≥ 1)

$p>0$

such that

$q \, \omega\gamma \, p^q+p^2 - xp - \gamma = 0$
[Function]
[Prox]
[LogPower] [Chaux et al., 2007]
Logarithm + inverse $\left\{\begin{aligned} &-\log(x) + \omega x^{-1} &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with ω>0)

$p>0$

such that

$\quad p^3 -xp^2 - \gamma p -\omega\gamma = 0$
[Function]
[Prox]
[LogInverse] [Chaux et al., 2007]
Logarithmic barriers $\left\{\begin{aligned} &-\log(x-a) -\log(b-x) &&\textrm{if $a < x < b$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with a < b)

$p\in\left]a,b\right[$

such that

$p^3 - (a+b+x) p^2 + (ab + (a+b)x - 2\gamma)p - abx + (a+b)\gamma=0$
[Function]
[Prox]
[LogBarrier] [Chaux et al., 2007]
Fair potential $\omega|x|-\log(1+\omega|x|)$

(with ω>0)

$\operatorname{sign}(x)\dfrac{\omega|x|-\omega^2\gamma-1+\sqrt{(\omega|x|-\omega^2\gamma-1)^2+4\omega|x|}}{2\omega}$ [Function]
[Prox]
[FairPotential] [Chaux et al., 2007]
PReLu $\left\{\begin{aligned} &0 &&\textrm{if $x > 0$}\\ &(\dfrac{1}{\alpha}+1)\dfrac{x^2}{2} &&\textrm{otherwise}\end{aligned}\right.$ If $\gamma = 1$:

$\operatorname{PReLu}(x) = \left\{\begin{aligned} &x &&\textrm{if $x >0$}\\ &\alpha x &&\textrm{otherwise}\end{aligned}\right.$
[Function]
[Prox]
[PRelu] [P.L. Combettes and J.-C. Pesquet 2018]
Bent Identity Activation $\left\{\begin{aligned} &\frac{x}{2}-\frac{\ln{(x+1/2)}}{4} &&\textrm{if $x > -\frac{1}{2}$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ If $\gamma = 1$:

$\frac{x+\sqrt{x^2 +1}-1}{2}$
[Function]
[Prox]
[BentIdentity] [P.L. Combettes and J.-C. Pesquet 2018]
Inverse Square Root Unit Activation $\left\{\begin{aligned} &-\frac{x^2}{2}-\sqrt{1-x^2} &&\textrm{if $|x| \leqslant 1$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ If $\gamma = 1$:

$\frac{x}{\sqrt{1+x^2}}$
[Function]
[Prox]
[ISRU] [P.L. Combettes and J.-C. Pesquet 2018]
Inverse Square Root Linear Unit Activation $\left\{\begin{aligned} & 0 &&\textrm{if $x \geqslant 0$}\\&1-\frac{x^2}{2}-\sqrt{1-x^2} &&\textrm{if $-1 \leqslant x < 0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ If $\gamma = 1$:

$\left\{\begin{aligned} &x &&\textrm{if $x \geqslant 0$}\\&\frac{x}{\sqrt{1+x^2}} &&\textrm{otherwise}\end{aligned}\right.$
[Function]
[Prox]
[ISRLU] [P.L. Combettes and J.-C. Pesquet 2018]
Arctangent Activation $\left\{\begin{aligned} &-\frac{2}{\pi}\ln{(\cos{(\frac{\pi x}{2})})}-\frac{x^2}{2} &&\textrm{if $|x| < 1$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ If $\gamma = 1$:

$\frac{2}{\pi} \arctan(x)$
[Function]
[Prox]
[ArctanActi] [P.L. Combettes and J.-C. Pesquet 2018]
Hyperbolic Tangent Activation $ \left\{\begin{aligned} &x \operatorname{artanh}{(x)}+\frac{1}{2}(\ln{(1-x^2)}-x^2) &&\textrm{if $|x| < 1$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ If $\gamma = 1$:

$ \tanh(x)$
[Function]
[Prox]
[TanhActi] [P.L. Combettes and J.-C. Pesquet 2018]
Unimodal Sigmoid Activation $ \left\{\begin{aligned} &(x+\frac{1}{2} )\ln{(x+\frac{1}{2})} +(\frac{1}{2}-x )\ln{(\frac{1}{2}-x)} -\frac{1}{2}(x^2+\frac{1}{4}) &&\textrm{if $|x| < \frac{1}{2}$}\\&-\frac{1}{4} &&\textrm{if $|x| = \frac{1}{2}$}\\ &+\infty &&\textrm{if $|x| > \frac{1}{2}$}\end{aligned}\right.$ If $\gamma = 1$:

$ \frac{1}{1+\exp{(-x)}}-\frac{1}{2}$
[Function]
[Prox]
[UnimodalSigmoid] [P.L. Combettes and J.-C. Pesquet 2018]
Elliot Activation $ \left\{\begin{aligned} &-|x|-\ln{(1-|x|)}-\frac{x^2}{2} &&\textrm{if $|x| < 1$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ If $\gamma = 1$:

$ \frac{x}{1+|x|}$
[Function]
[Prox]
[ElliotActi] [P.L. Combettes and J.-C. Pesquet 2018]
Inverse Hyperbolic Sine Activation $ \text{cosh}(x)-\frac{x^2}{2}$ If $\gamma = 1$:

$ \text{arcsinh}(x)$
[Function]
[Prox]
[ArgsinhActi] [P.L. Combettes and J.-C. Pesquet 2018]
Logarithmic Activation $ \exp{(|x|)}-|x|-1-\frac{x^2}{2}$ If $\gamma = 1$:

$ \text{sign}(x) \ln{(1+|x|)}$
[Function]
[Prox]
[LogActi] [P.L. Combettes and J.-C. Pesquet 2018]
ELU Activation $\left\{\begin{aligned} &0 &&\textrm{if $x \geqslant 0$}\\ &(x + \omega) \ln\left( \frac{x + \omega}{\omega}\right)-x - \frac{x^2}{2} &&\textrm{if $-\omega < x < 0$}\\ &\omega - \frac{\omega^2}{2} &&\textrm{if $x = - \omega$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$

(with $\omega \geqslant 1$)

If $\gamma = 1$:

$\left\{\begin{aligned} &x &&\textrm{if $x \geqslant 0$}\\ &\omega (\exp(x)-1) &&\textrm{otherwise} \end{aligned}\right.$
(coming soon) [ELUacti] [P.L. Combettes and J.-C. Pesquet 2020]
Geman McClure Activation $\left\{\begin{aligned} &\mu \textrm{arctan}\sqrt{\frac{|x|}{\mu-|x|}}-\sqrt{|x|(\mu-|x|)}-\frac{x^2}{2} &&\textrm{if $|x|< \mu$}\\ & \frac{\mu(\pi - \mu)}{2} &&\textrm{if $|x|= \mu$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$

with $\mu = \frac{8}{3 \sqrt{3}}$
If $\gamma = 1$:

$\dfrac{\mu \textrm{sign}(x)x^2}{1 + x^2}$
(coming soon) [GMacti] [P.L. Combettes and J.-C. Pesquet 2020]

Bregman

Name f(x)

(∀ x ∈ ℝ)

ψ(u)

(∀ u ∈ ℝ)


proxψγf(x)

(∀ γ ∈ ℝ+)

Matlab Python Ref
Absolute value $|x-\delta|$

(with δ > 0)

$\left\{\begin{aligned} &u\log u &&\textrm{if $u>0$}\\ &0 &&\textrm{if $u=0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $\left\{\begin{aligned} &x \, \exp(\gamma) &&\textrm{if $x<\delta\,\exp({-\gamma})$}\\ &\delta &&\textrm{if $\delta\,\exp({-\gamma})\le x \le \delta\,\exp({\gamma})$}\\ &x \, \exp({-\gamma}) &&\textrm{otherwise} \end{aligned}\right.$ (coming soon) [BregAbsEntropy] [Bauschke et al., 2017]
Absolute value $|x-\delta|$

(with δ > 0)

$\left\{\begin{aligned} &-\log u &&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $\left\{\begin{aligned} &\frac{x}{1+\gamma x} &&\textrm{if $x<\frac{\delta}{1-\gamma\delta}$}\\ &\delta &&\textrm{if $\frac{\delta}{1-\gamma\delta}\le x \le \frac{\delta}{1+\gamma\delta}$}\\ &\frac{x}{1-\gamma x} &&\textrm{otherwise} \end{aligned}\right.$ (coming soon) [BregAbsLog] [Bauschke et al., 2017]
Exponential $\exp(x)$ $\exp(u)$ $x - \log(1+\gamma)$ (coming soon) [BregExp] [Bauschke et al., 2017]
Square $\frac{1}{2}x^2$ $\left\{\begin{aligned} &-\log u &&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $\dfrac{\sqrt{1+ 4\gamma x^2}-1}{2\gamma x}$ (coming soon) [BregSquareLog] [Bauschke et al., 2017]
$\ell_p$ norm $\frac{1}{p}|x|^p$

(with $p \ge 1$)

$\psi(u)=\left\{\begin{aligned} & u \log u - u&&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $ \left\{\begin{aligned} & \left( \frac{ W\big(\gamma (p-1) \exp({(p-1)x})\big) }{\gamma (p-1)} \right)^{\dfrac{1}{p-1}} && \textrm{if $p>1$}\\ & \exp({x - \gamma}) && \textrm{if $p=1$} \end{aligned}\right. $ (coming soon) [BreLpNorm] [Combettes et al., 2016]
Boltzmann-Shannon entropy $\left\{\begin{aligned} & x \log x - w x&&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$

(with $w \in \mathbb{R}$)

$\psi(u)=\left\{\begin{aligned} & u \log u - u&&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $\exp({\frac{x + \gamma (w - 1)}{\gamma + 1}})$ (coming soon) [BregBoltzShannon] [Combettes et al., 2016]
Boltzmann-Shannon entropy $\left\{\begin{aligned} & x \log x - w x&&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$

(with $w \in \mathbb{R}$)

$\psi(u)=\left\{\begin{aligned} & u \log u + (1-u) \log(1-u) &&\textrm{if $u\in ]0,1[$}\\ & 0 && \textrm{if $u\in \{0,1 \}$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $-\frac{1}{2}\exp({x + w - 1}) + \sqrt{\frac{1}{4}\exp\big({2 (x + w - 1)}\big) + \exp({x + w - 1})} $ (coming soon) [BregBoltzShannon2] [Combettes et al., 2016]