pip install proxop
from proxop import AbsValue
import numpy as np
x = np.array([ -3., 1., 6., 3.])
AbsValue().prox(x)
# result: array([-2., 0., 5., 2.])
from proxop import AbsValue
x = np.array([ -3., 1., 6.])
AbsValue().prox(x, gamma=2)
#result: array([-1., 0., 4.])
Name |
f(x)
(∀ x ∈ ℝ) |
proxγf(x)
(∀ γ ∈ ℝ+) |
Matlab | Python | Ref |
---|---|---|---|---|---|
Support function |
$\sigma_{[a,b]}(x)=\left\{\begin{aligned} &a x &&\textrm{if $x < 0$}\\&0 &&\textrm{if $x = 0$}\\ &bx &&\textrm{if $x > 0$}\end{aligned}\right.$
(with a < b) |
$\operatorname{soft}_{[a\gamma,b\gamma ]}(x) = \left\{\begin{aligned} &x-a\gamma &&\textrm{if $x < a\gamma$}\\&0 &&\textrm{if $a\gamma\le x \le b\gamma$}\\ &x-b\gamma &&\textrm{otherwise}\end{aligned}\right.$ |
[Function]
[Prox] |
[Thresholder] | [Combettes et al., 2008] |
Absolute value | $\sigma_{[-1,1]}(x) =|x|$ | $\operatorname{soft}_{[-\gamma,\gamma]}(x)$ |
[Function]
[Prox] |
[AbsValue] | [Donoho, 1995] |
Hinge loss | $\max\{1-x,0\}$ | $\left\{\begin{aligned} &x + \gamma &&\textrm{if $x < 1-\gamma$}\\&1 &&\textrm{if $1 -\gamma \le x \le 1$}\\ &x &&\textrm{if $x > 1$}\end{aligned}\right.$ |
[Function]
[Prox] |
[HingeLoss] | [Combettes et al., 2008] |
$\omega$-insensitive loss | $\max\{|x|-\omega,0\}$ (with ω>0) |
$\left\{ \begin{aligned} &x &&\textrm{if $|x| < \omega$}\\ &\omega\operatorname{sign}(x) &&\textrm{if $\omega\le|x| \le \omega+\gamma$}\\ &x - \gamma\operatorname{sign}(x) &&\textrm{otherwise} \end{aligned} \right.$ |
[Function]
[Prox] |
[Bathtub] | [Combettes et al., 2011] |
Square | $\dfrac{1}{2}x^2$ | $\dfrac{x}{\gamma+1}$ |
[Function]
[Prox] |
[Square] | [Moreau, 1965] |
Squared hinge loss | $\dfrac{1}{2}\big(\max\{x,0\}\big)^2$ | $\left\{ \begin{aligned} &x &&\textrm{if $x \le 0$}\\ &\dfrac{x}{\gamma+1} &&\textrm{otherwise} \end{aligned} \right.$ |
[Function]
[Prox] |
[SquaredHinge] | |
Squared $\omega$-insensitive loss | $\dfrac{1}{2}\big(\max\{|x|-\omega,0\}\big)^2$
(with ω>0) |
$\left\{ \begin{aligned} &x &&\textrm{if $|x|\le\omega$}\\ &\dfrac{x+\omega\gamma\operatorname{sign}(x)}{\gamma+1} &&\textrm{otherwise} \end{aligned} \right.$ |
[Function]
[Prox] |
[SquaredBathtub] | |
Huber |
$\left\{
\begin{aligned}
&\frac{1}{2} x^2 &&\textrm{if $|x| \le \omega$}\\
&\omega |x|-\frac{\omega^2}{2} &&\textrm{otherwise}
\end{aligned}
\right.$
(with ω>0) |
$\left\{ \begin{aligned} &\dfrac{x}{\gamma+1} &&\textrm{if $|x| \le \omega(\gamma+1)$}\\ &x - \omega\gamma \operatorname{sign}(x) &&\textrm{otherwise} \end{aligned} \right.$ |
[Function]
[Prox] |
[Huber] | [Combettes et al., 2007] |
Berhu |
$\left\{
\begin{aligned}
& \frac{x^2 + \omega^2}{2 \omega} &&\textrm{if $|x| > \omega$}\\
& |\omega|&&\textrm{otherwise}
\end{aligned}
\right.$
(with ω>0) |
If $\gamma = 1$: $\left\{ \begin{aligned} &\dfrac{\omega x}{\omega+1} &&\textrm{if $|x| > \omega +1$}\\ & x - \textrm{sign}(x) && \textrm{if $1 < |x| \leq \omega +1$}\\ &0 &&\textrm{otherwise} \end{aligned} \right.$ |
(coming soon) | [Berhu] | [Bauschke et al., 2017] |
Hyperbolic |
$\sqrt{x^2 + \delta^2}$
(with $\delta$>0) |
$\operatorname{sign}(x) p$ with $p$ unique root in $[0,|x|]$ of:
$\psi(p)= p^4 + (-2 |x|)p^3 + (x^2 - \gamma^2 + \delta^2) p^2 + (-2 |x| \delta^2) p + \delta^2 x^2$ |
[Function]
[Prox] |
[Hyperbolic] | |
Power | $|x|^q$ (with q ≥ 1) |
$\left\{\begin{aligned} &\operatorname{sign}(x) \max\{0, |x| - \gamma\} &&\textrm{if $q=1$}\\[0.5em] &x + \dfrac{4\gamma}{3\sqrt[3]{2}}\left(\sqrt[3]{\xi-x}-\sqrt[3]{\xi+x}\right) \scriptstyle\quad\textrm{with $\; \xi=\sqrt{x^2+256\gamma^3/729}$} &&\textrm{if $q=\dfrac{4}{3}$}\\[0.5em] &x + \frac{9}{8}\gamma^2\operatorname{sign}(x)\left(1-\sqrt{1+\frac{16|x|}{9\gamma^2}}\right) &&\textrm{if $q=\dfrac{3}{2}$}\\[0.5em] &\dfrac{x}{2\gamma+1} &&\textrm{if $q=2$}\\[0.5em] &\operatorname{sign}(x)\dfrac{\sqrt{1+12\gamma|x|}-1}{6\gamma} &&\textrm{if $q=3$}\\[0.5em] &\sqrt[3]{\dfrac{\xi+x}{8\gamma}}-\sqrt[3]{\dfrac{\xi-x}{8\gamma}} \scriptstyle\qquad\textrm{with $\;\xi=\sqrt{x^2+1/(27\gamma)}$} &&\textrm{if $q=4$}\\[0.5em] &\operatorname{sign}(x) p \scriptstyle\quad\textrm{with $p\ge0$ such that $\quad\displaystyle\gamma \, q \, p^{q-1} + p - |x| = 0$} &&\textrm{otherwise} \end{aligned}\right.$ |
[Function]
[Prox] |
[Power] | [Chaux et al., 2007] |
Inverse | $\left\{\begin{aligned} &\dfrac{1}{x^q} &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$
(with q ≥ 1) |
$p > 0$ such that $p^{q+2}-xp^{q+1} - \gamma \, q = 0$ |
[Function]
[Prox] |
[Inverse] | [Combettes et al., 2011] |
Negative root | $\left\{\begin{aligned} &-\sqrt[q]{x} &&\textrm{if $x\ge0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$
(with q ≥ 1) |
$\left\{\begin{aligned} &\max\{0, x+\gamma\} &&\textrm{if $q=1$}\\ &\sqrt[q]{p} \scriptstyle\quad\textrm{with $p >0$ such that $\quad p^{2q-1}-xp^{q-1} - \gamma \, q^{-1} = 0$}&&\textrm{otherwise} \end{aligned}\right.$ |
[Function]
[Prox] |
[NegativeRoot] | [Combettes et al., 2011] |
Exponential | $\exp(x)$ | $x - W\Big( \gamma \exp(x) \Big)$ where $W$ denotes the Lambert W-function |
[Function]
[Prox] |
[Exp] | [Combettes et al., 2011] |
Logistic loss | $\log\Big(1+\exp(x)\Big)$ |
$x - \operatorname{W}_{\exp(x)} \Big(\gamma \exp \left(x \right) \Big)$
where $\operatorname{W}_{\theta}$ is the generalized Lambert function |
[Function]
[Prox] |
[LogisticLoss] | [Briceno-Arias et al., 2017] |
Fermi-Dirac entropy | $\left\{\begin{aligned} &x\log(x)+(1-x)\log(1-x) &&\textrm{if $0 < x < 1$}\\ &0 &&\textrm{if $x\in\{0,1\}$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
$\gamma \operatorname{W}_{\exp(\gamma^{-1}x)} \Big(\gamma^{-1} \exp \left(\gamma^{-1}x \right) \Big)$
where $\operatorname{W}_{\theta}$ is the generalized Lambert function |
[Function]
[Prox] |
[FermiDiracEntropy] | [Chierchia et al., 2017] |
Smoothed Fermi-Dirac entropy | $\left\{\begin{aligned} &x\log(x)+(1-x)\log(1-x) - \frac{1}{2}x^2 &&\textrm{if $0 < x < 1$}\\ &0 &&\textrm{if $x\in\{0,1\}$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
If $\gamma = 1$: $\dfrac{1}{1 + \exp(-x)}$ |
[Function]
[Prox] |
[SmoothedFermiDiracEntropy] | [Bauschke et al., 2017] |
Entropy | $\left\{\begin{aligned} &x\log(x) &&\textrm{if $x>0$}\\ &0 &&\textrm{if $x=0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ | $\gamma W\Big(\gamma^{-1}\exp\big(\gamma^{-1}x-1\big)\Big)$ where $W$ denotes the Lambert W-function |
[Function]
[Prox] |
[Entropy] | [Chaux et al., 2007] |
Logarithm | $\left\{\begin{aligned} &-\log(x) &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ | $\dfrac{x + \sqrt{x^2 + 4\gamma}}{2}$ |
[Function]
[Prox] |
[Log] | [Combettes et al., 2005] |
Logarithm + power | $\left\{\begin{aligned} &-\log(x) + \omega x^q &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$
(with ω>0 and q ≥ 1) |
$p>0$ such that $q \, \omega\gamma \, p^q+p^2 - xp - \gamma = 0$ |
[Function]
[Prox] |
[LogPower] | [Chaux et al., 2007] |
Logarithm + inverse | $\left\{\begin{aligned} &-\log(x) + \omega x^{-1} &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$
(with ω>0) |
$p>0$ such that $\quad p^3 -xp^2 - \gamma p -\omega\gamma = 0$ |
[Function]
[Prox] |
[LogInverse] | [Chaux et al., 2007] |
Logarithmic barriers |
$\left\{\begin{aligned} &-\log(x-a) -\log(b-x) &&\textrm{if $a < x < b$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$
(with a < b) |
$p\in\left]a,b\right[$ such that $p^3 - (a+b+x) p^2 + (ab + (a+b)x - 2\gamma)p - abx + (a+b)\gamma=0$ |
[Function]
[Prox] |
[LogBarrier] | [Chaux et al., 2007] |
Fair potential | $\omega|x|-\log(1+\omega|x|)$
(with ω>0) |
$\operatorname{sign}(x)\dfrac{\omega|x|-\omega^2\gamma-1+\sqrt{(\omega|x|-\omega^2\gamma-1)^2+4\omega|x|}}{2\omega}$ |
[Function]
[Prox] |
[FairPotential] | [Chaux et al., 2007] |
PReLu | $\left\{\begin{aligned} &0 &&\textrm{if $x > 0$}\\ &(\dfrac{1}{\alpha}+1)\dfrac{x^2}{2} &&\textrm{otherwise}\end{aligned}\right.$ |
If $\gamma = 1$: $\operatorname{PReLu}(x) = \left\{\begin{aligned} &x &&\textrm{if $x >0$}\\ &\alpha x &&\textrm{otherwise}\end{aligned}\right.$ |
[Function]
[Prox] |
[PRelu] | [P.L. Combettes and J.-C. Pesquet 2018] |
Bent Identity Activation | $\left\{\begin{aligned} &\frac{x}{2}-\frac{\ln{(x+1/2)}}{4} &&\textrm{if $x > -\frac{1}{2}$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
If $\gamma = 1$: $\frac{x+\sqrt{x^2 +1}-1}{2}$ |
[Function]
[Prox] |
[BentIdentity] | [P.L. Combettes and J.-C. Pesquet 2018] |
Inverse Square Root Unit Activation | $\left\{\begin{aligned} &-\frac{x^2}{2}-\sqrt{1-x^2} &&\textrm{if $|x| \leqslant 1$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
If $\gamma = 1$: $\frac{x}{\sqrt{1+x^2}}$ |
[Function]
[Prox] |
[ISRU] | [P.L. Combettes and J.-C. Pesquet 2018] |
Inverse Square Root Linear Unit Activation | $\left\{\begin{aligned} & 0 &&\textrm{if $x \geqslant 0$}\\&1-\frac{x^2}{2}-\sqrt{1-x^2} &&\textrm{if $-1 \leqslant x < 0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
If $\gamma = 1$: $\left\{\begin{aligned} &x &&\textrm{if $x \geqslant 0$}\\&\frac{x}{\sqrt{1+x^2}} &&\textrm{otherwise}\end{aligned}\right.$ |
[Function]
[Prox] |
[ISRLU] | [P.L. Combettes and J.-C. Pesquet 2018] |
Arctangent Activation | $\left\{\begin{aligned} &-\frac{2}{\pi}\ln{(\cos{(\frac{\pi x}{2})})}-\frac{x^2}{2} &&\textrm{if $|x| < 1$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
If $\gamma = 1$: $\frac{2}{\pi} \arctan(x)$ |
[Function]
[Prox] |
[ArctanActi] | [P.L. Combettes and J.-C. Pesquet 2018] |
Hyperbolic Tangent Activation | $ \left\{\begin{aligned} &x \operatorname{artanh}{(x)}+\frac{1}{2}(\ln{(1-x^2)}-x^2) &&\textrm{if $|x| < 1$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
If $\gamma = 1$: $ \tanh(x)$ |
[Function]
[Prox] |
[TanhActi] | [P.L. Combettes and J.-C. Pesquet 2018] |
Unimodal Sigmoid Activation | $ \left\{\begin{aligned} &(x+\frac{1}{2} )\ln{(x+\frac{1}{2})} +(\frac{1}{2}-x )\ln{(\frac{1}{2}-x)} -\frac{1}{2}(x^2+\frac{1}{4}) &&\textrm{if $|x| < \frac{1}{2}$}\\&-\frac{1}{4} &&\textrm{if $|x| = \frac{1}{2}$}\\ &+\infty &&\textrm{if $|x| > \frac{1}{2}$}\end{aligned}\right.$ |
If $\gamma = 1$: $ \frac{1}{1+\exp{(-x)}}-\frac{1}{2}$ |
[Function]
[Prox] |
[UnimodalSigmoid] | [P.L. Combettes and J.-C. Pesquet 2018] |
Elliot Activation | $ \left\{\begin{aligned} &-|x|-\ln{(1-|x|)}-\frac{x^2}{2} &&\textrm{if $|x| < 1$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ |
If $\gamma = 1$: $ \frac{x}{1+|x|}$ |
[Function]
[Prox] |
[ElliotActi] | [P.L. Combettes and J.-C. Pesquet 2018] |
Inverse Hyperbolic Sine Activation | $ \text{cosh}(x)-\frac{x^2}{2}$ |
If $\gamma = 1$: $ \text{arcsinh}(x)$ |
[Function]
[Prox] |
[ArgsinhActi] | [P.L. Combettes and J.-C. Pesquet 2018] |
Logarithmic Activation | $ \exp{(|x|)}-|x|-1-\frac{x^2}{2}$ |
If $\gamma = 1$: $ \text{sign}(x) \ln{(1+|x|)}$ |
[Function]
[Prox] |
[LogActi] | [P.L. Combettes and J.-C. Pesquet 2018] |
ELU Activation |
$\left\{\begin{aligned}
&0 &&\textrm{if $x \geqslant 0$}\\
&(x + \omega) \ln\left( \frac{x + \omega}{\omega}\right)-x - \frac{x^2}{2} &&\textrm{if $-\omega < x < 0$}\\
&\omega - \frac{\omega^2}{2} &&\textrm{if $x = - \omega$}\\
&+\infty &&\textrm{otherwise}
\end{aligned}\right.$
(with $\omega \geqslant 1$) |
If $\gamma = 1$: $\left\{\begin{aligned} &x &&\textrm{if $x \geqslant 0$}\\ &\omega (\exp(x)-1) &&\textrm{otherwise} \end{aligned}\right.$ |
(coming soon) | [ELUacti] | [P.L. Combettes and J.-C. Pesquet 2020] |
Geman McClure Activation |
$\left\{\begin{aligned}
&\mu \textrm{arctan}\sqrt{\frac{|x|}{\mu-|x|}}-\sqrt{|x|(\mu-|x|)}-\frac{x^2}{2} &&\textrm{if $|x|< \mu$}\\
& \frac{\mu(\pi - \mu)}{2} &&\textrm{if $|x|= \mu$}\\
&+\infty &&\textrm{otherwise}
\end{aligned}\right.$
with $\mu = \frac{8}{3 \sqrt{3}}$ |
If $\gamma = 1$: $\dfrac{\mu \textrm{sign}(x)x^2}{1 + x^2}$ |
(coming soon) | [GMacti] | [P.L. Combettes and J.-C. Pesquet 2020] |
Name |
f(x)
(∀ x ∈ ℝ) |
ψ(u) (∀ u ∈ ℝ) |
proxψγf(x)
(∀ γ ∈ ℝ+) |
Matlab | Python | Ref |
---|---|---|---|---|---|---|
Absolute value | $|x-\delta|$ (with δ > 0) |
$\left\{\begin{aligned} &u\log u &&\textrm{if $u>0$}\\ &0 &&\textrm{if $u=0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ | $\left\{\begin{aligned} &x \, \exp(\gamma) &&\textrm{if $x<\delta\,\exp({-\gamma})$}\\ &\delta &&\textrm{if $\delta\,\exp({-\gamma})\le x \le \delta\,\exp({\gamma})$}\\ &x \, \exp({-\gamma}) &&\textrm{otherwise} \end{aligned}\right.$ | (coming soon) | [BregAbsEntropy] | [Bauschke et al., 2017] |
Absolute value | $|x-\delta|$ (with δ > 0) |
$\left\{\begin{aligned} &-\log u &&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ | $\left\{\begin{aligned} &\frac{x}{1+\gamma x} &&\textrm{if $x<\frac{\delta}{1-\gamma\delta}$}\\ &\delta &&\textrm{if $\frac{\delta}{1-\gamma\delta}\le x \le \frac{\delta}{1+\gamma\delta}$}\\ &\frac{x}{1-\gamma x} &&\textrm{otherwise} \end{aligned}\right.$ | (coming soon) | [BregAbsLog] | [Bauschke et al., 2017] |
Exponential | $\exp(x)$ | $\exp(u)$ | $x - \log(1+\gamma)$ | (coming soon) | [BregExp] | [Bauschke et al., 2017] |
Square | $\frac{1}{2}x^2$ | $\left\{\begin{aligned} &-\log u &&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ | $\dfrac{\sqrt{1+ 4\gamma x^2}-1}{2\gamma x}$ | (coming soon) | [BregSquareLog] | [Bauschke et al., 2017] |
$\ell_p$ norm |
$\frac{1}{p}|x|^p$ (with $p \ge 1$) |
$\psi(u)=\left\{\begin{aligned} & u \log u - u&&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ | $ \left\{\begin{aligned} & \left( \frac{ W\big(\gamma (p-1) \exp({(p-1)x})\big) }{\gamma (p-1)} \right)^{\dfrac{1}{p-1}} && \textrm{if $p>1$}\\ & \exp({x - \gamma}) && \textrm{if $p=1$} \end{aligned}\right. $ | (coming soon) | [BreLpNorm] | [Combettes et al., 2016] |
Boltzmann-Shannon entropy |
$\left\{\begin{aligned}
& x \log x - w x&&\textrm{if $x>0$}\\
&+\infty &&\textrm{otherwise}
\end{aligned}\right.$
(with $w \in \mathbb{R}$) |
$\psi(u)=\left\{\begin{aligned} & u \log u - u&&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ | $\exp({\frac{x + \gamma (w - 1)}{\gamma + 1}})$ | (coming soon) | [BregBoltzShannon] | [Combettes et al., 2016] |
Boltzmann-Shannon entropy |
$\left\{\begin{aligned}
& x \log x - w x&&\textrm{if $x>0$}\\
&+\infty &&\textrm{otherwise}
\end{aligned}\right.$
(with $w \in \mathbb{R}$) |
$\psi(u)=\left\{\begin{aligned} & u \log u + (1-u) \log(1-u) &&\textrm{if $u\in ]0,1[$}\\ & 0 && \textrm{if $u\in \{0,1 \}$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ | $-\frac{1}{2}\exp({x + w - 1}) + \sqrt{\frac{1}{4}\exp\big({2 (x + w - 1)}\big) + \exp({x + w - 1})} $ | (coming soon) | [BregBoltzShannon2] | [Combettes et al., 2016] |