Functions of a scalar variable

Standard - Bregman

Standard

Name f(x)

(∀ x ∈ ℝ)

proxγf(x)

(∀ γ ∈ ℝ+)

Matlab Python Ref
Support function $\sigma_{[a,b]}(x)=\left\{\begin{aligned} &a x &&\textrm{if $x < 0$}\\&0 &&\textrm{if $x = 0$}\\ &bx &&\textrm{if $x > 0$}\end{aligned}\right.$

(with a < b)

$\operatorname{soft}_{[a\gamma,b\gamma ]}(x) = \left\{\begin{aligned} &x-a\gamma &&\textrm{if $x < a\gamma$}\\&0 &&\textrm{if $a\gamma\le x \le b\gamma$}\\ &x-b\gamma &&\textrm{otherwise}\end{aligned}\right.$ [Function]
[Prox]
[Class] [Combettes et al., 2008]
Absolute value $\sigma_{[-1,1]}(x) =|x|$ $\operatorname{soft}_{[-\gamma,\gamma]}(x)$ [Function]
[Prox]
[Class] [Donoho, 1995]
Hinge loss $\sigma_{[0,1]}(x) = \max\{x,0\}$ $\operatorname{soft}_{[0,\gamma]}(x)$ [Function]
[Prox]
[Class] [Combettes et al., 2008]
$\omega$-insensitive loss $\max\{|x|-\omega,0\}$

(with ω>0)

$\left\{ \begin{aligned} &x &&\textrm{if $|x| < \omega$}\\ &\omega\operatorname{sign}(x) &&\textrm{if $\omega\le|x| \le \omega+\gamma$}\\ &x - \gamma\operatorname{sign}(x) &&\textrm{otherwise} \end{aligned} \right.$ [Function]
[Prox]
[Class] [Combettes et al., 2011]
Square $\dfrac{1}{2}x^2$ $\dfrac{x}{\gamma+1}$ [Function]
[Prox]
(coming soon) [Moreau, 1965]
Squared hinge loss $\dfrac{1}{2}\big(\max\{x,0\}\big)^2$ $\left\{ \begin{aligned} &x &&\textrm{if $x \le 0$}\\ &\dfrac{x}{\gamma+1} &&\textrm{otherwise} \end{aligned} \right.$ [Function]
[Prox]
(coming soon)
Squared $\omega$-insensitive loss $\dfrac{1}{2}\big(\max\{|x|-\omega,0\}\big)^2$

(with ω>0)

$\left\{ \begin{aligned} &x &&\textrm{if $|x|\le\omega$}\\ &\dfrac{x+\omega\gamma\operatorname{sign}(x)}{\gamma+1} &&\textrm{otherwise} \end{aligned} \right.$ [Function]
[Prox]
(coming soon)
Huber $\left\{ \begin{aligned} &\frac{1}{2} x^2 &&\textrm{if $|x| \le \omega$}\\ &\omega |x|-\frac{\omega^2}{2} &&\textrm{otherwise} \end{aligned} \right.$

(with ω>0)

$\left\{ \begin{aligned} &\dfrac{x}{\gamma+1} &&\textrm{if $|x| \le \omega(\gamma+1)$}\\ &x - \omega\gamma \operatorname{sign}(x) &&\textrm{otherwise} \end{aligned} \right.$ [Function]
[Prox]
(coming soon) [Combettes et al., 2007]
Power $|x|^q$

(with q ≥ 1)

$\left\{\begin{aligned} &\operatorname{sign}(x) \max\{0, |x| - \gamma\} &&\textrm{if $q=1$}\\[0.5em] &x + \dfrac{4\gamma}{3\sqrt[3]{2}}\left(\sqrt[3]{\xi-x}-\sqrt[3]{\xi+x}\right) \scriptstyle\quad\textrm{with $\; \xi=\sqrt{x^2+256\gamma^3/729}$} &&\textrm{if $q=\dfrac{4}{3}$}\\[0.5em] &x + \frac{9}{8}\gamma^2\operatorname{sign}(x)\left(1-\sqrt{1+\frac{16|x|}{9\gamma^2}}\right) &&\textrm{if $q=\dfrac{3}{2}$}\\[0.5em] &\dfrac{x}{2\gamma+1} &&\textrm{if $q=2$}\\[0.5em] &\operatorname{sign}(x)\dfrac{\sqrt{1+12\gamma|x|}-1}{6\gamma} &&\textrm{if $q=3$}\\[0.5em] &\sqrt[3]{\dfrac{\xi+x}{8\gamma}}-\sqrt[3]{\dfrac{\xi-x}{8\gamma}} \scriptstyle\qquad\textrm{with $\;\xi=\sqrt{x^2+1/(27\gamma)}$} &&\textrm{if $q=4$}\\[0.5em] &\operatorname{sign}(x) p \scriptstyle\quad\textrm{with $p\ge0$ such that $\quad\displaystyle\gamma \, q \, p^{q-1} + p - |x| = 0$} &&\textrm{otherwise} \end{aligned}\right.$ [Function]
[Prox]
(coming soon) [Chaux et al., 2007]
Inverse $\left\{\begin{aligned} &\dfrac{1}{x^q} &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with q ≥ 1)

$p > 0$

such that

$p^{q+2}-xp^{q+1} - \gamma \, q = 0$
[Function]
[Prox]
(coming soon) [Combettes et al., 2011]
Negative root $\left\{\begin{aligned} &-\sqrt[q]{x} &&\textrm{if $x\ge0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with q ≥ 1)

$\left\{\begin{aligned} &\max\{0, x+\gamma\} &&\textrm{if $q=1$}\\ &\sqrt[q]{p} \scriptstyle\quad\textrm{with $p >0$ such that $\quad p^{2q-1}-xp^{q-1} - \gamma \, q^{-1} = 0$}&&\textrm{otherwise} \end{aligned}\right.$ [Function]
[Prox]
(coming soon) [Combettes et al., 2011]
Exponential $\exp(x)$ $x - W\Big( \gamma \exp(x) \Big)$

where $W$ denotes the Lambert W-function

[Function]
[Prox]
(coming soon) [Combettes et al., 2011]
Logistic loss $\log\Big(1+\exp(x)\Big)$ $x - \operatorname{W}_{\exp(x)} \Big(\gamma \exp \left(x \right) \Big)$

where $\operatorname{W}_{\theta}$ is the generalized Lambert function

[Function]
[Prox]
(coming soon) [Briceno-Arias et al., 2017]
Fermi-Dirac entropy $\left\{\begin{aligned} &x\log(x)+(1-x)\log(1-x) &&\textrm{if $0 < x < 1$}\\ &0 &&\textrm{if $x\in\{0,1\}$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ $\gamma \operatorname{W}_{\exp(\gamma^{-1}x)} \Big(\gamma^{-1} \exp \left(\gamma^{-1}x \right) \Big)$

where $\operatorname{W}_{\theta}$ is the generalized Lambert function

[Function]
[Prox]
(coming soon) [Chierchia et al., 2017]
Smoothed Fermi-Dirac entropy $\left\{\begin{aligned} &x\log(x)+(1-x)\log(1-x) - \frac{1}{2}x^2 &&\textrm{if $0 < x < 1$}\\ &0 &&\textrm{if $x\in\{0,1\}$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ If $\gamma = 1$:

$\dfrac{1}{1 + \exp(-x)}$
[Function]
[Prox]
(coming soon) [Bauschke et al., 2017]
Entropy $\left\{\begin{aligned} &x\log(x) &&\textrm{if $x>0$}\\ &0 &&\textrm{if $x=0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ $\gamma W\Big(\gamma^{-1}\exp\big(\gamma^{-1}x-1\big)\Big)$

where $W$ denotes the Lambert W-function

[Function]
[Prox]
(coming soon) [Chaux et al., 2007]
Logarithm $\left\{\begin{aligned} &-\log(x) &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$ $\dfrac{x + \sqrt{x^2 + 4\gamma}}{2}$ [Function]
[Prox]
(coming soon) [Combettes et al., 2005]
Logarithm + power $\left\{\begin{aligned} &-\log(x) + \omega x^q &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with ω>0 and q ≥ 1)

$p>0$

such that

$q \, \omega\gamma \, p^q+p^2 - xp - \gamma = 0$
[Function]
[Prox]
(coming soon) [Chaux et al., 2007]
Logarithm + inverse $\left\{\begin{aligned} &-\log(x) + \omega x^{-1} &&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with ω>0)

$p>0$

such that

$\quad p^3 -xp^2 - \gamma p -\omega\gamma = 0$
[Function]
[Prox]
(coming soon) [Chaux et al., 2007]
Logarithmic barriers $\left\{\begin{aligned} &-\log(x-a) -\log(b-x) &&\textrm{if $a < x < b$}\\ &+\infty &&\textrm{otherwise}\end{aligned}\right.$

(with a < b)

$p\in\left]a,b\right[$

such that

$p^3 - (a+b+x) p^2 + (ab + (a+b)x - 2\gamma)p - abx + (a+b)\gamma=0$
[Function]
[Prox]
(coming soon) [Chaux et al., 2007]
Fair potential $\omega|x|-\log(1+\omega|x|)$

(with ω>0)

$\operatorname{sign}(x)\dfrac{\omega|x|-\omega^2\gamma-1+\sqrt{(\omega|x|-\omega^2\gamma-1)^2+4\omega|x|}}{2\omega}$ [Function]
[Prox]
(coming soon) [Chaux et al., 2007]
Sigmoid I $\left\{ \begin{aligned} &-\sqrt{1 - x^2} - \frac{1}{2}x^2 &&\textrm{if $|x| \le 1$}\\ &+ \infty &&\textrm{otherwise} \end{aligned} \right.$ If $\gamma = 1$:

$\dfrac{x}{\sqrt{1 + x^2}}$
[Function]
[Prox]
(coming soon) [Bauschke et al., 2017]
Sigmoid II $\left\{ \begin{aligned} & x\text{arctanh}^{-1}(x) + \frac{1}{2} \left(\log(1-x^2)-x^2\right)&&\textrm{if $|x| \le 1$}\\ &+ \infty &&\textrm{otherwise} \end{aligned} \right.$ If $\gamma = 1$:

$\text{tanh}(x)$
[Function]
[Prox]
(coming soon) [Bauschke et al., 2017]
Sigmoid III $\left\{ \begin{aligned} & -\frac{2}{\pi} \log \left( \cos (\frac{\pi}{2} x) \right) - \frac{1}{2} x^2 &&\textrm{if $|x| \le 1$}\\ &+ \infty &&\textrm{otherwise} \end{aligned} \right.$ If $\gamma = 1$:

$\frac{2}{\pi}\text{arctan}(x)$
[Function]
[Prox]
(coming soon) [Bauschke et al., 2017]

Bregman

Name f(x)

(∀ x ∈ ℝ)

ψ(u)

(∀ u ∈ ℝ)


proxψγf(x)

(∀ γ ∈ ℝ+)

Matlab Python Ref
Absolute value $|x-\delta|$

(with δ > 0)

$\left\{\begin{aligned} &u\log u &&\textrm{if $u>0$}\\ &0 &&\textrm{if $u=0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $\left\{\begin{aligned} &x \, \exp(\gamma) &&\textrm{if $x<\delta\,\exp({-\gamma})$}\\ &\delta &&\textrm{if $\delta\,\exp({-\gamma})\le x \le \delta\,\exp({\gamma})$}\\ &x \, \exp({-\gamma}) &&\textrm{otherwise} \end{aligned}\right.$ (coming soon) (coming soon) [Bauschke et al., 2017]
Absolute value $|x-\delta|$

(with δ > 0)

$\left\{\begin{aligned} &-\log u &&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $\left\{\begin{aligned} &\frac{x}{1+\gamma x} &&\textrm{if $x<\frac{\delta}{1-\gamma\delta}$}\\ &\delta &&\textrm{if $\frac{\delta}{1-\gamma\delta}\le x \le \frac{\delta}{1+\gamma\delta}$}\\ &\frac{x}{1-\gamma x} &&\textrm{otherwise} \end{aligned}\right.$ (coming soon) (coming soon) [Bauschke et al., 2017]
Exponential $\exp(x)$ $\exp(u)$ $x - \log(1+\gamma)$ (coming soon) (coming soon) [Bauschke et al., 2017]
Square $\frac{1}{2}x^2$ $\left\{\begin{aligned} &-\log u &&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $\dfrac{\sqrt{1+ 4\gamma x^2}-1}{2\gamma x}$ (coming soon) (coming soon) [Bauschke et al., 2017]
$\ell_p$ norm $\frac{1}{p}|x|^p$

(with $p \ge 1$)

$\psi(u)=\left\{\begin{aligned} & u \log u - u&&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $ \left\{\begin{aligned} & \left( \frac{ W\big(\gamma (p-1) \exp({(p-1)x})\big) }{\gamma (p-1)} \right)^{\dfrac{1}{p-1}} && \textrm{if $p>1$}\\ & \exp({x - \gamma}) && \textrm{if $p=1$} \end{aligned}\right. $ (coming soon) (coming soon) [Combettes et al., 2016]
Boltzmann-Shannon entropy $\left\{\begin{aligned} & x \log x - w x&&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$

(with $w \in \mathbb{R}$)

$\psi(u)=\left\{\begin{aligned} & u \log u - u&&\textrm{if $u>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $\exp({\frac{x + \gamma (w - 1)}{\gamma + 1}})$ (coming soon) (coming soon) [Combettes et al., 2016]
Boltzmann-Shannon entropy $\left\{\begin{aligned} & x \log x - w x&&\textrm{if $x>0$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$

(with $w \in \mathbb{R}$)

$\psi(u)=\left\{\begin{aligned} & u \log u + (1-u) \log(1-u) &&\textrm{if $u\in ]0,1[$}\\ & 0 && \textrm{if $u\in \{0,1 \}$}\\ &+\infty &&\textrm{otherwise} \end{aligned}\right.$ $-\frac{1}{2}\exp({x + w - 1}) + \sqrt{\frac{1}{4}\exp\big({2 (x + w - 1)}\big) + \exp({x + w - 1})} $ (coming soon) (coming soon) [Combettes et al., 2016]