Math for Non-Geeks/ Lipschitz continuity

{{#invoke:Math for Non-Geeks/Seite|oben}}

Lipschitz continuity is an even stronger form of uniform continuity and is often used in the theory of differential equations.

Derivation

By introducing the notion of Lipschitz continuity, we will be able to say more about the rate at which a function changes. As we already know, continuous functions have the neat property that sufficiently small changes in the arguments will result in arbitrary small changes in the function value. On top of that, Lipschitz continuity allows us give an estimate of that change. We can thus talk very precisely about how "fast" the changes of the output become smaller. For better understanding, let us first recap what we mean by changes of a function. Let $f:D\to ℝ$ be an function with domain $D$.

Let's take two arbitrary points $\tilde{x}$ and $x$ from the domain of $f$ and form the straight line through the points $(\tilde{x},f(\tilde{x}))$ und $(x,f(x))$. The slope of the line increases, as the difference between the values $f(\tilde{x})$ and $f(x)$ gets bigger. The mean rate of change between function values is equal to the slope $\frac{f(x)-f(\tilde{x})}{x-\tilde{x}}$ of the secant through those points and can be calculated with the so-called slope triangle:

Steigungsdreieck zum Bestimmen der Sekantensteigung

Let's assume now that the rate of change is bounded, i.e. the slopes of the secants cannot get arbitrary large or small. Thus the absolute value $\left|\frac{f(x)-f(\tilde{x})}{x-\tilde{x}}\right|$ has an upper bound (because the absolute value is bounded, both positive and negative values are bounded). There exists a constant $L\in ℝ$, so that for all $\tilde{x},x\in D$ with $x\ne \tilde{x}$ the inequality $\left|\frac{f(x)-f(\tilde{x})}{x-\tilde{x}}\right|\le L$ holds: This number $L$ is called the Lipschitz constant. By multiplying both sides of the equation with $|x-\tilde{x}|$, we get:

This inequality $|f(x)-f(\tilde{x})|\le L|x-\tilde{x}|$ is the basis for the definition of Lipschitz continuity. If we can find such a constant $L$ that satisfies this inequality for all $x,\tilde{x}\in R$, then we have also found an absolute bound for the rate of change of the function. Because the inequality $|f(x)-f(\tilde{x})|\le L|x-\tilde{x}|$ is still true even for $x=\tilde{x}$ this will result in a nice simplification of the definition, because we can drop the assumption that $x\ne \tilde{x}$, which we otherwise would have needed, had we tried to work with the original mean rate of change $\frac{f(x)-f(\tilde{x})}{x-\tilde{x}}$.

Definition

More formally the definition could be written using quantifiers:

The right-hand side of the above equivalence can be translated as follows:

Why do we need Lipschitz continuity?

The Lipschitz constant of a Lipschitz continuous functions gives us an absolute upper bound for the rate of change. This is useful if we want to estimate function values.

Let us assume we are given a point $x\in D$ from the domain and the corresponding function value $f(x)$. Now imagine you want to estimate the value $f(y)$ for a new point $y\in D$. We can obtain such an estimate if we use our Lipschitz constant to find an upper and lower bound for $f(y)$. From Lipschitz continuity it follows that:

By adding $f(x)$ we obtain:

This way we have found a lower and upper bound and know that $f(y)$ must be located somewhere in between.

Intuition

Picturing Lipschitz continuity with cones

Let's try to visualize the Lipschitz condition: Given a Lipschitz continuous function

$f:D\to ℝ$

we draw the straight lines

$g_1$

and

$g_2$

through the point

$(\tilde{x},f(\tilde{x}))$

and make their slope be equal to our Lipschitz constant

$L$

and

$-L$

respectively. Above we already have seen that the Lipschitz condition is given by

$\frac{f(x)-f(\tilde{x})}{x-\tilde{x}}\le L$

, which means that the slope of all secants through

$(\tilde{x},f(\tilde{x}))$

is bounded. This means that the graph of the function must run between those two lines:

Visualisierung Lipschitz

This constraints is true for all $\tilde{x}$ in the domain. In the picture we could let the lines move along the graph of $f$ and in turn the graph of $f$ will always lie between those two lines:

Lipschitz Animation

If the function is differentiable the derivative of the function taken at a point will match the slope of the tangent at that point. Using this visualization we can see that the derivative of a Lipschitz continuous function cannot get bigger than $L$ (or less than $-L$). This means that the absolute value of the derivative is bounded by the Lipschitz constant.

Difference between Lipschitz and uniform continuity

With uniform continuous functions to every $ϵ>0$ we can find a $\delta >0$, so that the function values lie in the $ϵ$-$\delta$-rectangle. But this rectangle only allows for a worse estimation compared to the Lipschitz inequality.

Both the Lipschitz and uniform continuity are so-called global properties of a function (normal continuity is only a local property!). In contrast to Lipschitz continuity, uniform continuity gives no information about the rate of change. This could be a disadvantage if you are trying to estimate a value $f(y)$ from another value $f(x)$ .

Lipschitz continuity implies a linear relationship between the distance of the arguments $y-x$ and the output values $f(y)-f(x)$. This is not the case with uniform continuity because the constraint $|f(y)-f(x)|\le ϵ$ is the same for all $x,y$ with $|y-x|\le \delta$. Hence the cones of Lipschitz continuity that become progressively smaller as we approach $x$ give us a much better estimate than the „$ϵ$-$\delta$-rectangle“ of uniform continuity. We could try choose $ϵ$ arbitrary small, to improve the approximation of $f(y)$. But this could mean that our $\delta$ values must become really small too, i.e. we have to be really close the the original point $x$. You can observe this phenomenon by studying the behaviour of the square-root function near $x=0$.

Link to uniform continuity

How is this new notion of Lipschitz continuity linked to the old concepts of normal and uniform continuity? We already made the claim that Lipschitz continuity is stronger than uniform continuity: Every Lipschitz continuous function is also uniform continuos. But the converse is not generally true.

Not all uniformly continuous functions are Lipschitz continuous

To show this fact we only need to find a counterexample, i.e. a function which we know is uniformly continuous, but not Lipschitz continuous. We will see that the square root function in ${ℝ}_0^{+}$ is such a counterexample. It was already proven previously that this function is uniformly continuous. We now will show that it is not Lipschitz continuous. Thus consider the function:

We will give a prove by contradiction. Let's assume this function is Lipschitz continuous. By definition there must be a constant $L\ge 0$, so that for all $x,y\in {ℝ}_0^{+}$ the following holds: $|\sqrt{x}-\sqrt{y}|<L\cdot |x-y|$. But this would have further implications for all $x,y\in {ℝ}_0^{+}$ with $x>0$ or $y>0$:

But by choosing $x=\frac{1}{4L^2}>0$ and $y=0$, we see that:

This is a contradiction! Therefore the assumption that $f$ is Lipschitz continuous in ${ℝ}_0^{+}$ must be false.

Lipschitz continuity implies (normal) continuity

From the previous section we see that Lipschitz continuity implies (normal) continuity: Since Lipschitz continuous functions are uniformly continuous, they are also continuous (remember that uniform continuity is stronger than normal continuity). To view it from a different angle, we can ask ourself why a discontinuous function cannot be Lipschitz continuous:

Let's re-evoke the first intuitive image we had about continuity: A function is continuous if there aren't any "jumps" or "gaps":

Sprungstelle

The pictured function is discontinuous at $\tilde{x}=1$. Imagine tracing a straight line through the point at the $\tilde{x}$-value of the graph, and through another point $x>\tilde{x}$. This line is a secant of that graph.

Sekante an der Sprungstelle

If you imagine moving the right intersection point of that secant closer and closer to the point $x$ which is the jump point, you will observe that the slope of the secant gets bigger and bigger and will eventually reach infinity. This means that there cannot be any constant $L\in ℝ$ that will bound the absolute value of the slope of the secant. If you think you have found such a constant $L$ just move the intersection point of the secant even closer to $x$, until you will eventually reach a point where the slope $\frac{f(x)-f(\tilde{x})}{x-\tilde{x}}$ will be greater than $L$.

So now it should be clear why every discontinuous function cannot be Lipschitz continuous. You can reverse this proposition logically by contraposition and obtain the original proposition that Lipschitz continuity implies (normal) continuity. In summary, functions which rate of change is bounded are Lipschitz continuous and cannot have any points of discontinuity.

Sekante einer Funktion ohne Sprungstelle

A useful criterion for Lipschitz continuity

The Mean value theorem can be used to show that on a compact interval, every continuous differentiable function is Lipschitz continuous:

The derivative being bounded by $|f^{\prime }(x)|\le L$ intuitively corresponds to having a maximum steepness, which in turn implies Lipschitz continuity. However, there are non-differentiable functions with maximum steepness, so Lipschitz continuity actually requires less than being differentiable and classifies more functions.

{{#invoke:Math for Non-Geeks/Seite|unten}}