datarekha

Lipschitz & One-Insight Problems

Some GATE calculus problems look brutal but turn on a single idea. The signature one: a squared bound |f(x) − f(y)| ≤ C(x − y)² secretly forces f to be constant.

7 min read Advanced GATE DA Lesson 46 of 122

What you'll learn

  • How to spot a 'looks hard, one idea' calculus problem on the exam
  • A squared bound |f(x) − f(y)| ≤ C(x − y)² divides down to force f'(x) = 0 everywhere
  • f' ≡ 0 on an interval ⇒ f is constant, so any difference f(b) − f(a) = 0
  • Why the SQUARE matters: a linear bound |f(x) − f(y)| ≤ C|x − y| only bounds the slope, it does NOT force constancy

Before you start

Convexity closed with a challenge: when the exam hands you a fresh problem and never says which tool to use, how do you find it fast? This last calculus lesson is the sharpest version of that challenge. Every year, GATE slips in a question that looks brutal on first read — an abstract function f with no formula, a strange-looking inequality, and a request for one specific number. There is nothing to differentiate, nothing to plug in. It seems unfair.

It is not. These problems are one-trick ponies: each hinges on a single insight that, once you see it, collapses the whole thing to a one-line answer. This lesson teaches the most famous example — and, more usefully, how to recognise the species on sight, so the next abstract f does not rattle you.

The star concept here, the Lipschitz bound (a cap on how fast a function can change), is not mere exam trivia: bounding a function’s or a gradient’s Lipschitz constant is how modern machine learning keeps training stable — it underpins WGAN’s weight clipping, certified robustness bounds, and the convergence guarantees for gradient descent.

The signature problem: a squared bound

Here is the setup that has appeared more than once. You are told a function satisfies

|f(x) − f(y)| ≤ C · (x − y)²      for all real x, y

for some constant C. There is no formula for f. The question asks for something like f(1) − f(0). It looks hopelessly underspecified — but the squared (x − y)² is quietly doing all of the work.

The one idea: divide, then take the limit

The derivative of f at a point x is the limit of the difference quotient — the very definition from the differentiability lesson. So look at that quotient and feed it the bound. For y ≠ x, divide both sides by |x − y|:

| f(x) − f(y) |
| ----------- |  ≤  C · |x − y|
|    x − y    |

The left side is the difference quotient whose limit (as y → x) is |f'(x)|. The right side C·|x − y| goes to 0 as y → x. Squeezed between, the slope has nowhere to go:

0 ≤ |f'(x)| ≤ lim (C·|x − y|) = 0      ⇒      f'(x) = 0   for every x

A function whose derivative is zero everywhere is constant. So f takes the same value at every point, and therefore any difference vanishes:

f'(x) = 0 everywhere   ⇒   f is constant   ⇒   f(1) − f(0) = 0
slope 0 everywhere → f is constantx = 0x = 1f(1) − f(0) = 0
A zero-slope function is a flat line — every value equals every other, so all differences are 0.

How GATE asks this

Almost always a NAT whose answer is a clean 0. The wrapper changes — it might ask for f(2) − f(−3), or “how many such non-constant f exist?”, or hand you the bound as |f(x) − f(y)| ≤ 5(x − y)² — but the engine never changes: divide by |x − y|, let y → x, conclude f' ≡ 0, so f is constant and the difference is 0. The skill GATE is really testing is recognition: an abstract f plus a squared (or higher-power) bound is the tell, every time.

Worked example — a real GATE DA question

A function f satisfies |f(x) − f(y)| ≤ (x − y)² for all real x, y. Find f(1) − f(0).

Run the pattern. Fix x, take any y ≠ x, and divide the given bound by |x − y|:

| f(x) − f(y) |
| ----------- |  ≤  |x − y|
|    x − y    |

Now let y → x. The left side tends to |f'(x)| (the definition of the derivative), and the right side |x − y| → 0. By the squeeze,

|f'(x)| ≤ 0   ⇒   f'(x) = 0      for every x.

So f has zero slope everywhere and is therefore constant. A constant function takes the same value at 1 and at 0, hence

f(1) − f(0) = 0.

The answer is 0 — daunting on first read, a single line once you spot that the squared bound kills the derivative. That recognition, not the algebra, is the mark.

A question to carry forward

That closes the calculus chapter. Step back and notice what every lesson in it shared: a world of smooth, continuous change — curves that bend, slopes that vary by infinitesimal amounts, quantities approached but never quite reached. It is the mathematics of the analog world. Yet the data scientist’s daily instrument is nothing like a smooth curve. It is a handful of lines of code that must run exactly — discrete, literal, and unforgiving of a single off-by-one. The next chapter changes register completely, from the continuous to the computational. Here is the thread onward: when you are handed a short Python snippet and asked what it prints, how do you read it the way the machine does — tracing each step precisely enough to predict the output character for character, with no rounding and no “approximately”?

In one breath

  • Some GATE calculus problems are one-insight: an abstract f, a strange bound, and a single idea that collapses it to a one-line answer.
  • The signature: |f(x) − f(y)| ≤ C(x − y)² ⇒ divide by |x − y|, let y → x, get |f'(x)| ≤ 0f' ≡ 0f constant ⇒ any difference like f(1) − f(0) = 0.
  • The answer is almost always a clean 0, and the constant C never matters.
  • The square is everything: a linear bound ≤ C|x − y| only gives |f'| ≤ C (bounded slope, Lipschitz) — f(x) = x meets it and is not constant. It needs power > 1.
  • The real skill is recognition: abstract f + squared (or higher) bound ⇒ reach for divide-and-limit.

Practice

Quick check

0/6
Q1Recall: why does the bound |f(x) − f(y)| ≤ C(x − y)² force f to be constant? Which statements are correct? (select all that apply)select all that apply
Q2Trace: f satisfies |f(x) − f(y)| ≤ (x − y)² for all real x, y. Find f(1) − f(0). (NAT)numerical answer — type a number
Q3Apply: f satisfies |f(x) − f(y)| ≤ 7(x − y)² for all x, y. Find f(2) − f(−3). (NAT)numerical answer — type a number
Q4Apply: how many NON-constant functions f satisfy |f(x) − f(y)| ≤ (x − y)² for all real x, y? (NAT)numerical answer — type a number
Q5Apply: a function g satisfies |g(x) − g(y)| ≤ 3|x − y| for all x, y. Which statements are correct? (select all that apply)select all that apply
Q6Create: which feature of the bound |f(x) − f(y)| ≤ C(x − y)^p is responsible for forcing f to be constant?

Sign in to track your progress

Completed lessons, your XP, level, and streak save to your account — it's free and takes a few seconds.

Practice this in an interview

All questions
Why does sigmoid saturation cause vanishing gradients, and why is tanh only a partial fix?

Sigmoid's derivative peaks at 0.25 and approaches zero in both tails, so the chain of gradient multiplications collapses exponentially in deep networks. Tanh's derivative peaks at 1 and is zero-centered, which helps weight update symmetry, but it still saturates at large magnitudes and the gradient still shrinks to near-zero in both tails.

What does the Universal Approximation Theorem guarantee — and what doesn't it guarantee?

The theorem proves that a single-hidden-layer network with enough neurons and a non-linear activation can approximate any continuous function on a compact domain to arbitrary precision. It guarantees existence, not learnability — it says nothing about how many neurons are needed, whether gradient descent will find the solution, or how the network will generalize.

Compare sigmoid, tanh, ReLU, leaky ReLU, and GELU — when would you pick each?

Sigmoid squashes to (0,1) and saturates at extremes, causing vanishing gradients. Tanh is zero-centered but still saturates. ReLU avoids saturation for positive inputs and trains fast but can produce dead neurons. Leaky ReLU fixes dying neurons. GELU is smooth and probabilistic, now the default in most transformer architectures.

How do LSTM gates solve the vanishing gradient problem?

An LSTM maintains a cell state that flows through time via additive updates controlled by learned gates, giving gradients a near-linear path across many steps. The forget, input, and output gates let the network selectively retain, write, and expose information rather than crushing every signal through a squashing non-linearity at every step.

Related lessons

Explore further

Skip to content