Convexity & Single-Variable Optimization

The last lesson ended on an anxiety. Finding a minimum has so far meant solving f'(x) = 0 and hoping the dip you land in is the lowest one — and on a wiggly function with several valleys, “local minimum” is the most you can honestly claim. Convexity is the property that makes the anxiety vanish. A convex function is one single bowl, so the first dip you find is the dip; there is nothing lower hiding elsewhere. It is the quiet reason so much of machine learning is built on convex loss functions.

What convex means

Picture a bowl. A function is convex if its graph lies below (or on) any chord — pick two points on the curve, draw the straight segment between them, and the curve never pokes above that segment. A non-convex curve has wiggles: it rises above some chords and dips into several separate valleys, each one a trap an optimiser could fall into.

The test you actually use: the second derivative

The chord definition is the meaning, but for a twice-differentiable function the working test is much simpler — and it reuses the curvature idea from two lessons back. A curve bends upward exactly when its slope is increasing, which is to say when the second derivative is non-negative:

f is convex  ⇔  f''(x) ≥ 0  for all x

f''(x) > 0 for all x   ⇒   f is strictly convex

So convexity is a statement about f'' everywhere, not at one lonely point. A parabola x² has f'' = 2 > 0, so it is strictly convex; so is eˣ (f'' = eˣ > 0). A straight line has f'' = 0, so it is convex but not strictly so.

Why convexity is the prize

Two consequences make convex functions the ones optimisers love:

• Any local minimum is a global minimum. With no rival valleys, the moment you find a dip you are done — there is nothing lower hiding elsewhere. Gradient descent cannot get trapped in a bad local minimum, because there are none to be trapped in. This is the exact property the last lesson wished for.
• A strictly convex function has at most one minimizer. Two distinct lowest points would force a flat-or-bulging stretch between them, contradicting strict convexity. So the solution, if it exists, is unique.

This is precisely why linear regression (squared loss) and logistic regression have convex objectives: a single global optimum, reachable by simple descent, with no fear of a worse valley around the corner.

Watch it happen. Drop the marker anywhere on the convex bowl below and run gradient descent — wherever you start, the path slides to the same point at the bottom. No “bad starts,” no second valleys to fall into. That single picture is the convexity payoff made concrete.

How GATE asks this

The signature question is an MSQ: “f is twice differentiable with f''(x) > 0 for all x — which of the following are always true?” You must tick the genuine consequences (at most one stationary point; any local min is global) and reject the tempting “f has a minimum,” which the eˣ counterexample kills. Occasionally it appears as an MCQ asking for the single guaranteed property, or a NAT pinning down the unique minimizer of a convex quadratic.

Worked example — a real GATE DA 2025 question

Let f be twice differentiable with f''(x) > 0 for all real x. Which of the following statements are always true?

1. f is strictly convex.
2. f has at most one point where f'(x) = 0.
3. Any local minimum of f is a global minimum.
4. f has a minimum.

Take the verdicts one at a time, the way the always-true drills taught you.

• (1) TRUE. f'' > 0 everywhere is the definition of strictly convex.
• (2) TRUE. f'' > 0 means f' is strictly increasing, so f' can cross zero at most once. Hence at most one stationary point.
• (3) TRUE. For a convex function every local minimum is global — no lower valley can exist elsewhere.
• (4) FALSE — the trap. Strict convexity does not force a minimum to exist. Take f(x) = eˣ: here f'' = eˣ > 0 for all x, so it satisfies the hypothesis, yet f' = eˣ is never zero and f has no minimum (it slides toward 0 as x → −∞ without ever attaining it).

So the always-true statements are (1), (2), and (3); statement (4) is false. This is a real GATE DA 2025 MSQ, and “f has a minimum” is precisely the distractor most students tick.

A question to carry forward

That completes the calculus toolkit — limits and how to compute them, the derivative and its rules, Taylor series, critical points, the second-derivative test, global optimisation on an interval, and now convexity. But notice something about every worked example so far: the problem told you which tool to use. The real exam does not. It hands you an unfamiliar expression and a single blank, and the first — often hardest — step is simply recognising which idea unlocks it: a conjugate here, a Taylor expansion there, an endpoint check, a convexity argument. Here is the thread into the final lesson of this chapter: how do you read a fresh problem and reach for the right instrument fast, under exam pressure, when nobody labels it for you?

In one breath

• Convex = graph stays below every chord (one bowl); for twice-differentiable f this is exactly f''(x) ≥ 0 everywhere. f'' > 0 everywhere ⇒ strictly convex.
• The prize: for a convex f, any local min is global; for a strictly convex f, the minimizer is unique (at most one).
• The trap: convex does not imply a minimum exists — eˣ is strictly convex yet has none (the real GATE DA 2025 distractor).
• Strict vs non-strict: f'' = 0 stretches are allowed in plain convexity (a line, a flat valley) but forbidden in strict convexity — strictness is what buys uniqueness.
• It is why squared-loss and logistic regression are easy to optimise: one global optimum, reachable by plain descent.

Practice