Why 450 / 500 gives a thinner likelihood than 45 / 50

Same hit rate 0.90. Six views, from most visual to most compact. Likelihood only, no prior.

500 trials, 450 hits. The first 50 trials hold 45 hits. Curve = likelihood L(\theta)\propto\theta^{x}(1-\theta)^{n-x}, scaled to area 1.

Speed

Likelihood of hit rate \theta: L(\theta)\propto\theta^{x}(1-\theta)^{n-x}. Curves are scaled to area 1. Two data sets, same hit rate, different n.

Data A: n

Data B: n

Hit rate (both)

Presets:

1. Likelihood, scaled to area 1

A B. Bars under the axis = central 95% interval. Dotted line = probe \theta = hit rate - 0.10.

2. Why: log-likelihood relative to the peak

\log L(\theta)-\log L(\hat\theta). Near the peak this is a parabola with curvature n/\bigl(\hat\theta(1-\hat\theta)\bigr). B has curvature n_B/n_A times larger, so it drops faster.

Term used everywhere: "likelihood", L(\theta)\propto\theta^{x}(1-\theta)^{n-x}. Each arrow states its reason.

A. From data to width

B. The same chain with your two data sets

The whole argument as one chain. Click a step to open its detail steps. Each arrow says why the next step is needed. Use Read the full section to jump to the long text in tab D.

1. Both data sets give the same best estimate: \hat\theta=\tfrac{45}{50}=\tfrac{450}{500}=0.9.
2. Only the width of the likelihood differs. 500 trials give a much narrower curve.
3. Plan: (a) find the peak, (b) measure how sharp the peak is, (c) turn the sharpness into a width.
Because we first need to say exactly what the curve is.
1. Define the object:
  L(\theta)=\binom{n}{x}\theta^x(1-\theta)^{n-x}
2. Symbols: \theta candidate success probability, x successes, n trials.
3. Experiment A: L_A(\theta)=\binom{50}{45}\theta^{45}(1-\theta)^5.
4. Experiment B: L_B(\theta)=\binom{500}{450}\theta^{450}(1-\theta)^{50}.
5. No prior is used. The same candidates meet two amounts of data.
Because the two functions look related. The next step shows how.
1. Rewrite the powers: \theta^{450}=(\theta^{45})^{10} and (1-\theta)^{50}=[(1-\theta)^5]^{10}.
2. So L_B(\theta)\propto[L_A(\theta)]^{10}. The constants differ, the shape is the 10th power.
3. Test at \theta=0.8, relative to the peak at 0.9:
  \frac{L_A(0.8)}{L_A(0.9)}\approx0.160\qquad\frac{L_B(0.8)}{L_B(0.9)}=0.15969^{10}\approx1.08\times10^{-8}
4. Result: \theta=0.8 stays plausible with 50 trials and is almost ruled out with 500.
Because this shows that the curve narrows, not by how much. We need a number for the width, so we use derivatives. The log makes them easy.
1. Take the log: \ell(\theta)=\log L(\theta). Rules: \log(ab)=\log a+\log b and \log(a^k)=k\log a. The peak does not move.
2. Result of the log:
  \ell(\theta)=\log\tbinom{n}{x}+x\log\theta+(n-x)\log(1-\theta)
3. Differentiate (the binomial coefficient is constant):
  \ell'(\theta)=\frac{x}{\theta}-\frac{n-x}{1-\theta}
4. Set \ell'(\theta)=0, cross-multiply, expand, cancel -x\theta: x=n\theta.
5. Result: \hat\theta=x/n=0.9 for A and for B. Same peak.
Because the peaks match, the width must come from how sharply the curve bends. That is the second derivative.
1. Toy check: f''=-2 bends gently, g''=-20 bends sharply. A more negative second derivative means a sharper peak.
2. Differentiate \ell' again:
  \ell''(\theta)=-\frac{x}{\theta^2}-\frac{n-x}{(1-\theta)^2}
3. Substitute x=n\hat\theta and n-x=n(1-\hat\theta), cancel, combine fractions (numerator becomes 1):
  \ell''(\hat\theta)=-\frac{n}{\hat\theta(1-\hat\theta)}
4. Numbers: A gives -50/0.09=-555.56. B gives -500/0.09=-5555.56. B is 10 times sharper.
Because sharpness is still not a width. A Taylor approximation turns it into a parabola, and the exponential turns that into a bell curve.
1. Taylor expansion of \ell around \hat\theta: value, plus slope term, plus curvature term.
2. The slope term vanishes because \ell'(\hat\theta)=0.
3. Substitute the curvature:
  \ell(\theta)\approx\ell(\hat\theta)-\frac{n(\theta-\hat\theta)^2}{2\hat\theta(1-\hat\theta)}
4. Exponentiate (e^{\log a}=a) to return to the likelihood:
  L(\theta)\approx L(\hat\theta)\exp\left[-\frac{n(\theta-\hat\theta)^2}{2\hat\theta(1-\hat\theta)}\right]
Because this has the same shape as a normal density. Matching the two gives the width.
1. Normal exponent: -\frac{(\theta-\mu)^2}{2\sigma^2}. Our exponent has the same form.
2. Centres match: \mu=\hat\theta.
3. Coefficients match: \frac{1}{2\sigma_L^2}=\frac{n}{2\hat\theta(1-\hat\theta)}. Cancel the 2, invert both sides: \sigma_L^2=\frac{\hat\theta(1-\hat\theta)}{n}.
4. Square root: \sigma_L\approx\sqrt{\hat\theta(1-\hat\theta)/n}.
5. Caution: \sigma_L is an approximate width of the likelihood, not the SD of a distribution over \theta.
Because now we can put in the numbers for both experiments.
1. A: \sigma_A=\sqrt{0.09/50}=0.0424.
2. B: \sigma_B=\sqrt{0.09/500}=0.0134.
3. Ratio: \sigma_A/\sigma_B=\sqrt{500/50}=\sqrt{10}\approx3.16.
4. Ten times more data gives \sqrt{10} times less width, not 10 times. This is the square-root law of sample size.

Side branches

1. Slider: number of trials. Solid curve: exact relative likelihood. Dashed: 50-trial reference. Dotted: the normal approximation from step 5.
2. At n=200: curvature 2222.2 and width 0.0212.
1. One trial: SD \sqrt{\theta(1-\theta)}.
2. Sample proportion \hat\theta=X/n: SD \sqrt{\theta(1-\theta)/n}. This is sampling uncertainty over repeated experiments.
3. Likelihood as a function of \theta: width \sqrt{\hat\theta(1-\hat\theta)/n}, from one observed dataset.
4. Different concepts, same square-root expression in the binomial case.

Why does the likelihood become narrower when we observe 450 successes out of 500 trials instead of 45 out of 50?

Let's build this explanation from first principles, step by step.

We will understand not only why the likelihood becomes more concentrated, but also where the formula for its width comes from. We will connect the likelihood, its maximum, the first and second derivatives, the normal distribution, and the standard deviation.

The central observation is:

\frac{45}{50}=\frac{450}{500}=0.9

Both experiments suggest a success probability of 90%. Yet the second experiment gives much stronger evidence that the true probability is close to 0.9.

Why?

1. First, what exactly is the likelihood curve?

Imagine we are evaluating the detection rate of a banking fraud system. We run two experiments. Experiment A: 45 / 50 detections. Experiment B: 450 / 500 detections. Both show 90% success.

We want the system's unknown true probability of detecting fraud, which we call \theta (theta). The likelihood curve asks: for each hypothetical value of \theta, how probable is the data we actually observed?

For a binomial experiment:

\boxed{L(\theta)=\binom{n}{x}\theta^x(1-\theta)^{n-x}}

Let's decode the notation.

Symbol	Meaning
L(\theta)	Likelihood of the observed data if the success probability were \theta
n	Total number of trials
x	Number of successes
\theta	Candidate success probability
1-\theta	Candidate failure probability
\binom{n}{x}	Number of ways to arrange x successes among n trials

For Experiment A:

L_A(\theta)=\binom{50}{45}\theta^{45}(1-\theta)^5

For Experiment B:

L_B(\theta)=\binom{500}{450}\theta^{450}(1-\theta)^{50}

These are two different functions of the same unknown \theta. The horizontal axis holds candidate values of \theta; the vertical axis gives the likelihood at each candidate. No prior distribution is involved. We compare the same candidates against two amounts of data.

2. Look at the two likelihood curves

Tabs 1 and 2 draw the exact curves for 45/50 and 450/500, each divided by its own maximum so both peaks have height 1. That makes the widths comparable.

Two things stand out:

Both curves peak at \theta=0.9.
The curve for 500 trials is much narrower than the curve for 50 trials.

We need to explain both observations mathematically.

3. The simplest explanation: why 500 observations narrow the likelihood

Before any derivative, notice a relationship between the two likelihoods.

L_A(\theta)=\binom{50}{45}\theta^{45}(1-\theta)^5,\qquad L_B(\theta)=\binom{500}{450}\theta^{450}(1-\theta)^{50}

But:

\theta^{450}=(\theta^{45})^{10}\qquad\text{and}\qquad(1-\theta)^{50}=\bigl[(1-\theta)^5\bigr]^{10}

Therefore:

L_B(\theta)\propto\bigl[L_A(\theta)\bigr]^{10}

The symbol \propto means proportional to: the two sides differ only by a constant that does not depend on \theta. The second likelihood has the shape of the first one raised to the 10th power.

A numerical illustration

Compare the candidate \theta=0.8 with the best-fitting candidate \theta=0.9. For 50 trials:

\frac{L_A(0.8)}{L_A(0.9)}=\left(\frac{0.8}{0.9}\right)^{45}\left(\frac{0.2}{0.1}\right)^{5}\approx0.160

So the likelihood at 0.8 is about 16.0% of the maximum. For 500 trials, the relative likelihood is the 10th power of that number:

\frac{L_B(0.8)}{L_B(0.9)}=(0.15969)^{10}\approx1.08\times10^{-8}

Candidate \theta	Relative likelihood, 50 trials	Relative likelihood, 500 trials
0.8	0.160	1.08\times10^{-8}
0.9	1	1

The second experiment penalizes a candidate like 0.8 far more strongly. This already explains the narrowing: with the same 90% rate over many more trials, values away from 0.9 become much less compatible with the data. Now let's quantify how narrow the curves become.

4. First derivative: why both curves peak at 0.9

A derivative gives the slope of a curve. Imagine walking along a hill: uphill has positive slope, the summit has zero slope, downhill has negative slope. We want the summit of the likelihood, so we look for the point where the slope is zero:

L'(\theta)=0

The prime means the first derivative. The likelihood contains powers and products, so we simplify it with logarithms first.

Why take the logarithm?

Two rules turn products and powers into sums and multiples:

\log(ab)=\log a+\log b\qquad\text{and}\qquad\log(a^k)=k\log a

We define the log-likelihood:

\boxed{\ell(\theta)=\log L(\theta)}

Be careful with the notation:

Symbol	Meaning
Big L(\theta)	Original likelihood
Small \ell(\theta)	Logarithm of the likelihood
L(\hat\theta)	Maximum likelihood value

The logarithm is an increasing function, so the location of the maximum does not change when we take it. Starting from L(\theta)=\binom{n}{x}\theta^x(1-\theta)^{n-x}, take logarithms:

\ell(\theta)=\log\binom{n}{x}+x\log\theta+(n-x)\log(1-\theta)

Differentiate with respect to \theta. We use two rules:

\frac{d}{d\theta}\log\theta=\frac1\theta\qquad\text{and}\qquad\frac{d}{d\theta}\log(1-\theta)=-\frac1{1-\theta}

The binomial coefficient does not depend on \theta, so its derivative is zero. Therefore:

\boxed{\ell'(\theta)=\frac{x}{\theta}-\frac{n-x}{1-\theta}}

At the maximum the slope is zero. We set it to zero and solve one rule at a time:

Set the slope to zero: \frac{x}{\theta}-\frac{n-x}{1-\theta}=0.
Move one term across: \frac{x}{\theta}=\frac{n-x}{1-\theta}.
Cross-multiply: x(1-\theta)=\theta(n-x).
Expand both sides: x-x\theta=n\theta-x\theta.
Cancel -x\theta on both sides: x=n\theta.

Hence:

\boxed{\hat\theta=\frac{x}{n}}

The hat on \theta marks the value of the parameter that maximizes the likelihood. For Experiment A, \hat\theta_A=\frac{45}{50}=0.9. For Experiment B, \hat\theta_B=\frac{450}{500}=0.9. This proves why both curves have the same peak. Their widths differ, so we need another derivative.

5. Second derivative: why one curve is sharper

The first derivative tells us the slope. The second derivative tells us how quickly that slope changes. Consider two simple curves:

f(\theta)=10-(\theta-0.9)^2\qquad g(\theta)=10-10(\theta-0.9)^2

Both reach their maximum value of 10 at \theta=0.9, but g falls much faster as we move away. Differentiate twice:

f''(\theta)=-2\qquad g''(\theta)=-20

The more negative second derivative marks sharper downward curvature. The same principle applies to the log-likelihood. Start from:

\ell'(\theta)=\frac{x}{\theta}-\frac{n-x}{1-\theta}

Differentiate each term once more:

\frac{d}{d\theta}\left(\frac{x}{\theta}\right)=-\frac{x}{\theta^2}\qquad\frac{d}{d\theta}\left(-\frac{n-x}{1-\theta}\right)=-\frac{n-x}{(1-\theta)^2}

Therefore:

\boxed{\ell''(\theta)=-\frac{x}{\theta^2}-\frac{n-x}{(1-\theta)^2}}

Evaluate it at the maximum

We know \hat\theta=x/n, so x=n\hat\theta and n-x=n(1-\hat\theta). Substitute:

\ell''(\hat\theta)=-\frac{n\hat\theta}{\hat\theta^2}-\frac{n(1-\hat\theta)}{(1-\hat\theta)^2}

Cancel common factors:

=-\frac{n}{\hat\theta}-\frac{n}{1-\hat\theta}

Take n outside:

=-n\left(\frac1{\hat\theta}+\frac1{1-\hat\theta}\right)

Combine the fractions over a common denominator:

=-n\left(\frac{1-\hat\theta+\hat\theta}{\hat\theta(1-\hat\theta)}\right)

The numerator simplifies to 1. Thus:

\boxed{\ell''(\hat\theta)=-\frac{n}{\hat\theta(1-\hat\theta)}}

We have derived this formula entirely from the likelihood.

Calculate it for both experiments

\ell_A''(0.9)=-\frac{50}{0.9(0.1)}=\boxed{-555.56}\qquad\ell_B''(0.9)=-\frac{500}{0.9(0.1)}=\boxed{-5555.56}

The log-likelihood for 500 trials has 10 times the downward curvature. The number of trials n sits in the numerator, so for a fixed observed proportion the curvature grows with n. The likelihood therefore falls more sharply away from its peak.

6. How do we get a normal distribution from the likelihood?

This is the most important connection. We know the second likelihood is sharper. To turn sharpness into a number for width, we use a Taylor approximation: near a point, a smooth function is approximated by its value, slope and curvature there. Near \hat\theta:

\ell(\theta)\approx\ell(\hat\theta)+\ell'(\hat\theta)(\theta-\hat\theta)+\frac12\ell''(\hat\theta)(\theta-\hat\theta)^2

Let's decode the three terms.

Term	Meaning
\ell(\hat\theta)	Height of the log-likelihood at its maximum
\ell'(\hat\theta)(\theta-\hat\theta)	Adjustment based on slope
\tfrac12\ell''(\hat\theta)(\theta-\hat\theta)^2	Adjustment based on curvature

At the maximum \ell'(\hat\theta)=0, so the slope term disappears:

\ell(\theta)\approx\ell(\hat\theta)+\frac12\ell''(\hat\theta)(\theta-\hat\theta)^2

Substitute the curvature formula:

\ell(\theta)\approx\ell(\hat\theta)-\frac{n(\theta-\hat\theta)^2}{2\hat\theta(1-\hat\theta)}

Near the peak, the log-likelihood behaves like a downward-opening parabola.

Why do we now exponentiate?

We converted the likelihood to its logarithm only to simplify differentiation. We now want the original likelihood back. The exponential reverses the logarithm: e^{\log a}=a. Exponentiate both sides and use e^{a-b}=e^a e^{-b}:

e^{\ell(\theta)}\approx e^{\ell(\hat\theta)}\exp\left[-\frac{n(\theta-\hat\theta)^2}{2\hat\theta(1-\hat\theta)}\right]

Replace each exponentiated log-likelihood with its likelihood:

\boxed{L(\theta)\approx L(\hat\theta)\exp\left[-\frac{n(\theta-\hat\theta)^2}{2\hat\theta(1-\hat\theta)}\right]}

What does this formula mean?

L(\theta): the likelihood at any candidate \theta.
L(\hat\theta): the maximum height of the likelihood.
The exponential term: how much the likelihood falls as we move away from the maximum.

At \theta=\hat\theta the exponent is zero, so the factor equals 1. Farther away, the squared distance grows, the exponent becomes more negative, and the likelihood decreases.

7. Where does the standard deviation enter?

Look at the normal density:

f(\theta)=\frac{1}{\sigma\sqrt{2\pi}}\exp\left[-\frac{(\theta-\mu)^2}{2\sigma^2}\right]

Here \mu is the mean (the centre), \sigma is the standard deviation (the spread), and \sigma^2 is the variance. Compare the exponentials. Our likelihood approximation has \exp\left[-\frac{n(\theta-\hat\theta)^2}{2\hat\theta(1-\hat\theta)}\right]. The normal density has \exp\left[-\frac{(\theta-\mu)^2}{2\sigma^2}\right]. They have the same structure.

First, compare the centres:

\boxed{\mu=\hat\theta}

Next, compare the coefficients that multiply the squared distance, and solve one rule per line:

Match: \frac{1}{2\sigma_L^2}=\frac{n}{2\hat\theta(1-\hat\theta)}.
Cancel the 2: \frac1{\sigma_L^2}=\frac{n}{\hat\theta(1-\hat\theta)}.
Invert both sides: \sigma_L^2=\frac{\hat\theta(1-\hat\theta)}{n}.
Take the square root:

\boxed{\sigma_L\approx\sqrt{\frac{\hat\theta(1-\hat\theta)}{n}}}

We have derived the approximate width of the likelihood from its shape. An important distinction: the likelihood is not a probability distribution over \theta. So \sigma_L is an approximate width parameter, not literally the standard deviation of the raw likelihood. It also equals the usual plug-in estimate of the standard error of the observed success proportion.

8. Calculate the widths

\sigma_A\approx\sqrt{\frac{0.9(1-0.9)}{50}}=\sqrt{\frac{0.09}{50}}=\boxed{0.0424}\qquad\sigma_B\approx\sqrt{\frac{0.9(1-0.9)}{500}}=\sqrt{\frac{0.09}{500}}=\boxed{0.0134}

With 50 trials the width is 0.0424, which is 4.24 percentage points. With 500 trials it is 0.0134, which is 1.34 percentage points. The ratio:

\frac{\sigma_A}{\sigma_B}=\sqrt{\frac{500}{50}}=\boxed{\sqrt{10}\approx3.16}

The likelihood for 500 trials is about 3.16 times narrower. Notice that ten times more observations does not make the width ten times smaller. It makes it \sqrt{10} times smaller. This is the familiar square-root relationship between sample size and uncertainty.

9. Experiment: change the number of observations

The graph holds the observed success proportion at 90%. Move the slider to change the number of trials and watch the width. The solid curve is the exact relative likelihood. The dashed curve is the 50-trial reference. The dotted curve is the normal approximation from section 6.

Total number of trials

Both exact curves are scaled to a peak height of 1. The approximate width and the curvature update with the sample size.

10. One final distinction: mean and standard deviation of what?

We have discussed three different mathematical objects.

Object	Centre or mean	Spread
One success/failure trial X_i	\theta	SD =\sqrt{\theta(1-\theta)}
Sample success proportion \hat\theta=X/n	Expected value =\theta	SD =\sqrt{\theta(1-\theta)/n}
Likelihood as a function of \theta	Maximum at x/n	Approximate width =\sqrt{\hat\theta(1-\hat\theta)/n}

The sample proportion is a random variable. Repeat experiments of 50 trials and each gives a slightly different proportion. The standard deviation of those proportions describes sampling uncertainty.

The likelihood is a function of candidate values of \theta, calculated after one particular dataset is observed. Its width describes how quickly the relative likelihood falls away from its maximum. These are different concepts, even though they lead to the same square-root expression in this binomial example.

Putting the entire argument together

Start with the likelihood:

L(\theta)=\binom{n}{x}\theta^x(1-\theta)^{n-x}

The first derivative locates the maximum:

\hat\theta=\frac{x}{n}

The second derivative measures sharpness:

\ell''(\hat\theta)=-\frac{n}{\hat\theta(1-\hat\theta)}

The Taylor approximation gives a normal-like shape:

L(\theta)\approx L(\hat\theta)\,e^{-\frac{n(\theta-\hat\theta)^2}{2\hat\theta(1-\hat\theta)}}

Matching the normal curve gives the approximate width:

\boxed{\sigma_L\approx\sqrt{\frac{\hat\theta(1-\hat\theta)}{n}}}

The final intuition: both 45/50 and 450/500 suggest the same best estimate, \theta=0.9. With 500 observations, the likelihood decreases much faster as we move away from 0.9. Its log-curvature becomes 10 times greater and its approximate width becomes \sqrt{10} times smaller. That is why more data can make us more confident about an estimate without changing the estimate itself.

Term used everywhere: "likelihood". L(\theta)\propto\theta^{x}(1-\theta)^{n-x}. Here \theta = hit rate, x = hits, n = trials. Full derivation: tab D.

Setup

Take two data sets: A = 45 hits in 50 trials, B = 450 hits in 500 trials.
Note: both have the same peak, \hat\theta = x/n = 0.90.
Scale each likelihood to area 1 to compare the shapes.

Result: the peak is at the same place. Only the width differs.

Find the cause

Take the log of L(\theta):
\log L(\theta)=x\ln\theta+(n-x)\ln(1-\theta)
Write x=n\hat\theta. Factor out n:
\log L(\theta)=n\Bigl[\hat\theta\ln\theta+(1-\hat\theta)\ln(1-\theta)\Bigr]
Note: the bracket does not depend on n.

Result: n multiplies the log-likelihood. B has 10 times the log-likelihood of A at every \theta.

Compute the width

Take the curvature at the peak:
c=-\ell''(\hat\theta)=\frac{n}{\hat\theta(1-\hat\theta)}
Compute for A: c=50/0.09=555.6.
Compute for B: c=500/0.09=5555.6.
Take the width:
\sigma_L=\frac{1}{\sqrt{c}}=\sqrt{\frac{\hat\theta(1-\hat\theta)}{n}}
Compute for A: 0.0424. Compute for B: 0.0134.

Result: B is \sqrt{10}=3.16 times thinner. The exact curve gives 0.0439 and 0.0135, a ratio of 3.26.

See it at \theta=0.8

Compute the ratio of likelihoods:
\frac{L(0.8)}{L(0.9)}=e^{-n\cdot D},\qquad D=0.9\ln\frac{0.9}{0.8}+0.1\ln\frac{0.1}{0.2}=0.0367
For A (n=50): e^{-1.83}=0.160.
For B (n=500): e^{-18.3}=1.08\times10^{-8}.

Result: A still allows \theta=0.8. B almost excludes it.

Read the intervals

Data	95% interval	Width	Peak height
A: 45 / 50	0.786 \text{ to } 0.956	0.170	9.4
B: 450 / 500	0.871 \text{ to } 0.923	0.053	29.7

Result: the width falls by about 1/\sqrt{n}. A thinner curve is a taller curve, because the area stays 1.

Warnings

Do not read width as model accuracy. Width shows data size and noise only.
Do not use the width formula near \theta=0 or \theta=1. The curve is skewed there.
Do not compare unscaled likelihoods. Their areas differ by many orders of magnitude.

Check yourself

What is the width for 90 hits in 100 trials? (Answer: \sqrt{0.09/100}=0.03.)
How many trials give half the width of n=50? (Answer: 200.)
Why is the n=500 curve taller at its peak? (Answer: the area stays 1, so a thinner curve rises.)