RELATIVE EFFICIENCIES OF KERNEL AND LOCAL LIKELIHOOD DENSITY ESTIMATORS

Peter Hall^{1} Terence Tao^{1,2}

ABSTRACT. Local likelihood methods enjoy advantageous properties, such as good performance in the presence of edge effects, that are rarely found in other approaches to nonparametric density estimation. However, as we argue in the present paper, standard kernel methods can have distinct advantages when edge effects are not present. We show that, while the integrated variances of the two methods are virtually identical, the integrated squared bias of a conventional kernel estimator is less than that of a local log-linear estimator by as much as a factor of four. Moreover, the greatest bias improvements offered by kernel methods occur when they are needed most — i.e. when the impact of bias is particularly high. Similar comparisons can also be made when high-degree local log-polynomial fits are assessed against high-order kernel methods. For example, although (as is well known) high-degree local polynomial fits offer potentially infinite efficiency gains relative to their kernel competitors, the converse is also true. Indeed, the asymptotic value of integrated squared bias of a local log-quadratic estimator can exceed any given constant multiple of that for the competing kernel method. In all cases the densities that suffer problems in the context of local log-likelihood methods can be chosen to be symmetric, either unimodal or bimodal, supported on the interval [−1, 1] and to have arbitrarily many derivatives as functions on the real line. They are not pathological. However, our results reveal quantitative differences between global performances of local log-polynomial estimators applied to unimodal or multimodal distributions.

KEYWORDS. Bandwidth, global performance, high-order methods, integrated squared bias, local linear, log polynomial, mean squared error, nonparametric curve estimation. SHORT TITLE. Bump hunting. AMS SUBJECT CLASSIFICATION. Primary 62G07; Secondary 62G20.

1Centre for Mathematics and its Applications, Australian National University, Canberra, ACT 0200, Australia

2Department of Mathematics, UCLA, Box 951555, Los Angeles, CA 90095-1555, USA

1. INTRODUCTION Properties of local likelihood methods for density estimation have been ex-

plored by Copas (1995), Simonoff (1996, p. 64ff), Hjort and Jones (1996) and Loader (1996), among others. These techniques have much to offer relative to standard methods, such as those based on kernels. In particular, their good performance in the presence of edge effects should be stressed. Moreover, high-degree local log-polynomial density estimators offer optimal rates of convergence without the spurious “wiggles”, and without problems of negativity, associated with highorder kernel techniques. However, the relative global performances of local likelihood and kernel methods have not been addressed in detail before. The purpose of the present paper is to make such global comparisons.

Our results will address local log-polynomial estimators of all degrees, but let us first outline our conclusions in the log-linear case. It is well known that there, in the absence of edge effects, neither the local likelihood estimator nor its kernel competitor can consistently outperform the other in local terms. In fact, a local comparison depends very much on the point at which the density is estimated.

However, it can be shown that in a global sense, and in the absence of edge effects, the kernel estimator is preferable on performance grounds. This follows from the fact that the asymptotic integrated squared bias (ISB) of a local loglinear estimator is strictly greater than its counterpart in the kernel case, while asymptotic integrated squared variances are identical. Moreover, ISB for local loglinear estimators can be up to four times greater, for densities that have two squareintegrable derivatives. Moreover, this excess of bias occurs in cases where bias is already large, and that fact tends to particularly exacerbate global performance difficulties experienced by local log-linear likelihood.

If an optimal global bandwidth is employed, then the effect on mean integrated squared error of inflating ISB by a factor of 4 is equivalent to reducing sample size by a factor of 4^{1/4}. Therefore the difficulties caused by large ISB are of practical importance. The densities that suffer from these problems are those which decrease to zero relatively steeply in their tails. Nevertheless, these densities are not pathological; they can be chosen to be symmetric, unimodal and compactly supported, and to be arbitrarily smooth on the full real line.

The situation is a little different if exponentiated polynomials of higher degree are used in the local likelihood approach. In that case there is a special class of smooth densities which are exactly expressible as exponentiated polynomials of degree two or more. In such instances the local likelihood approach can strongly outperform standard high-order kernel methods. However, while this might be taken to indicate superiority of local log-polynomial methods, we shall show that the diametrically opposite view is also valid, since if f lies outside the special class then the relative performances can be reversed. In particular, the ratio of the minimum mean integrated squared error of the local likelihood estimator to that of its kernel counterpart can be unboundedly large, even in the asymptotic limit as sample size increases. Densities that suffer from these problems include many that are symmetric, bimodal, compactly supported and arbitrarily smooth; they are far from pathological.

On the other hand, if the target density is constrained to be unimodal then the greatest inefficiency of local log-polynomial estimators, relative to kernel methods, is finite rather than infinite. Nevertheless the inefficiency in the unimodal case, while finite, can be large. For example, when the fitted polynomial is of degree 2 and bandwidth is chosen to minimise mean squared error, a local log-polynomial estimator can require 1.85 times as much data to achieve the same level of performance as its kernel counterpart. (If ω̂_{1} and ω̂_{2} are estimators then the inefficiency of ω̂_{1} relative to ω̂_{2} is here denoted by the ratio of the respective mean integrated squared errors, or MISEs. The efficiency is the inverse of the ratio. The sample size necessary to achieve a given level of asymptotic MISE, after optimisation over the bandwidth, increases with ISB.)

Such dichotomy between unimodal and multimodal distributions is unusual. The present instance is apparently the first where it has been noted that qualitative properties of a density can have a direct, quantitative and significant influence on L^{2} performance. This arises because the relative efficiency of log-polynomial and kernel estimators depends intimately on the relationship between the size of a density and the sizes of its derivatives in the neighbourhood of a point where the density achieves a local minimum. If the density is multimodal then a trough between two adjacent modes provides an opportunity, not available in the unimodal case, for the density to achieve a minimum without actually vanishing there. The example used, in Appendix (iii), to show that unbounded inefficiency of local logpolynomial estimators occurs for bimodal densities, exploits the opportunity. Minor modifications of the example show that the same result holds for any density with two or more modes.

Recent developments of the concept of local likelihood density estimation include those of Eguchi and Copas (1998) and Fan, Farmen and Gijbels (1998), on general approaches to local maximum likelihood; Betensky, Lindsey, Ryan and Wand (1999) and Bebchuk and Betensky (2000, 2001), on hazard rate estimation; Cheng, Hall and Turlach (1999), on combining parametric and nonparametric methods; Tenreiro (1999) and Hili (2001), on applications to dependent data; and El Barmi and Simonoff (2000), on density estimation under monotonicity constraints. Note too the surveys of smoothing methods by Jones and Signorini (1997) and Simonoff (1998).

Our main results will be given in section 2, although technical details will be deferred to the appendices. We conclude this section by defining the estimators. Suppose we are given a random sample X_{1}, …, X_{n} from a distribution with density f, which function we wish to estimate. A local log-polynomial estimator f̂_{pol}, of degree p, is constructed by fitting, by maximum likelihood, a density of the form

ψ(u) = ψ(u − x|θ) ≡ exp { ∑_{j=0}^{p} θ_{j}(u − x)^{j}}

in the neighbourhood of x. Specifically, θ = (θ_{0}, …, θ_{p}) is chosen to maximise

∑_{i=1}^{n} K(\frac{X_{i} − x}{h}) log ψ(X_{i} − x|θ) − n ∫ K(\frac{u − x}{h}) ψ(u − x|θ) du ,

where K is a kernel and h a bandwidth. Taking θ̂ = θ̂(x) = (θ̂_{0}, …, θ̂_{p}) to be the extremum, we define the local log-polynomial estimator of f(x), of degree p, by f̂_{pol}(x) = exp(θ̂_{0}). More simply, a kernel estimator of f(x) is

f̂_{ker}(x) = \frac{1}{nh} ∑_{i=1}^{n} K(\frac{X_{i} − x}{h}) .  (1.1)

2. MAIN RESULTS

2.1. Local log-linear estimators. Denote by f̂_{lin} the local log-linear estimator, representing the version of f̂_{pol} when p = 1. As noted by Hjort and Jones (1996, section 5.2) and Loader (1996, section 5), f̂_{lin} has asymptotic bias h^{2} b and asymptotic variance (nh)^{−1}s^{2}, where b = b_{lin} ≡ \frac{1}{2} κ_{2} \{f′′ − (f′)^{2} f^{−1}\}, s^{2} = κ f, κ_{j} = ∫ u^{j} K(u) du and κ = ∫ K^{2}. Indeed, assuming f has two continuous derivatives in a neighbourhood of x, that f(x) > 0, that K is a bounded, symmetric and compactly supported density, and that h → 0 and nh → ∞ as n → ∞, one may write f̂ = f̂_{lin} as

f̂(x) = f(x) + h^{2} b(x) + (nh)^{−1/2} s(x) Z_{n}(x) + o_{p}{h^{2} + (nh)^{−1/2}} ,  (2.1)

where the random variable Z_{n}(x) is asymptotically normal N(0, 1).

The analogous formula for a standard kernel estimator f̂ = f̂_{ker}, for which the kernel has the same properties, is identical to (2.1) except that b = b_{lin} ≡ \frac{1}{2} κ_{2} f′′. This formula is also obtained when the fitted density is a log-polynomial of degree 0. For a particular value of x, either |b_{lin}(x)| or |b_{ker}(x)| can be the larger. It is only when making a global comparison, such as that below, that performance differences become clear.

Analogously to (2.1),

∫ E(f̂− f)^{2} = h^{4} ∫ b^{2} + (nh)^{−1} κ + o{h^{4} + (nh)^{−1}} .  (2.2)

This is of course well-known in the case of standard kernel estimators. Indeed, a sufficient condition on f for (2.2) to hold for f̂ = f̂_{ker} is that, for k = 2, f is in the class C_{k} of densities that are compactly supported, have k−1 continuous derivatives on the real line, have k piecewise monotone derivatives interior to the support of f, and satisfy ∫ (f^{(k)})^{2} < ∞. Of course, (2.2) also holds for densities that are not compactly supported, but there the tail properties of f need to be more carefully prescribed.

Results such as (2.2) may also be established for the local log-linear estimator f̂ = f̂_{lin}, provided f is in C_{2} and the integrals are taken over regions where f is bounded away from 0: for each ϵ > 0,

∫_{S(ϵ)} E(f̂− f)^{2} = h^{4} ∫_{S(ϵ)} b^{2} + (nh)^{−1} κ ∫_{S(ϵ)} f + o{h^{4} + (nh)^{−1}} ,  (2.3)

where S(ϵ) = {x : f(x) > ϵ}. The only reason for the restriction is that asymptotic properties of f̂_{lin} are awkward to determine for points x where f(x) may be very close to 0. It is also necessary to slightly modify the definition of f̂_{lin} to take account of rare cases where data are particularly sparse. This is analogous to using a bandwidth adjustment or a ridging method to adjust a local polynomial estimator for sparse design in nonparametric regression, and is only a technical issue. It is sufficient to define f̂_{lin} = min(f̃_{lin}, C), where f̃_{lin} is the “raw”, unmodified local likelihood estimator, C is any constant that exceeds sup f, and it is understood that f̂_{lin} ≤ C when the raw local likelihood estimator cannot be defined at all.

The right hand side of (2.3) converges to the right hand side of (2.2) as ϵ ↓ 0, and so in both cases we may view

MISE(h) = h^{4} ∫ b^{2} + (nh)^{−1} κ  (2.4)

as asymptotic mean integrated squared error of f. Importantly, the variance contribution to this formula, (nh)^{−1}κ, is identical for both f̂_{ker} and f̂_{lin}. It is only in the bias term, ∫ b^{2}, that there are differences.

An integration by parts argument shows that ∫ (f′)^{2} f′′ f^{−1} = \frac{1}{3} ∫ (f′)^{4} f^{−2}, whence it follows that

∫ b_{lin}^{2} = ∫ b_{ker}^{2} + \frac{1}{3} κ_{2}^{2} ∫ \frac{(f′)^{4}}{f^{2}} .  (2.5)

Result (2.5) implies that ∫ b_{lin}^{2} > ∫ b_{ker}^{2} for f ∈ C_{2}, and so the kernel estimator is superior in terms of global performance. To indicate the extent of the superiority we shall show in Appendix (i) that

∫ (f′)^{4} f^{−2} < 9 ∫ (f′′)^{2}  (2.6)

for all f ∈ C_{2}, and that the constant 9 is best possible, in the sense that there exists a sequence of functions f = f_{ϵ} ∈ C_{2}, indexed by a parameter ϵ > 0 converging to 0, such that the ratio {∫ (f′)^{4} f^{−2}}/ ∫ (f′′)^{2} converges to 9 as ϵ ↓ 0.

The densities f_{ϵ} are far from pathological. For example, they can be chosen to be unimodal, supported in the interval [−1, 1], and to have ν continuous derivatives on the real line, for any given ν; see Appendices (i) and (ii) for details. Moreover, the densities have the property that ∫ (f′′_{ϵ})^{2} → ∞ as ϵ ↓ 0. Therefore the excessive integrated squared bias (ISB) of the local likelihood estimator occurs just when it is least wanted, i.e. when ISB is already very large.

Combining (2.5) and (2.6), and noting that the constant 9 at (2.6) is best possible, we deduce that, among functions f in C_{2}, the asymptotic value of ISB can be up to four times greater for local log-linear estimators than it is for their kernel competitors. Increasing ISB by a factor of 4 requires an increase in sample size by 4^{1/4} if we are to maintain the value of min_{h} MISE(h). 2.2. High-order estimators. Except in the presence of edge effects, the asymptotic bias and variance of a pth degree local log-polynomial density estimator have the same form as those for a standard kernel estimator that uses a high-order kernel L derived from the basic kernel K. In particular, if f̂ = f̂_{pol} is defined as in section 1 then, for p ≥ 2, its error as an estimator of f may be expressed by the following analogue of (2.1):

f̂(x) = f(x) + h^{k} b(x) + (nh)^{−1/2} s(x) Z_{n}(x) + o_{p}{h^{k} + (nh)^{−1/2}} ,  (2.7)

where k = p+1 or p+2 according as p is odd or even, b = b_{pol} ≡ \frac{1}{k!} λ_{k} β_{pol p}, s^{2} = λ f, λ_{k} = ∫ u^{k} L_{k}(u) du, λ = ∫ L_{k}^{2}, L_{k} is a kernel of the form L_{k}(u) = Q_{k}(u) K(u), Q_{k} is an even polynomial of degree k − 2 with coefficients depending only on K, β_{pol p} = (log f)^{(k)}f if p is odd, β_{pol p} = (log f)^{(k)}f + k (log f)^{(k−1)}f′ if p is even, and the superscript ^{(j)} denotes the operation of taking j derivatives. In each case L_{k} is a conventional kth order kernel, meaning that ∫ u^{j} L_{k}(u) du = 1 if j = 0, equals 0 if 1 ≤ j ≤ k − 1, and is nonzero if j = k. Result (2.7) follows from Theorem 2 of Loader (1996), and in low-order cases from work in sections 3–5 of Hjort and Jones (1996).

The analogue of (2.7) for a kernel estimator is of course well known. If, when defining f̂_{ker}, we replace the kernel at (1.1) by L, to facilitate comparison with the local polynomial case, the kernel version of (2.7) is identical except that now b = b_{ker} ≡ \frac{1}{k!} λ_{k} f^{(k)}. In analogy to (2.4), and taking b = b_{pol} and b = b_{ker} in the local log-polynomial and kernel cases respectively, we may interpret

MISE(h) = h^{2k} ∫ b^{2} + (nh)^{−1} λ

as asymptotic mean integrated squared error.

When p ≥ 2 the class of densities with k continuous derivatives includes some for which b_{pol} is identically 0, and there the local log-polynomial estimator is incontrovertibly the better performer. However, in other cases the log-polynomial estimator can suffer from extreme versions of the problems noted in section 2.1. Indeed, we shall show in Appendix (iii) that if p ≥ 2, or equivalently k ≥ 4,

sup_{f∈C_{k}} ( ∫ b_{pol}^{2})/( ∫ b_{ker}^{2}) = ∞ .  (2.8)

If the ratio of integrals on the left hand side of (2.8) is denoted by R(f) then an increase in R(f) by a factor C > 1 requires an increase in sample size by C^{1/2k} if the value of min_{h} MISE(h) is to remain unaltered. Result (2.8) implies that local logpolynomial estimators can have unboundedly low efficiency relative to their kernel counterparts.

It should be noted that (2.8) does not hold when p = 1, where k = 2. There, (2.6) implies that the right hand side of (2.8) equals 4 rather than infinity.

Result (2.8) remains true if the supremum is taken over symmetric, m-modal densities in C_{k} that are supported on [−1, 1] and have any desired number of derivatives on the full real line, for any given m ≥ 2. This will follow from our derivation in Appendix (iii), which will confine attention to bimodal examples and show that R(f) is unbounded there. However, (2.8) fails if the densities are unimodal. Indeed, if C_{uni k} denotes the set of all unimodal densities in C_{k} then

S_{p} ≡ sup_{f∈C_{uni k}} ( ∫ b_{pol}^{2})/( ∫ b_{ker}^{2}) < ∞ .  (2.9)

See Appendix (iv) for a proof, and recall that k = p + 1 or p + 2 in the respective cases of odd and even p. Therefore, while the potential inefficiency of local logpolynomial methods is infinite if the target density is permitted to be bimodal, it is finite in the unimodal case. Nevertheless the finite inefficiency can be large, as we now show.

To obtain lower bounds to S_{p} we consider densities f_{ϵ} = f_{ϵ,k} that are supported on [−1, 1] and which equal C_{k} (1 − x^{2})^{k−(1/2)+ϵ} when 1 − δ_{k} ≤ |x| < 1, for some C_{k} > 0, 0 < δ_{k} < 1 and 0 < ϵ < \frac{1}{2}. In such cases ∫ (f_{ϵ}^{(k)})^{2} < ∞, although |f_{ϵ}^{(k)}| is unbounded for each ϵ. However, the latter property can be removed, and in fact f_{ϵ} given an arbitrarily large number of derivatives as a function on the real line, by using tapers of the type suggested in Appendix (ii).

For the given construction of f_{ϵ}, the limit, as ϵ ↓ 0, of the ratio R(f_{ϵ}) equals

T_{p} = (\frac{(k − 1)! (k − 2)! 2^{2k−3}}{(2k − 3)!} )^{2} or T_{p} = (\frac{\{(k − 2)!\}^{2} (k^{2} − \frac{3}{2} k + 1) 2^{2k−3}}{(2k − 3)!} )^{2}

in the cases of odd and even p respectively. Therefore S_{p} ≥ T_{p}. The value of T_{p} diverges to infinity in asymptotic proportion to p or p^{3} in the cases of odd and even p, respectively. Successive values of T_{p}, for p = 2, …, 6, are 138, 10.2, 518, 16.5 and 1310, respectively, to three significant figures. In particular, if the target density is unimodal then a local quadratic fit can require almost 138^{1/8} ≈ 1.85 times the sample size for a third-order kernel method, if it is to achieve the same level of performance when both estimators are optimised over bandwidth.

Appendix (i): Derivation of (2.6) and of the optimality of that bound To show that (2.6) holds, observe that, with ⟨·, ·⟩ denoting the usual inner

product and ∥ · ∥ the associated norm,

⟨\frac{(f′)^{2}}{f} , f′′⟩ = ⟨\frac{\{(f′)^{3}\}′}{3} , \frac{1}{f} ⟩ = −⟨\frac{(f′)^{3}}{3} , \frac{f′}{f^{2}} ⟩ = −\frac{1}{3} ∥\frac{(f′)^{2}}{f} ∥^{2} .

Hence, by the Cauchy-Schwarz inequality,

\frac{1}{3} ∥\frac{(f′)^{2}}{f} ∥^{2} = |⟨\frac{(f′)^{2}}{f} , f′′⟩| ≤ ∥\frac{(f′)^{2}}{f} ∥ ∥f′′∥ .

That is, ∥(f′)^{2}/f∥^{2} ≤ 9 ∥f′′∥^{2}, which is identical to (2.6) except that the inequality at (2.6) is strict. However, the unique density that gives equality in the Cauchy-Schwarz inequality is not in C_{k}, and so we may take the inequality to be strict.

To see why the constant 9 is best possible, put ρ(f) = {∫ (f′)^{4} f^{−2}}/ ∫ (f′′)^{2} and, for an appropriately chosen density f_{ϵ}, consider the limit of ρ(f_{ϵ}) as ϵ ↓ 0. In particular, take f_{ϵ} to be a symmetric density satisfying (a) the support of f_{ϵ} equals [−1, 1], (b) f_{ϵ}(x) = (1 − x)^{(3/2)+ϵ} for x ∈ [δ, 1], (c) |f′′_{ϵ}(x)| is bounded uniformly in x ∈ [−2δ, 2δ] and 0 < ϵ < 1, (d) f_{ϵ} is bounded away from 0 in the same sense, and (e) δ ∈ (0, \frac{1}{2}) is fixed. (We may construct f_{ϵ} to be unimodal.) For such an f_{ϵ} it may be proved that, for a constant C > 0,

∫ (f′_{ϵ})^{4} f_{ϵ}^{−2} ∼ 9 C ϵ^{−1} , ∫ (f′′_{ϵ})^{2} ∼ C ϵ^{−1} .  (A.1)

Hence ρ(f_{ϵ}) converges to 9.

Appendix (ii): Showing that the densities f_{ϵ} can be chosen to be smooth We shall show how to construct tapers which, when applied to the tails of the

densities f_{ϵ}, allow them to have two continuous derivatives on the real line without losing the unimodality or compact support of f_{ϵ}, or upsetting the properties at (A.1). Obvious extensions of our argument show that f_{ϵ} may be chosen to have any number of continuous derivatives without affecting those properties.

Note that (A.1) continues to hold if we replace f_{ϵ} by ψ_{ϵ} where ψ_{ϵ}(x) = x^{(3/2)+ϵ} for x ∈ [0, 1], and if we take the integrals at (A.1) over [0, 1]. Let ξ = ξ(ϵ) be a positive number converging to 0, as ϵ ↓ 0, so rapidly that ξ^{ϵ} → 0. Let ϕ_{ξ} denote a function defined on (−∞, 1], equalling ψ_{ϵ} on [ξ, 1] and vanishing on (−∞, 0]. We shall show how to construct ϕ_{ξ} on (0, ξ) in such a way that (i) ϕ_{ξ} has two continuous derivatives on (−∞, 1], (ii) ϕ_{ξ} > 0 on (0, 1], (iii) ∫_{[0,ξ]} ϕ_{ξ}(x) dx = O(ξ^{(5/2)+ϵ}), and (iv)

∫_{0}^{ξ} {(ϕ′_{ξ})^{4} ϕ_{ξ}^{−2} + (ϕ′′_{ξ})^{2}} = O(ξ^{2ϵ}) .  (A.2)

It then follows that (A.1) continues to hold if we replace f_{ϵ} by ϕ_{ξ} and take the integrals over [0, 1]. The taper represented by ϕ_{ξ} can be incorporated into the density f_{ϵ}, using obvious changes of location and scale.

We conclude by verifying (i)–(iv). Put a = (\frac{3}{2} + ϵ) (−\frac{15}{4} + \frac{1}{2}ϵ) + 10, b = (\frac{3}{2} + ϵ) (\frac{13}{2} − ϵ) − 15 and c = \frac{1}{2} (\frac{3}{2} + ϵ) (−\frac{11}{2} + ϵ) + 6. In this notation, define

ϕ_{ξ}(x) = {a (x/ξ)^{3} + b (x/ξ)^{4} + c (x/ξ)^{5}} ξ^{(3/2)+ϵ}

for x ∈ (0, ξ). If ϵ > 0 is sufficiently small then the quadratic a + bx + cx^{2} is strictly positive definite, and in particular, for constants 0 < C_{1} < C_{2} < ∞,

C_{1}x^{3}ξ^{−(3/2)+ϵ} ≤ ϕ_{ξ}(x) ≤ C_{2}x^{3}ξ^{−(3/2)+ϵ}  (A.3)

for all x ∈ (0, ξ). This implies (ii) and (iii). Note too that for j = 1, 2,

|ϕ_{ξ}^{(j)}(x)| ≤ C_{3}x^{3−j}ξ^{−(3/2)+ϵ}  (A.4)

for all x ∈ (0, ξ). Property (A.2), and hence (iv), follows from (A.3) and (A.4). Note too that ϕ_{ξ}(0+) = ϕ′_{ξ}(0+) = ϕ′′_{ξ}(0+) = 0, and that by choice of a, b and c, ϕ_{ξ}^{(j)}(ξ−) = ψ_{ϵ}^{(j)}(ξ) for j = 0, 1, 2. Therefore property (i) is satisfied.

Appendix (iii): Derivation of (2.8) Let C ≥ 1, put g(x) = 1 + x^{2} for |x| ≤ C, and extend g to the real line

in such a way that g is symmetric, bimodal, nonnegative everywhere, has support (−2C, 2C) and has k + 1 derivatives on the real line. This may be achieved with sup |g^{(j)}(x)| ≤ B_{j} C^{2−j} for 0 ≤ j ≤ k + 1, where B_{j} denotes a constant depending only on j. Then, taking β to denote either (log g)^{(k)}g or (log g)^{(k)}g+k (log g)^{(k−1)}g′, we deduce that

∫_{−∞}^{∞} β(x)^{2} dx ≥ ∫_{−1}^{1} β(x)^{2} dx = B ,  (A.5)

say, where B > 0 does not depend on C. Note too that

∫_{−∞}^{∞}(g^{(k)})^{2} = 2 ∫_{C}^{2C}(g^{(k)})^{2} ≤ 2 B_{k} C^{5−2k} .  (A.6)

Rescale g so that it vanishes outside [−1, 1], normalise it so that it integrates to 1, let f denote the resulting density, and write β_{pol} for either (log f)^{(k)}f or (log f)^{(k)}f + k (log f)^{(k−1)}f′, respective to the definitions of β. Then f has k continuous derivatives on its support, which equals [−1, 1], and in particular f ∈ C_{k}. It follows from (A.5) and (A.6) that

∫_{−∞}^{∞}(f^{(k)})^{2} = o( ∫_{−∞}^{∞} β_{pol}^{2})

as C → ∞. This property implies (2.8).

Appendix (iv): Derivation of (2.9) Noting the dilation invariance of (2.9), and the definition of a unimodal function

as a combination of two monotone functions, we see that it suffices to prove the existence of a constant C_{k} > 0 such that, if g is a nonnegative monotone decreasing function on [0, ∞) with k square-integrable derivatives, then

∫_{0}^{∞} |(log g)^{(k)}(x) g(x)|^{2} dx ≤ C_{k} ∫_{0}^{∞} |g^{(k)}(x)|^{2} dx .

Introduce the one-sided Hardy-Littlewood maximal function,

Mψ(x) = sup_{r>0} \frac{1}{r} ∫_{0}^{r} |ψ(x + y)| dy .

Clearly Mψ(x) ≥ |ψ(x)|. The converse inequality is true in an L^{2} sense: ∥Mψ∥ ≤ C ∥ψ∥, where ∥ · ∥ denotes the usual L^{2} norm on [0, ∞). See e.g. Stein (1993). Therefore it suffices to show that for each x,

|(log g)^{(k)}(x) g(x)| ≤ C_{k} M{g^{(k)}}(x) .

Because this inequality is invariant under mappings g(x) ↦ g(σx) and g(x) ↦ c g(x), we may normalise so that for given x, g(x) = M{g^{(k)}}(x) = 1. In particular,

∫_{0}^{r} |g^{(k)}(x + y)| dy ≤ r  (A.7)

for all r ≥ 0.

Repeated applications of the chain rule, quotient rule, product rule and the normalisation g(x) = 1 show that (log g)^{(k)}(x) is a polynomial in the first k derivatives g^{(j)}(x) for 1 ≤ j ≤ k. Thus it suffices to show that

|g^{(j)}(x)| ≤ C_{k}  (A.8)

for all 1 ≤ j ≤ k. Since |g^{(k)}(x)| ≤ Mg^{(k)}(x) = 1 then it suffices to do this for 1 ≤ j ≤ k − 1.

From Taylor’s theorem with remainder we have the expansion

g(x + r) = P (r) + ∫_{0}^{r} \frac{(r − y)^{k−1}}{(k − 1)!} g^{(k)}(y) dy

where P (r) = ∑_{0≤i≤k−1} (r^{i}/i!) g^{(i)}(x). By monotonicity and the normalisation g(x) = 1 we have 0 ≤ g(x + r) ≤ 1 for all r ≥ 0. Combining this with (A.7) we deduce that −1 ≤ P (r) ≤ 2 for 0 ≤ r ≤ 1. Thus P is a polynomial of degree k − 1 which is bounded on the interval [0, 1]. This implies that all the coefficients of P are bounded, whence (A.7) follows.

REFERENCES

BEBCHUK J.D., LINDSEY J.C., RYAN, L.M. AND WAND, M.P. (2000). Local EM estimation of the hazard function for interval-censored data. Biometrics 55, 238–245.

BEBCHUK J.D. AND BETENSKY, R.A. (2000). Multiple imputation for simple estimation of the hazard function based on interval censored data. Statist. Med. 19, 405–419.

BEBCHUK J.D. AND BETENSKY, R.A. (2001). Local likelihood analysis of survival data with censored intermediate events. J. Amer. Statist. Assoc. 96, 449–445.

BETENSKY, R.A., LINDSEY, J.C., RYAN, L.M. AND WAND, M.P. (1999). Local EM estimation of the hazard function for interval censored data. Biometrics 55, 238–245.

CHENG M.-Y., HALL, P. AND TURLACH, B.A. (1999). High-derivative parametric enhancements of nonparametric curve estimators. Biometrika 86, 417-428

COPAS, J.B. (1995). Local likelihood based on kernel censoring. J. Roy. Statist. Soc. Ser. B 57, 221–235.

EGUCHI, S. AND COPAS, J. (1998). A class of local likelihood methods and nearparametric asymptotics. J. Roy. Statist. Soc. Ser. B 60, 709–724.

EL BARMI, H. AND SIMONOFF, J.S. (2000). Transformation-based density estimation for weighted distributions. J. Nonparam. Statist. 12, 861–887.

FAN, J.Q., FARMEN, M. AND GIJBELS, I. (1998). Local maximum likelihood estimation and inference. J. Roy. Statist. Soc. Ser. B 60, 591–608.

HILI, O. (2001). Nonparametric density estimation of a stationary mixing process. C. R. Acad. Sci. Ser. I–Math. 332, 841–844.

HJORT, N.L. & JONES, M.C. (1996). Locally parametric nonparametric density estimation. Ann. Statist. 24, 1619–1647.

JONES, M.C. AND SIGNORINI, D.F. (1997). A comparison of higher-order bias kernel density estimators. J. Amer. Statist. Assoc. 92, 1063–1073.

LOADER, C.R. (1996). Local likelihood density estimation. Ann. Statist. 24, 1602– 1618.

SIMONOFF, J.S. (1996). Smoothing Methods in Statistics. Springer, New York.

SIMONOFF, J.S. (1998). Three sides of smoothing: Categorical data smoothing, nonparametric regression, and density estimation. Internat. Statist. Rev. 66, 137–156.

STEIN, E. (1993). Harmonic Analysis: Real Variable Methods, Orthogonality, and Oscillatory Integrals. Princeton University Press, Princeton.

TENREIRO, C. (1999). Asymptotic normality of local polynomial estimators of density function derivatives for strong mixing real processes. C. R. Acad. Sci. Ser. I–Math. 328, 1085–1088.