From daemon Wed Nov  8 14:34:27 1989
Received: from bacchus.eng.umd.edu by eneevax.eng.umd.edu  (5.52/4.7)
	id AA23855; Wed, 8 Nov 89 14:34:25 EST
Received: from adonis.eng.umd.edu by bacchus.eng.umd.edu (4.0/SMI-4.0)
	id AA06130; Wed, 8 Nov 89 14:35:14 EST
Message-Id: <8911081935.AA06130@bacchus.eng.umd.edu>
Date: Wed, 8 Nov 89 14:35:44 EST
From: ejf@bacchus.eng.umd.edu (Eleanor J. Fisher)
To: Armand@bacchus.eng.umd.edu
Subject: Narayan's Problems and Exams
Status: RO

\magnification=\magstep1
\pageno=1
\baselineskip=11pt
\input /homes/eefac/ejf/macro.tex
\hfill Spring 89
\smallskip
\hfill Problem Set 2
\smallskip
\hfill PN
\smallskip
\baselineskip=18pt
\medskip\par
Let $X,~Y,$ and $Z$ be $\R$-valued r.v.'s such that $\alpha X +
\beta Y + \gamma Z$ is Gaussian for every $\alpha,~Beta$ and
$\gamma$ in $\R$.  Using the orthogonality principle, show that
the conditional expectation of $X$ given $Y$ and $Z$ satisfies
$$E[X/Y,Z] = E\left [X/Y,Z - E[Z/Y]\right ].$$
\bigskip
\par
Two observations $Y_1$ and $Y_2$ of a $\R$-valued r.v. $\theta \sim N
(1,2)$ are made as fillows:
$$\eqalign{Y_1 &= (1 + N_1)\theta\cr
Y_2 &= (1 + N_2)\theta\cr}$$
where $(N_1,N_2) \coprod \theta ,~E[N_1] = E[N_2] = 0 ,~E[N_1^2] =
E[N_2^2] = 1$ and $N_1 \perp N_2$.
\medskip
\itemitem{(a)}  Find $\hat E[\theta /Y_1]$.
\medskip
\itemitem{(b)}  Find $\hat E[\theta/Y,Y_2]$ in terms of $\hat E[\theta/Y_1]$
and $Y_2$, i.e., without re-doing the computations from scratch.
\bigskip\par
Consider the one-dimensional system
$$\theta_{k+1} = \theta_k + bW_k , \quad\quad k=0,1,\ldots$$
where the observation process is
$$Y_k = \theta_k + W_k.$$
Assume that $\{W_k\}_o^\infty$ is a sequence of zero mean Gaussian
r.v.'s with $E[W_kW_\ell] = Q_k \delta_{k\ell}$.  Further, let
$\theta_o \sim N(0,\Sigma_o)$ and $\rm{cov}(\theta_o,W_k) = 0~~
\forall~k; b$ is a (known) constant.  Define $Y^k \buildrel 
\Delta \over = (Y_1,Y_2,\ldots Y_k)$.
\medskip
\itemitem{(a)}  Find a recursive scheme for generating $\hat E[\theta_k
/Y^k]$.
\itemitem{(b)}  Find a recursive scheme for generating $\hat E
[\theta_{k+N} /Y^k].$
\bigskip\par
Let $Y$ be a $\R^n$-valued r.v.   Let $\theta$ be in $\R$ and let $g$
be any estimator of $\theta$ based on $Y$.  Define $\beta(\theta)
\buildrel \Delta \over = E_\theta [g(Y)] - \theta$ to be the bias of
the estimator; assume $\beta(1)$ is differentiable.  Then, under
appropriate smoothness conditions, show that
$$\Sigma (g/\theta) \geq {\left (1 + {d\over d\theta} \beta(\theta)\right )^2
\over E_\theta \left [\left |{\partial \over \partial\theta} \ell n~
f_\theta (Y)\right |\right ]}\qquad \forall~~\theta \in \Theta$$
(Cram\'er-Rao lower bound for biased estimators.)
\bigskip\par
Let $\theta$ be $\R$-valued and let $g$ be an estimator of $\theta$
based on the observation $Y$.
\medskip
\itemitem{(a)}  Derive the Cram\'er-Rao lower bound on the variance of
$g$.
\medskip
\itemitem{(b)}  If $f(\theta)$ is the density of the r.v. $\theta$ and
$\beta(\theta) = E_\theta [g(Y)] - \theta$ is the bias of $g$, then,
under the assumption that $\lim_{|\theta |\to\infty} f(\theta)\beta(\theta)
= 0$, $g$ is said to be efficient if the C-R bound is satisfied with
equality, i.e., $E[|\theta - g(Y)|^2] = M^{-1}$.  Show that if an
efficient estimator exists, it must coincide with $g_{MAP}$ and
$g_{MSE}$.
\vfill
\eject