From daemon Wed Nov 8 14:42:43 1989 Received: from bacchus.eng.umd.edu by eneevax.eng.umd.edu (5.52/4.7) id AA23923; Wed, 8 Nov 89 14:42:29 EST Received: from mona.eng.umd.edu by bacchus.eng.umd.edu (4.0/SMI-4.0) id AA06175; Wed, 8 Nov 89 14:43:05 EST Message-Id: <8911081943.AA06175@bacchus.eng.umd.edu> Date: Wed, 8 Nov 89 14:43:34 EST From: saroj@bacchus.eng.umd.edu (Saroj Bhandari) To: armand@bacchus.eng.umd.edu Subject: Dr. Makowski here is a file I am mailing you for your doing the Cc: saroj@bacchus.eng.umd.edu Status: RO needful. I have lost some pages probably in 1983 text. I did not have originals with me so I couldn't type in. Sorry about that. \magnification=\magstep1 \nopagenumbers \hfill Fall 79 \smallskip \hfill Problem set 5 \smallskip \hfill AM \medskip \itemitem{}($Y_1,Y_2,\ldots,Y_n$) are $i.i.d.$ $r.v.'s$ distributed $\sim N(m,\sigma^2$). \medskip \itemitem{(a)}Find the (joint) maximum likelihood estimates of m and $\sigma^2$ in terms of\break $(Y_1,Y_2,\ldots Y_i$), i=1,2,...,n \medskip \itemitem{(b)}Evaluate each estimate for bias \medskip \itemitem{(c)} Evaluate the error covariance $E[(\hat m - m)^2], E[(\hat s -s)^2]$ and $E[(\hat m - m) (\hat s - s)]$. \medskip \itemitem{}Let $Y$ be a random variable with density $f_{\theta}(Y)$, where $\theta$ is an unknown (deterministic) parameter. Let g(y) be an estimate of $\theta$ based on the observation Y = y and define the bias of the estimator $g(\cdot)$ as $\beta(\theta) = E_\theta[g(y) - \theta]$. Show that the variance of g is bounded below as $$var_\theta [g(y)] \geq~~{(1 + {\partial \over \partial \theta} \beta (\theta))^{2}\over E[\{ {\partial \over \partial \theta} ln (f_\theta (y))\}^{2}]}$$ \medskip \def\xx{\mathop{\raise1ex\hbox{$<$}\kern-.62em\lower.5ex\hbox{$>$}}\limits} \def\R{\mathop{\rm I\kern -0.20em R}\nolimits} \hfill Fall 79 \smallskip \hfill Problem Set 5 \smallskip \hfill AM \bigskip\noindent Consider the following binary hypothesis testing problem \medskip $$\eqalign { H_0:&Y = N\cr H_1:&Y = S + N\cr}$$ \noindent where $S$ and $N$ are independent, $\R$-valued random variables with densities given by $$f_s(s) = \left\{\vbox{\halign{$\displaystyle{#}$\hfil&\hfil$\displaystyle{#} $\cr a \exp (-as)&,\quad s\geq 0\cr 0&,\quad s < 0\cr}}\right.$$ \noindent and $$f_N(n) = \left\{\vbox{\halign{$\displaystyle{#}$\hfil&\hfil$\displaystyle{#}$\cr b \exp (-bn)&,\quad n\geq 0\cr 0\hfill &,\quad n < 0\cr}}\right.$$ \item{(a)} Prove that the likelihood ratio test reduces to $$Y \xx_{H_1}^{H_0} c.$$ \medskip \item{}where c is a suitably chosen constant. Specify c in terms of cost and a prior probabilities for a Bayes test. \medskip \item{(b)}Evaluate the miss and false alarm probabilities as a function of c and plot the receiver operating characteristics - a plot of the detection probability as a function of the false alarm probability for various values of c. \medskip \item{(c)}If the test is to be designed to maximize the detection probability when the false alarm probability is 0.0001, determine the corresponding value of c and the detection probability. \medskip \item{(d)}Describe the functional form of the receiver operating characteristics, i.e. $P_D = f(P_F)$. \vfill \eject Year 82 files start from here \hfill Spring 82 \smallskip \hfill Spring Set 2 \smallskip \bigskip \noindent Under Hypothesis $H_0$, let $y(t) = s_0(t) + v(t)$ and under $H_1$, let $y(t)= s_1(t) + v(t)$ where $s_0(t) = A,~~s_1(t)=\sqrt 2~ A ~cos (\omega t)$ and $v(t)$ is white Gaussian noise with zero mean. Find a Neyman-Pearson test and the detection probability $P_D$ for \medskip\noindent (i)~~Unknown~$A~ \epsilon \R$ \medskip\noindent (ii)~~Known~~~$A~ \epsilon \R$ \vfill \eject \hfill Spring 82 \smallskip \hfill Problem set 2 \smallskip \hfill PN \bigskip\noindent Let ($N_1, N_2,\ldots, N_n)$ and ($S_1, S_2,\ldots, S_n$) be mutually independent $i.i.d$ sequences, the $N_1\prime s$ being distributed $N(0,\sigma_N^2)$ and the $S_i's$ being distributed $N(0,\sigma_S^2$). Consider the\break hypothses \bigskip \settabs 8\columns \+&$H_0$:&$Y_i$ = $N_i$i,&&1$\leq i \leq$ n\cr \medskip \+&$H_1$:&$Y_i$ = $S_i+N_i$,&&1$\leq i \leq$ n.\cr \medskip\noindent If $\sigma_N$ is known but $\sigma_S$ is unknown, does a UMP test exist for testing $H_0$ against $H_1$? \vfill \eject \hfill Spring 82 \smallskip \hfill Final \smallskip \hfill PN \bigskip\noindent \underbar {Problem 1.} \medskip\noindent Consider the following hypothesis testing problem: \medskip \settabs 8\columns \+&$H_0$:&$X_i$ = $n_i$,&&i = 1,$\ldots$ ,k\cr \+&$H_1$:&$X_i$ = $s_i$ +$n_i$,&&i = 1,$\ldots$ ,k\cr \medskip\noindent where the $\bigl\{n_i\bigr\}$ are zero mean, Gaussian r.v.'s with $E\bigl[n_i n_j\bigr]$ = minimum $\{i,j\}$. \medskip\noindent For a given value of the false-alarm probability $P_F = \alpha$, find the Neyman-Pearson test for the following cases \medskip\noindent \settabs 8\columns \+&(a)&$s_i = 1$,&&i=1,$\ldots$ ,k;\cr \medskip \+&(b)&$s_i$ = i,&&i = 1, $\ldots$ k.\cr \medskip\noindent (c) Compute the detection probability $P_D$ for both cases. \vfill \eject \hfill Spring 82 \smallskip \hfill Final \smallskip \hfill PN \bigskip \item{}An \underbar {affine} estimator of a random variable X given the observations $Y_1, Y_2,\ldots, Y_n$ is of the form \medskip \itemitem{}$\hat X = \sum\limits_i \alpha_i Y_i + \beta$, where $\alpha_i$ and $\beta$ are constants. \medskip\noindent Show that if X has mean $m_X$ and the $Y_1$ are independent with means $E[Y_i] = m_i$, then the affine estimator which minimizes the mean square error is $$\hat X = \sum_i \alpha_i^* (Y_i - m_i) + m_X,$$ \noindent and specify $\alpha_i^*$ in terms of the second-order properties of X and the $\bigl\{Y_i\bigr\}$. \vfill \eject \smallskip \item{}Suppose we wish to estimate a signal which is a random walk: $$X_{k+1} = X_k + W_k~~~~,~~X_0=X$$ \noindent where $E[X] = 0, ~E[X^2] = \sigma_x^2$ and where $\bigl\{W_k\bigr\}$ is a sequence of independent r.v.'s with zero mean and variance $\sigma_W^2$, and is independent of $X_0=X$. The observations are $$Y_k = X_k + Z_k,$$ \noindent where $\bigl\{Z_k\bigr\}$ is a sequence of zero mean, independent r.v.'s, independent of $X_0=X$, with $E\bigl [Z_k^2\bigr] = \sigma_z^2~ \forall k.$ Also, the $\bigl \{W_k\bigr\}$ are independent of the $\bigl \{Z_k\bigr\}$. Let $\hat X_{k+1/k} = E^* \bigl \{X_{k+1}/ Y_0, \ldots, Y_k\bigr \}$, the best linear estimator of $X_{k+1}$ in terms of $Y_0,Y_1,\ldots, Y_k$ and let $\Sigma_{k+1} = E\bigl [\bigl\{ X_{k+1} - \hat X_{k+1/k}\bigr\}^2\bigr].$ \bigskip\noindent Without using the Kalman expression directly \bigskip \itemitem{(a)}Show that $\hat X_{k+1/k}$ is generated by $$\hat X_{k+1/k} = \hat X_{k/k-1} + \alpha_k \bigl(Y_k - \hat X_{k/k-1} \bigr), ~~\hat X_{0/-1} = 0$$ \noindent and determine $\alpha_k$ in terms of $\Sigma_k$ and $\sigma_z^2$. \medskip \itemitem{(b)}Show that $\Sigma_k$ is generated according to $$\Sigma_k = {\sigma_z^2~\Sigma_{k-1}\over {\sigma_z^2 + \Sigma_{k+1}}} + \sigma_W^2,~\Sigma_0 = \sigma_x^2$$. \vfill \eject \hfill Spring 82 \smallskip \hfill Final \smallskip \hfill PN \bigskip\noindent \underbar {Problem 1.} \medskip\noindent Consider the following hypothesis testing problem: \medskip \settabs 8\columns \+&$H_0$:&$X_i$ = $n_i$,&&i = 1,$\ldots$ ,k\cr \+&$H_1$:&$X_i$ = $s_i$ +$n_i$,&&i = 1,$\ldots$ ,k\cr \medskip\noindent where the $\bigl\{n_i\bigr\}$ are zero mean, Gaussian r.v.'s with $E\bigl[n_i n_j\bigr]$ = minimum $\{i,j\}$. \medskip\noindent For a given value of the false-alarm probability $P_F = \alpha$, find the Neyman-Pearson test for the following cases \medskip\noindent \settabs 8\columns \+&(a)&$s_i = 1$,&&i=1,$\ldots$ ,k;\cr \medskip \+&(b)&$s_i$ = i,&&i = 1, $\ldots$ k.\cr \medskip\noindent (c) Compute the detection probability $P_D$ for both cases. \vfill \eject \hfill Spring 82 \smallskip \hfill Final \smallskip \hfill PN \bigskip \item{}An \underbar {affine} estimator of a random variable X given the observations $Y_1, Y_2,\ldots, Y_n$ is of the form \medskip \itemitem{}$\hat X = \sum\limits_i \alpha_i Y_i + \beta$, where $\alpha_i$ and $\beta$ are constants. \medskip\noindent Show that if X has mean $m_X$ and the $Y_1$ are independent with means $E[Y_i] = m_i$, then the affine estimator which minimizes the mean square error is $$\hat X = \sum_i \alpha_i^* (Y_i - m_i) + m_X,$$ \noindent and specify $\alpha_i^*$ in terms of the second-order properties of X and the $\bigl\{Y_i\bigr\}$. \vfill \eject \smallskip \item{}Suppose we wish to estimate a signal which is a random walk: $$X_{k+1} = X_k + W_k~~~~,~~X_0=X$$ \noindent where $E[X] = 0, ~E[X^2] = \sigma_x^2$ and where $\bigl\{W_k\bigr\}$ is a sequence of independent r.v.'s with zero mean and variance $\sigma_W^2$, and is independent of $X_0=X$. The observations are $$Y_k = X_k + Z_k,$$ \noindent where $\bigl\{Z_k\bigr\}$ is a sequence of zero mean, independent r.v.'s, independent of $X_0=X$, with $E\bigl [Z_k^2\bigr] = \sigma_z^2~ \forall k.$ Also, the $\bigl \{W_k\bigr\}$ are independent of the $\bigl \{Z_k\bigr\}$. Let $\hat X_{k+1/k} = E^* \bigl \{X_{k+1}/ Y_0, \ldots, Y_k\bigr \}$, the best linear estimator of $X_{k+1}$ in terms of $Y_0,Y_1,\ldots, Y_k$ and let $\Sigma_{k+1} = E\bigl [\bigl\{ X_{k+1} - \hat X_{k+1/k}\bigr\}^2\bigr].$ \bigskip\noindent Without using the Kalman expression directly \bigskip \itemitem{(a)}Show that $\hat X_{k+1/k}$ is generated by $$\hat X_{k+1/k} = \hat X_{k/k-1} + \alpha_k \bigl(Y_k - \hat X_{k/k-1} \bigr), ~~\hat X_{0/-1} = 0$$ \noindent and determine $\alpha_k$ in terms of $\Sigma_k$ and $\sigma_z^2$. \medskip \itemitem{(b)}Show that $\Sigma_k$ is generated according to $$\Sigma_k = {\sigma_z^2~\Sigma_{k-1}\over {\sigma_z^2 + \Sigma_{k+1}}} + \sigma_W^2,~\Sigma_0 = \sigma_x^2$$. \vfill \eject \line {} \vskip .5truein \hfill Spring 82 \smallskip \hfill Problem Set 2 \bigskip \noindent \settabs 8\columns \medskip \+Let&$H_0$:&y(t) = v(t)&&0$\leq t \leq$ T\cr \+&$H_1$:&y(t) = s(t) + v(t)&&0$\leq t \leq$ T\cr \noindent where $s(\cdot)$ is a known signal that vanishes outside [0,T] and is differentiable, v(t) is Gaussian noise with zero mean and $E[v(t)v(s)] = min(t,s)$. Find a Neyman-Pearson test to test $H_0$ versus $H_1$ based on observing $y(t),~ 0\leq + \leq T$. \vfill \eject \hfuzz=4pt \def\xx{{\dot y}} \def\cov{\rm cov} \hfill Spring 82 \smallskip \hfill Final \smallskip \hfill PN \bigskip\noindent \bigskip \itemitem{}Consider the smooth n-dimensional linear system with state equation $$\dot x(t) = F(t)x(t) + G(t) \omega(t)$$ \itemitem{}and m-dim. output $$z(t) = H(t) x(t) + \nu (t)$$ \noindent The initial state $x(t_0)$ is a random vector with $E[x(t_0)] = x_0$, $\rm {cov} \bigl[x(t_0), x(t_0)\bigr] = \sigma_0$. The q- and m-dimensional random vectors $\omega(t)$ and $\nu (t)$ are white Gaussian noises with $$\left.\eqalign{E [\omega (t)]&= E[\nu (t)] = 0,\cr \cov \left [ w(t), w(s)\right ]&= Q(t) \delta (t,s),~ {\rm Q~non-neg.~definite}\cr \cov \left [ v(t), v(s)\right ]&=R(t)\delta (t-s),~{\rm R~positive~definite}\cr {\rm and,}~\cov \left [ w(t), v(s)\right ]&=0, ~\cov\left [ x(t_0),w(t) \right ]\cr &=\cov \left [ x(t_0), v(t)\right ] = 0\cr}\right\} \forall t,s$$ \noindent For a particular output of the system $$y(t) = M(t) x(t)$$ \noindent find the \underbar {optimal differentiator}, i.e., find \smallskip $\hat\xx (t/t) = E^*\bigl\{\dot y(t)/Z_t\bigr\}$ in terms of $\hat X(t/t) = E^* \bigl\{x(t)/Z_t\bigr\},$ and other system parameters. Could this be obtained by forming $\hat y(t/t) = M(t) \hat x(t/t)$ and then differentiating the result? Explain your answer. \vfill \eject \bigskip \itemitem{}Let us assume that an $\imath$-dim. measurement vector $z_0$ can be expressed as a linear combination of the n-dim. Constant vector X, plus a random, additive measurement error vector v, i.e., $$z_0 = H_0 x + \nu,$$ where $H_0$ is an $\imath \times n$ \noindent matrix. We choose the estimate $\hat x(\cdot)$ as the one which minimizes $$\bigl(z_0 - H_0 \hat x\bigr)^T R_0^{-1} (z_0 - H_0\hat x),$$ \noindent where $R_0^{-1}$ is an $\imath \times\imath $ symmetric, positive definite matrix. \medskip \itemitem{(a)}Show that $\hat x(\cdot)$ can be expressed in the form $\hat X(-) = \alpha_0 z_0,$ and find $\alpha_0$ in terms of $H_0$ and $R_0$. \medskip\noindent Now, suppose that an additional measurement vector, z, becomes available. Define the following matrices for the complete measurement set: $$H_1 = \biggl[\matrix{H_0\cr ---\cr H\cr}\biggr],~z_i = \biggl[\matrix{z_0 \cr ---\cr z\cr}\biggr], R_1 = \biggl[\matrix{R_0&|&0\cr --&--&--\cr 0&|&R\cr}\biggr]$$ \itemitem{(b)}Find the new estimate, $\hat x(t)$, in terms of $\underline {z_1, H_1 \rm {and}~ R_1}$. \medskip \itemitem{(c)}Using the definitions of $H_1, R_1, z_1$, above, show that $\hat x(t)$ can be manipulated into the form: $$\hat x(t) = \hat x(\cdot) + P(+) H^T R^{-1} \bigl[z - H\hat x(-)\bigr]$$ $$\rm {with}~ P^{-1} (+) = P^{-1} + H^T R^{-1} H$$ $$\rm {where}~ P^{-1}~ \rm {is~ defined~ by}$$ $$P^{-1} (-)\buildrel \Delta \over = H_0^T R_0^{-1} H_0.$$ \vfill \eject \hfill Spring 82 \smallskip \hfill Final \smallskip \hfill PN \bigskip\noindent \bigskip Consider the one-dimensional system \medskip $$\dot X(t) = X(t) + bw(t)$$ where the observation process is $$Z(t) = X(t) + W(t),$$ \noindent where $W(t)$ is white Gaussian noise with mean zero and unit spectral density. Let $X(0)\sim N (0,\sigma_0)$ and assume that $cov~[X(0), W(t)] = 0~ \forall~ t.$ Determine the best linear estimator $\hat X(t/t) \buildrel \Delta \over = E^* \bigl\{X(t)/Z_t\bigr\}$ of $X(t)$ given $Z_t$, as also the covariance $\sum (t)$ of the estimation error $\tilde X(t/t) \buildrel \Delta \over = X(t) - \hat X(t/t).$ \vfill \eject Now Year 1983 files from now onwards \hfill Spring 83 \smallskip \hfill Test 1 \smallskip \hfill AM \bigskip\noindent Consider the following binary hypothesis testing problem where under each hypothesis the decision is Rayleigh distributed with a known parameter; i.e. under $H_i,~i =0, 1$, the r.v. Y is distributed as $$f(y/H_i) = \cases{(y/\sigma_i^2)~\exp (- y^2/2\sigma_i^2),& $y\geq 0$\cr 0, &$y < 0$\cr}.$$ \noindent Assume that $0 \leq \sigma_0^2 \leq \sigma_1^2.$ \bigskip \itemitem{(a)} Construct a LRT for this situation and find the test statistic. In terms of the test statistic, describe the various decision regions induced by the test. Give an example of a test which is not of the likelihood ratio type. \medskip \itemitem{(b)} Compute the probability of detection, $P_D(\eta)$, and the probability of false alarm, $P_F(\eta)$, for the test constructed in (a). Give explicit expressions in terms of $\eta,~\sigma_0^2~ \rm {and}~\sigma_1^2$. \medskip \itemitem{(c)}Under the MPE criterion, find the minimax solution to this binary hypothesis testing problem. \medskip \itemitem{(d)} Describe completely the decision regions induced by a level-$\alpha$ Neyman-Pearson test, $\alpha \epsilon (0, 1)$, and express the power, $\beta (\alpha)$, of this test in terms of $\alpha,~\sigma_0^2$ and $\sigma_1^2$. \medskip \itemitem{(e)}Obtain analytical expressions for the receiver operating characteristics of this binary hypothesis testing problem (i.e., a relationship $P_D = f(P_F)$. Calculate $dP_D/dP_F$ and $d^2P_D/dP_F^2$ and infer from your calculations the geometric properties of the ROC. \vfill \eject \bigskip\noindent \hfill Spring 83 \smallskip \hfill Test 1 \smallskip \hfill AM \bigskip\noindent Consider the problem of detecting known signals with finite energy in the presence of Gaussian noise. Let , for $ i = 1, 0 $, \bigskip ~~~~~~~~~$H_i$~:~~~Y(t) = ~~$S_i(t)$ ~ + ~N(t)~,~~~t$\epsilon$ [0, T], \bigskip\noindent where N(t) is WGN and $S_i(t)$, i = 0, 1, are finite energy signals. \bigskip \noindent Taking the Neyman-Pearson approach one seeks, amongst all likelihood ratio tests, the one which maximizes the probability of detection for a given upper level on the probability of false alarm. How should the signals $S_0$ and $S_1$ be chosen to maximize the probability of detection provided the energy constraint. $${1\over 2} \int_0^T \bigl( \mid S_0(t)\mid^2 ~ + ~ \mid S_1(t)\mid ^2\bigl) dt ~=~E$$ \noindent is imposed? Is the answer surprising? Explain. \vfill \eject through a sequence of N independent tosses of the coin, $1 \leq N < $ \medskip \itemitem{(a)} Rephrase the problem as one of binary hypothesis testing. \medskip \itemitem{(b)} For $1 \leq N < \infty$, construct a LRT for testing the held belief and identify the natural sufficient statistic, $K_N$. Find the probability distribution of $K_N$ under each hypothesis and explain how this can be used to find the smallest number, $N^*$, of trials needed to guarantee that the probability that the coin is determined to be biased when it is actually fair is at most $\alpha \epsilon (0,1)$. \medskip \itemitem{(c)}For $0 < A < B$, write a sequential LRT with boundaries A and B. Conclude from the form of this test that it will (eventually) terminate in finitely many steps with probability one. \smallskip \itemitem{}(Hint: Recall the law of large number for Bernoulli trials). \vfill \eject \hfill Spring 83 \smallskip \hfill Test 2 \smallskip \hfill AM A $\R$-valued parameter $\theta$ is observed through two independent observations, $Y_1$ and $Y_2$, corrupted by independent Gaussian noise; i.e. ~~~~~$Y_1 = \theta + V_1$ ~ and ~ $Y_2 = \theta + V_2$ \medskip\noindent where $V_1 \sim N(0,\sigma_1^2)~,~V_2 \sim N(0, \sigma_2^2),~~V_1, V_2 \perp \theta ,~~V_1, \perp, V_2$ \bigskip \itemitem{(a)} If $\theta$ is modelled as a $r.v$. distributed $N(0,s^2)$, find $g_{MS}$, $g_{MAP}$ as functions of $Y_1$ and $Y_2$ and compute their variance. \medskip \itemitem{(b)} If $\theta$ is an unknown but deterministic constant, find $g_{ML}$ and compute its efficiency and variance. \vfill \eject \hfill Spring 83 \smallskip \hfill Test 2 \smallskip \hfill AM \medskip A $\R$-valued parameter $\theta$ is observed through two independent observations, $Y_1$ and $Y_2$, corrupted by independent Gaussian noise; i.e. ~~~~~$Y_1 = \theta + V_1$ ~ and ~ $Y_2 = \theta + V_2$ \medskip\noindent where $V_1 \sim N(0,\sigma_1^2)~,~V_2 \sim N(0, \sigma_2^2),~~V_1, V_2 \perp \theta ,~~V_1, \perp, V_2$ \bigskip \itemitem{(a)} If $\theta$ is modelled as a $r.v$. distributed $N(0,s^2)$, find $g_{MS}$, $g_{MAP}$ as functions of $Y_1$ and $Y_2$ and compute their variance. \medskip \itemitem{(b)} If $\theta$ is an unknown but deterministic constant, find $g_{ML}$ and compute its efficiency and variance. \vfill \eject \hfill Spring 83 \smallskip \hfill Test 2 \smallskip \hfill AM \bigskip\noindent Consider the $\R$-valued, zero mean, wide sense stationary stochastic processes $\{X(\tau), ~\tau \epsilon \R\}$ ~and ~ $\{N(\tau)~,~\tau \epsilon \R\}$ which are uncorrelated, i.e. $$E [X(\tau) N(\sigma)] = 0 ~~~\forall~~\tau, \sigma \epsilon \R.$$ \noindent and whose power spectral densities are given by $$\Phi_X (\nu) = {1\over 1+\nu^2}~,~ \Phi_N(\nu) = 1/2.$$ \noindent If the observation process is \medskip $$Y(t) ~ = ~ X(t)~+~ N(t) ~~~~~\forall ~t \epsilon \R$$ \noindent determine the optimum causal time-invariant filter that generates the linear mean square estimate of $X(t)$ based on $\{Y(\tau)~,~ \tau \leq t\}.$ \vfill \eject \hfill Spring 83 \smallskip \hfill Test 2 \smallskip \hfill AM \bigskip\noindent Let $\{X(t)~,~ t\geq 0\}$ be a zero-mean mean-square continuous $\R$-valued second-order stochastic process. Define two random variables X and Y as $$X = \int_0^1 X(t)dt~~and~~ Y = (X(0)~X(1/2)~X(1))^\prime$$ \noindent An approximation to the integral in terms of the value of the integrand at particular points is sought. \medskip\noindent Compute $\hat E [ X/Y=y ]$, when $\{X(t), t\geq 0\}$ is a standard Brownian motion starting from the origin, using the orthogonality principle. \vfill \eject \hfill Spring 83 \smallskip \hfill Test 3 \smallskip \hfill AM \bigskip\noindent Let $\{Y_k\}_1^n$ be a sequence of $i.i.d$ $r.v.'s$ distributed $N(\mu, \sigma^2)$. If $\theta = (\mu,\sigma^2)$ and $Y = \bigl(Y_1, Y_2,\ldots , Y_n\bigr)$, \bigskip \itemitem{(a)}Find the joint maximum likelihood estimates $\hat g_\mu$ and $\hat g_{{\sigma^{2}}}$ of $\mu$ and $\sigma^2$ in terms of Y. \medskip \itemitem{(b)}Evaluate the bias of each estimate. \medskip \itemitem{(c)}Which one of these estimates is consistent? \medskip \itemitem{(d)}Evaluate the error co-variance $$V(\mu) = E_\theta \bigl[\bigl\{\hat g_\mu (Y) - \mu\bigr\}^2\bigr]$$ \medskip $$V(\mu,\sigma^2) = E_\theta \bigl[\bigl\{\hat g_\mu (Y) - \mu \bigr\} \bigl\{\hat g_\sigma^{2}(Y) - \sigma^2\bigr\}\bigr]$$ \itemitem{}and decided if $\hat g_\mu (Y)$ and $\hat g_\sigma^{2} (Y)$ are independent. \medskip \itemitem{(e)}Show that $\hat g_\mu (Y)$ and $\hat g_{{\sigma^2}} (Y)$ are uncorrelated. \vfill \eject \hfuzz=2pt \def\R{\mathop{\rm I\kern -0.20em R}\nolimits} \hfill Spring 83 \smallskip \hfill Test 3 \smallskip \hfill AM \bigskip\noindent Let $\{X(t), t\geq 0\}$ be a $\R$-valued, zero-mean, mean-square continuous second-order stochastic process which is wide sense stationary with covariance structure $$R(\tau) = E [X(t) X(t+\tau)] = ~\sigma^2 e^{-\beta \tau}~~\forall t, \tau \geq 0.$$ \noindent where $\sigma$ and $\beta$ are positive constants. \medskip\noindent Define the $r.v.'s$ X and Y as $$X = \int_0^1 X(t)dt ~~~~\rm {and}~~Y = (X(0) ~X(1/2) ~~X(1))^\prime.$$ \noindent An approximation to the integral in terms of the integrand at particular points is sought. \medskip\noindent Compute $\hat E [X/Y=y]$ as one such approximation using the orthogonality principle. \vfill \eject \hfill Spring 83 \smallskip \hfill Final \smallskip \hfill AM \noindent \item{1.}Consider the following problem of detecting known discrete-time signals in noise: \medskip \itemitem{}In a simple binary communication system, the transmitter sends out one of two signals. The waveform $\bigl\{y_t^0\bigr\}_1^T$ and $\bigl\{y_t^1\bigr\}_1^T$ of these signals are \underbar {completely known}; however, when transmitted over the channel, these signals are corrupted by \underbar {additive noise} $\bigl\{n_t\bigr\}_1^T$. A receiver needs thus to be designed that operates on the received signal $\bigl\{z_t\bigr\}_1^T$ and that decides which signal was originally sent. \medskip \itemitem{(a)}Cast this problem as a binary hypothesis testing problem \medskip \itemitem{}From now on, assume the noise sequence $\bigl\{n_t\bigr\}_1^T$ to be a \underbar {zero-mean Gaussian} sequence with known, \underbar {invertible} covariance matrix P (not necessarily diagonal, i.e., the RV's $\bigl\{n_t\bigr\}_1^T$ are not necessarily independent of each other). Give a matched filter implementation of the result. Compute the corresponding probabilities of false alarm and detection in terms of the \underbar {standard normal distribution} and the \underbar {parameter} and $d^2$ where $$d^2 : \sum_{s-1}^T \sum_{t-1}^T R_{st} \Delta y_s \rm {and}~ \Delta y_t : y_t^1 - y_t^0~~~, 1 1: X(t) = 1\}\rm {if~ the~ set}~\{ t > 1 : X(t) = 1 \} is~non-empty$\cr \infty&$otherwise$\cr}$$ \noindent Show that $\theta$ is well defined and that the sample paths of $\bigl\{ X(t)\bigr\}^\infty$ can be entirely characterized in terms of $\theta$. Find the distribution of $\theta$ under both hypotheses. \medskip \itemitem{(c)}Express the SPRT (A,B) in terms of the RV $\theta$ and find a natural test statistic. Show by a direct computation that the SPRT (A,B) will terminate in finitely many steps (with probability one). \medskip \itemitem{d)}Study the problem of designing the boundaries A and B so that the corresponding SPRT (A0,$ i.e., A has a probability density function $P_A$ given by $$P_A(a)~=~\cases{(a/\alpha)\exp&$ [-a^2/2\alpha]~\rm {if}~a>0$\cr 0&$otherwise$\cr}$$ \itemitem{(a)} Compute the likelihood ratio function \underbar {explicitly} and write down the corresponding likelihood ratio tests. \medskip \itemitem{(b)} Identify a \underbar {natural test statistic} for these tests and \underbar {describe its probability distribution under both hypotheses}. \medskip \itemitem{(c)} Obtain an expression for the probabilities of false alarm and detection for these likelihood ratio tests, using known distributions. Specialize the results of (a) (b) (c) to the case when the MPE criterion is used with the assumption of equal prior probabilities. \vfill \eject \hfill Spring 83 \smallskip \hfill Finals \smallskip \hfill AM \bigskip \noindent 6.~Consider a pair of zero-mean R-valued \underbar {wide-sense stationary} stochastic processes \break $\{y(\tau), \tau \epsilon R\}$ and $\{n(\tau), \tau \epsilon R\}$ which are \underbar {uncorrelated} and which have power spectral densities $\phi_y$ and $\phi_n$ given by $$\phi_y(\nu) = \bigl(1+\nu^2\bigr)^{-2}$$ \itemitem{(a)} Find the optimum filter that generates the linear mean square estimate of the mean-square derivative dy(t)/dt on the basis of the observed data $z(\tau), \tau < t\}$ where $$z(\tau) = y(\tau) + n(\tau)~~~,\tau \epsilon R$$ \noindent Compute the corresponding \underbar {mean-square error} as explicitly as possible. \medskip \itemitem{(b)}Is it true that $$LMSE ({dy \over dt} (t)) = {d\over dt} [LMSE (y(t))]?$$ \noindent where derivation is understood in the mean square sense? \medskip\noindent Explain. \vfill \eject \hfill Spring 83 \smallskip \hfill Finals \smallskip \hfill Am \bigskip\noindent 7.~The Kalman filter algorithms assume that the system model and the prior probabilities are known exactly. If there is mismatch between the assumed model and the actual system, the estimates may become biased and the actual mean square error may be very large even though the theoretical error variance obtained by solving the error variance equation may be small. This phenomenon is known as \underbar {divergence} and is the motivation behind this question. \medskip The actual message model is assumed to be $$x_a(t+1) = x_a(t) + m~~~,~t = 0, 1,...,T$$ \noindent where m is a constant, and the observations are given by $$z(t) = x_a(t) + v(t) = m~~,~t = 0, 1,...T$$ \noindent Here, the random variable $x_a(0)$ has zero and unit variance, and is independent of the zero-mean noise sequence $\{v(t)\}_0^T$ for which $E[v(s)v(t)] = \delta (t-s)$. The estimator however is designed under the assumption that messages and observations are in fact generated through the model. $$x_d(t+1) = x_d(t), z(t) = x_d(t) + v(t), ~t = 0,1,...,T$$ \noindent with $x_d(0) = x_a(0)$. \medskip\noindent Denote the actual error in the estimate by $\hat x(t\mid t),$ i.e. $$\hat x(t \mid t) : = x_a(t) - \hat x_d(t \mid t).$$ \noindent Determine expressions for the propagation of the variance of $\hat x (t\mid t)$. How does the actual error variance compare with the theoretical error variance (i.e., the variance of $\hat x_d(t\mid t) = x_d(t) - \hat x_d(t\mid t)$ at the end of T stages? What do you conclude from this simple calculation? \vfill \eject \hfill Spring 83 \smallskip \hfill Finals \smallskip \hfill AM \bigskip\noindent 8.~Consider the discrete-time model $$x(t+1) = \theta_x(t) + w(t), ~z(t) = x(t)~+~ v(t),~ t = 0,1,...$$ \noindent where \itemitem{(i)}The initial condition x(0) is a Gaussian R-valued RV with mean m and variance $\sigma^2$. \medskip \itemitem{(ii)} Each one of noise sequence $\{w(t)\}_0^\infty$ and $\{v(t)\}_0^\infty$ is an R-valued sequence of zero mean i.i.d. Gaussian RV's with variance $\sigma_w^2$ and $\sigma_v^2$ respectively. \medskip \itemitem{(iii)}The RV x(0) and the sequences $\{w(t)\}_0^\infty$ and $\{v(t)\}_0^\infty$ are mutually independent. \medskip\noindent However, the parameter $\theta$ in the state equation is not known exactly; it is assumed instead that $\theta$ takes one of N value $\theta_1,..., \theta_N$ with \underbar {equal} probabilities, \underbar {independent} of x(0) and $\{w(t), v(t)\}_0^\infty$. \medskip \itemitem{(a)}Show that the conditional mean estimate $\hat x(t\mid t)$ of the state can be expressed as $$\hat x(t\mid t) = \sum_{i=1}^N \hat x_i (t\mid t) p_i (z^t)$$ \noindent where $x_i (t\mid T)$ denotes the conditional mean estimate of the system assuming that the model used corresponds to $\theta_i$ and $p_i (z^t)$ is the a posteriori probability that $\theta_i$ is the true parameter after observing the data string $z^t : = (z(0),...,z(t)).$ \medskip \itemitem{(b)}Obtain a complete set of algorithms of \underbar {recursively} generate the conditional mean estimate $\hat x(t\mid t)$ of x(t). \medskip \itemitem{(c)}Can you describe \underbar {recursive} estimate of $\theta$ on the basis of $z^t$? Is this estimate consistent? \vfill \eject Now starts year 1984 \hfill Spring 84 \smallskip \hfill Test 1 \smallskip \hfill AM \bigskip\noindent 1.~Consider the situation where the observation is a non-negative RV Z. Hypothesis \break $H(\lambda), 0<\lambda$, is said to hold true if the RV Z is \underbar {exponentially distributed with parameter} $\lambda$, i.e., $$P[Z\leq z \mid H = H(\lambda)] = \cases{1-exp(-\lambda z)&$if z\geq 0$\cr 0&$if~z\leq 0$.\cr}$$ \medskip Given $\lambda_0 > 0$, let $\wedge$ be a non-empty subset of $(0, + \infty)$ such that $\lambda_0 \epsilon \wedge$. Consider the (composite hypothesis testing problem of \underbar {testing} $H(\lambda), \lambda \epsilon \wedge$, \underbar {against} $H(\lambda_0)$. \bigskip\noindent \underbar {Case 1}:~~$\wedge = \{\lambda_12\}$ \medskip \itemitem{(i)} Describe \underbar {all} the likelihood ratio tests for testing $H(\lambda_0)$~~~~~(5 pts.); \medskip \itemitem{(ii)}Find the corresponding \underbar {test statistic} ~~~(5 pts.) \medskip \itemitem{(iii)} Find \underbar {explicit} expressions for the probabilities of alarm and detection ~~~~~(10 pts.); \medskip \itemitem{(iv)} Give an \underbar {explicit} functional representation of the ROC curve ~~~~~(5 pts.); \medskip\noindent \underbar {Case 2}: $\wedge = (0, \lambda_0)$ and $\lambda$ is modelled as an unknown constant \medskip \itemitem{(v)} Discuss the existence of UMP'tests for testing the composite hypothesis $H(\lambda), 0 < \lambda < \lambda_0$, against its alternative $H(\lambda_0)$~~~~~~(5 pts.); \medskip\noindent \underbar {Case 3}: $\wedge = (0,\lambda_0) 0(\lambda_0, + \infty)$ and $\lambda$ is modelled as an unknown constant \medskip \itemitem{(vi)}: Discuss the existence of UMP tests for testing the composite hypothesis \break $H(\lambda), 0 < \lambda < \lambda_0$ or $\lambda_0 < \lambda$, against its alternative $H(\lambda_0)$ ~~~(5 pts.); \medskip \itemitem{(vii)} If UMP tests do not exist in (vi), construct the corresponding generalized likelihood ratio function ~~~~(5 pts.) \vfill \eject \hfill 84 \smallskip \hfill Test 1 \smallskip \hfill AM \bigskip\noindent 2.~ Consider the following problem of detecting known signals in Gaussian colored noise, i.e., $$H_i : Z(t) = s_i(t) + N(t), ~~ 0\leq t \leq T,$$ \noindent where \underbar {under} $H_i,~~ 1 = 0$ or 1, \medskip \itemitem{(1)}: $s_i \epsilon L^2 (I)$ \underbar {known} signal, \medskip \itemitem{(2)}: the noise process $\{N(t),~0\leq t \leq T\}$ has the form $N(t): = V \phi(t)$ \medskip\noindent where V is a \underbar {standard} R-valued Gaussian RV with \underbar {zero} mean and \underbar {unit} variance, and $\phi \epsilon L^2(I)$ \underbar {known} with $\mid \phi \mid = 1.$ \medskip \itemitem{(i)}: Compute the \underbar {correlation} kernel of the noise processes $\{N(t),~0\leq t \leq T\}$ ~~(5 pts.) and find \underbar {all} the eigenpairs with non-zero eigenvalue for the corresponding integral operator ~~(5 pts.); \medskip \itemitem{(ii)}Find the Karhunen - Loeve expansion of the noise process $\{N(t), 0 \leq t \leq T\}$ ~~(5 pts.); \medskip \itemitem{(iii)}Describe \underbar {all} the likelihood ratio tests for testing $H_1$ against $H_0$ (10 pts.); \medskip \itemitem{(iv)}Find the corresponding test statistic (5 pts.) \medskip \itemitem{(v)} If the hypotheses are equally likely, find the optimal Bayesian receiver under the MPE criterion (5 pts.) and compute its performance ~~(5 pts.) \vfill \eject \hfill Spring 84 \smallskip \hfill Test 2 \smallskip \hfill AM \bigskip\noindent 1.~ Consider the situation where the observations are modelled by an $R^K$-valued RV Z, defined by $$Z = aS + N$$ \noindent where \bigskip \itemitem{(i)} The scaler a is an \underbar {unknown} constant; \medskip \itemitem{(ii)} The $R^K$-valued RV's S and N are \underbar {independent Gaussian} RV's with \underbar {zero} mean vectors and \underbar {identity} covariance matrices. \medskip\noindent It is desired to estimate $a^2$ (\underbar {not} a) on the basis of the observation Z. \medskip \itemitem{(i)} Compute the maximum likelihood estimate of $a^2$ on the basis of Z ~~(5 pts.); \medskip \itemitem{(ii)} Discuss the efficiency of this ML estimate ~~(5 pts.); \medskip \itemitem{(iv)} Compute the (conditional) variance of the ML estimate (5 pts.) \medskip \itemitem{(v)} Find a bound on the (conditional) variance of \underbar {any} unbiased estimator for $a^2$ on the basis of Z ~~(5 pts.) \medskip \itemitem{(vi)} Is the ML estimator given in (i) \underbar {strongly consistent}? Explain ~~ (10 pts.); \medskip \itemitem{(vii)} Is the ML estimator given in (i) a desirable estimator for $a^2$? Explain ~~~(5 pts.). \vfill \eject \hfill Spring 84 \smallskip \hfill Test 2 \smallskip \bigskip\noindent \bigskip\noindent 2. ~ Consider the situation where a scalar random parameter $\theta$ is to be estimated on the basis of the scalar observation $Z_1$ and $Z_2$, with $$ Z_1 = \theta + V_1, Z_2 = \theta + V_2$$ \noindent where it is assumed that the RV's $\theta$, $V_1$ and $V_2$ are \underbar {mutually uncorrelated} and that they all have \underbar {zero} means. Moreover, to fix the notation, pose $$var(\theta) = \sigma^2,~var(V_1) = var(V_2) = \epsilon^2 >0$$ \itemitem{(i)} Find the linear MMSE estimator of $\theta$ on the basis of $Z_1$ ~~(15 pts.) and compute the corresponding MSE ~~(5 pts.) \medskip \itemitem{(ii)}\underbar {Use} the estimator obtained in (i) to generate the linear MMSE of $\theta$ on the basis of \underbar {both} $Z_1$ \underbar {and }$Z_2$ ~~(15 pts.) and compute the corresponding MSE ~~(5 pts.) \vfill \eject \hfill 84 \smallskip \hfill Test 3 \bigskip\noindent 1. ~Consider two R-valued zero-mean \underbar {wide-sense stationary} stochastic processes \break $\{X(t), t\epsilon R\}$ and $\{N(t), t\epsilon R\}$ with power spectral density functions $$\phi_x(\nu) = {8\over (1+\nu^{2})^{2}}, \phi_n(\nu) = {1\over 1+\nu^{2}}, \nu\epsilon R.$$ \noindent Assume the two processes to be \underbar {correlated} and define a third process $\{Z(t), t\epsilon R\}$ by $$Z(t): = X(t) + N(t)~~,~~t\epsilon R.$$ \itemitem{(1)} Find a filter which is \underbar {casual} and produces a white noise output when the process $\{Z(t), ~t\epsilon R\}$ is passed through it. \medskip \itemitem{(2)}Find the transfer function $H_\infty (s)$ for the \underbar {non-casual linear} estimator \medskip \itemitem{}The problem is one of estimating X(t) on the basis of the process $\{Z(r), r\epsilon R\}$ in the minimum mean-square error sense. \medskip \itemitem{(3)}Find the transfer function $H_0(s)$ and the corresponding impulse response function $h_0(t)$ for;r the \underbar {casual linear} estimator, the so-called \underbar {Wiener filter}. Compute the corresponding mean-square error~~(10 pts.)> \bigskip\noindent 2. ~A scalar state parameter $\theta$ is modeled as a second order RV with $E(\theta) = \mu$ and $cov(\theta) = \Sigma$. The quantity $\theta$ is observed in additive noise, i.e., $$Z(t) = \theta + V(t), ~~ t = 0, 1,...$$ \noindent where the noise sequence $\{V(t)\}_0^\infty$ is a white noise sequence (with E(v(t)) = 0 and $E(V(t)V(s) = \delta(t-s)\sigma^2$ for all s and t in N) uncorrelated with $\theta$. \medskip \itemitem{(i)}Find the \underbar {recursive} algorithm to generate $\{\hat \sigma (t)\}_0^\infty$, where $\hat \theta (t)$ is the linear MMSE estimate of $\theta$ on the basis of $Z^t = (Z(0),\ldots, Z(t)$ \medskip \itemitem{(ii)} Derive the difference equation that generates the error variances $\{P(t)\}_0^\infty$, where $$P(t) : = var [\theta - \hat \theta (t)], t = 0,1,....$$ \medskip \itemitem{(iii)}Investigate the limiting behavior of P(t) as $t\uparrow^\infty$. \vfill \eject Here starts year 1985 \hfill Spring 85 \smallskip \hfill Finals \bigskip\noindent Consider the hypothesis testing problem where a r.v. Y has a density function given by $$H_1 : Y ~\sim~ f_1(y) = \left \{\eqalign{ {2 - \mid y\mid \over 4}&, \quad \mid y\mid \leq 2\cr 0&, \quad \mid y\mid > 2\cr}\right.$$ \noindent or by $$H_0 : Y ~\sim~ f_0(y) = \left \{\eqalign{1 - \mid y \mid &,\quad \mid y \mid \leq 1\cr 0&, \quad \mid y \mid > 1\cr}\right.$$ \noindent Assume that deciding $H_0$ when $H_1$ is true costs twice as much as deciding $H_1$ when $H_0$ is true and that correct decisions costs nothing. Further, assume that a priori $P\bigl\{H_1\bigr\} = \cal E$ where $\cal E \epsilon (0,1)$. Find a Baye's test as a function of $\cal E$. \vfill \eject \hfill Spring 85 \smallskip \hfill Exam I \bigskip \noindent Let $\{N_0(t),~t\geq 0 \}$ be a white Gaussian noise process with power spectral density No/2 and $\{N_1(t),~t\geq 0\}$ a white Gaussian noise process with power spectral density $N_1/2$ where $N_0,N_1 > 0$ and $N_0 \not = N_1$. Consider the hypothesis testing problem where $\{Y(t)~,~t\epsilon [0,T]\}$ is observed and the hypotheses are \bigskip \settabs 10\columns \+&$H_0$ :&Y(t) = $N_0(t)$&&,&$0\leq t \leq T$\cr \medskip \+&$H_1$ :&Y(t) = $N_1$&&,&$0\leq t \leq T$.\cr \medskip\noindent Assume that a priori the two hypotheses are equally likely and construct a likelihood ratio test based on the minimum probability of error criterion. \vfill \eject \hfill Spring 85 \smallskip \hfill Exam II \bigskip\noindent Let $\bigl\{X_i\bigr\}_1^\infty$ be a sequence of i.i.d.~r.v's distributed under the hypotheses as: \bigskip \settabs 10\columns \+&$H_1$ : &P$\bigl\{X_i = 1\bigr\}$ = 4/5&&,&&P$\bigl\{X_i = 1\}$ = 1/5\cr \smallskip \+&$H_1$ :&P$\bigl\{X_i = 0\bigl\}$ = 2/5&&,&&P$\bigl\{X_i = 1 \bigr\}$ = 2/5, P$\bigl\{X_i = 2\bigr\}$ = 1/5\cr \medskip\noindent A sequential LRT is to be constructed to test $H_0$ v/s $H_1$. Let $P_f = \alpha = 2^{-j}$~, $P_M = \beta 2^{-k}$ where $j,k > 1$ are integers. Let N be the number of samples at which the test terminates. \bigskip \itemitem{(i)}Determine $\wedge (X_i)$~, i = 1,2,... that you would use. \medskip \itemitem{(ii)}Calculate $E[N/H_1]$ and $E[N/H_0$] assuming Wald's approximation hold. \vfill \eject \hfill Spring 85 \smallskip \hfill Final \bigskip\noindent \underbar {Problem 1} \medskip\noindent Consider the following binary hypothesis testing problem. Let X be a discrete random variable taking only the values 0, 1, 2 .... Our observations are as follows: \medskip \itemitem{-} If X = 0, then X constitutes the only observations; \medskip \itemitem{-} If X = $n \geq$ 1, then we observe X \underbar {and} $Y^n = ( Y_1,\ldots, Y_n )$, where $Y_i$ is a R-valued random variable, i = 1,...n. \medskip\noindent We assume that we are given two hypotheses $H_0$ and $H_1$ together with the priors $P(H_0)$ and $P(H_1)$, with $0 < P(H_0),~ P(H_1( < 1.$ We are also given a Bayesian cost structure defined by the non-negative constants $C_{10} > C_{00}$, and $C_{01} > C_{11}$. \medskip \itemitem{}We wish to implement a decision rule of the following form: \bigskip \itemitem{-}If X = 0, always decide $H_k$ (you will determine $k\epsilon \{0,1\}$ later). \itemitem{-}If X = $n\geq 1$, decide $H_0$ if $Y^n\epsilon B_0^n \underline {C} R^n$ and decide $H_1$ if $Y^n \epsilon B_1^n \underline {C}R^n$. Here, $B_0^n$ and $B_1^n$ are \underbar {disjoint} (Borel) subjects of $R^n$ such that $B_0^n UB_1^n = R^n$ (you will determine $B_0^n$ later, noting that $B_1^n = R^n B_0^n$). \medskip\noindent Let $C_n$ be the cost of a decision when X = n. If C is the cost of a decision for any value of X, we can write $$C = \sum_{n=0}^\infty~ C_n~1(\{X=n\}).$$ \medskip\noindent It follows that the average cost, J, of a decision is given by $$J \buildrel \Delta \over = E[C_n 1 (\{X=n\})].$$ \noindent Let $J_n = E[C_n 1(\{X=n\})].$ We can simplify this to $$J_n =~\left\{\eqalign{ \sum_{i=0}^1 \sum_{j=0}^1~C_{ij}&P(\{X = 0, H_j\})\delta_{ik},\quad\quad (k\epsilon \{0,1\})\cr \sum_{i=0}^1 \sum_{j=0}^1~C_{ij}&P(\{X=n, ~Y^n \epsilon B_i^n,~H_j\}).\cr} \right.$$ \noindent In this problem, you will compute the likelihood ratio functions/tests which correspond to minimizing $J_n, ~n\geq 0.$ \medskip Specifically, for j = 0,1, and any event E, let $$P_j(E) \buildrel \Delta \over = P(E/H_j) = {P(E,H_j) \over P(H_j)}.$$ \noindent Do parts (a), (b), and (c) below: \medskip \itemitem{(a)} Show that, in order to minimize $J_0$, we should choose $H_0$ according to: $$(LRT)_0 : {P_1 (X=0)\over P_0 (X_0)}\xx_{H_0}^{H_1} {P(H_0)\over P(H_1)}~ {C_{10} - C_{00}\over (C_{01} - C_{11})}$$ \medskip \itemitem{(b)} Show that $$J_n = C_{10} P_0(X=n)P(H_0) + C_{11}P_1(X=n)P(H_1) + \tilde J_n,$$ \noindent where $$\tilde J_n = \bigl (C_{00} - C_{10}\bigr) P(H_0) P_0 \bigl(X=n, Y^n \epsilon B_0^n\bigr) + \bigl(C_{01} - C_{11}\bigr) P(H_1)P_1 \bigl(X=n,~Y^n \epsilon B_0^n\bigr).$$ \itemitem{(c)}Let $f_{Y^{n}/j} (y),~y\epsilon R^n$, be the conditional density of $Y^n$ given $H_j$. Use the fact that $$P_j\bigl(X=n,~Y^n\bigr)f_{Y^{n}/j} (y)dy,~~j = 0, 1,$$ \itemitem{}to show that $J_n$ will be minimized if we set $$B^n~=~\biggl\{y:{P_1(X = n/Y^n = y)fY_n/1(y)\over P_0(X=n/Y^n = y) f_{Y_n}/0(y)}~<~{P(H_0)\over P(H_1)}~\biggl({C_{10} - C_{00}\over C_{01} - C_{11}}\biggr)\biggr\}$$ \itemitem{}and conclude that $(LRT)_n$ is given by: $$(LRT)_n: {P_1 (X=n/Y^n = y)f_{Yn/1}(y)\over P_0(X = n/Y^n = y)f_{Y^n/0} (y)} \xx_{H_0}^{H_1}~{P(H_0)\over P(H_1)}.~ \biggl({C_{10} - C_{00}\over C_{01} - C_{11}}\biggr)$$ \vfill \eject \hfill Spring 85 \smallskip \hfill Final \bigskip\noindent \underbar {Problem 2} \medskip Consider two signals $$s_1(t) = Re[\sqrt{2E} \tilde s_1(t) exp~(j2\pi f_0t)],~0\leq t\leq T$$ $$s_2(t) = Re[\sqrt {2E} \tilde s_2(t) exp~(j2\pi f_0 t)],~0\leq t\leq T.$$ \noindent The signals are used as successive transmitted pulses in a radar system. For a reflector at delay $\tau$, and doppler f, the received signal is $$r(t) = Re[\sqrt {2E} \{\tilde b_1 \tilde s_1 (t-\tau) + \tilde b_2 \tilde s_2 (t-\tau-T_p)\}exp~\{exp\{j2\pi(f+f_0)t\}] + w(t), -\infty > T$. \medskip\noindent \underbar {Case A:} ~The same radar pulse is used for each target illumination, i.e., $ = \tilde s_1(t) =\tilde s_2(t) = \tilde s(t).$ \medskip \itemitem{(i)}Specify the receiver processing to estimate the delay and doppler of the reflector form r(t), $ -\infty < t <\infty$. You should use the bandpass matched filer, square-law envelope detector (BPMF/SLED) implementation. Indicate how you would modify your receiver if $N(>2)$ illuminations rather than 2 were used. \medskip \itemitem{(ii)}Describe the output of your receiver in the absence of noise. Can you relate this to the ambiguity function of $\tilde s(t)$? \medskip \noindent \underbar {Case B:} ~Different radar pulses are used for each illumination, i.e., $\tilde s_1(t) \not = s_2(t).$ \medskip \itemitem{(iii)} Specify the receiver processing in the form of a air of BPMF /SLED's. Indicate how the returns from each pulse are processed and combined. Indicate how you would modify your receiver if $N(> 1)$ pulse pairs were used rather than 1 pair; assume each pair to be the same as the single pair. \medskip \itemitem{(iv)} Describe the output of your receiver in the absence of noise. Relate this to the ambiguity function of $\tilde s_1(t)$ and $\tilde s_2(t)$. \medskip \itemitem{(v)} Consider the special case of the situations in parts (iii) and (iv), in which an up-chirp/down-chirp pair of signals is used, i.e., $$\tilde s_1(t) = (\pi T^2)^{-1/4} ~exp\bigl[- \bigl({1\over 2T^2}~- jb\bigr)t^2\bigr],~-\infty < t < \infty$$ $$\tilde s_2(t) = (\pi T^2)^{-1/4}~exp \bigl[ -\bigl( {1\over 2T^2}~+~ jb\bigl)t^2\bigl],~-\infty < t< \infty.$$ \noindent Describe the output of your receiver in the absence of noise. Discuss the implications for delay and doppler estimation accuracy an up-chirp/down-chirp pair versus that for an up- or down-chirp alone. \vfill \eject \hfill Spring 85 \smallskip \hfill Final \bigskip\noindent \underbar {Problem 3} \medskip (i)~(a) Suppose particles are arriving according to a Poisson process, $\{X_t,t \geq 0; X_0 = 0\}$ with parameter $\theta$. Each arriving particle has a probability $\lambda$ of passing through a barrier ($\lambda$ known). Assuming that $N_0$ particles have crossed this barrier, and that the times of crossing are observed (with a crossing occurring without loss of time), find the maximum likelihood (ML) estimate of $\theta$. \medskip \itemitem{(b)} Suppose, instead that every h$^{th}$ arriving event passes through the barrier successfully (h $\geq$ 1, known); assuming that $N_0$ particles have crossed the barrier, and that the times of crossing are observed (with a crossing occurring without loss of time), find the ML estimate of $\theta$. \medskip \itemitem{(c)} In cases (a) and (b) above, study the estimates for \underbar {bias} and \underbar {efficiency}. \medskip (ii) (a) Let $\{X(t): t\geq 0\}$ be a stationary Gauss-Markov process with covariance function R(s, t) = e$^{\beta \mid t-s\mid} (\beta >$, known). If the process is observed over [0, T], obtain an LRT for testing $H_0:m \buildrel \Delta \over =$ E [x(t)] = 0 versus $H_1:m \buildrel \Delta \over = E[X(t)] > 0$. \medskip \itemitem{(b)} Determine whether or not the test is U.M.P. \medskip \itemitem{(c)} Under $H_1$, obtain the ML estimate of m when the process is observed on [0, T]. Is the estimate efficient for $T < \infty$? Is it efficient as $T \uparrow \infty$? \vfill \eject \hfill Spring 85 \bigskip\noindent \underbar {Problem 4} \medskip \itemitem{(a)} Let $\{X_t, - \infty < t < \infty \}$ be a R-valued process with zero mean and with spectral density $$\phi_x(\nu) = {1 \over 4 + (2 \pi \nu)^4},- \infty < \nu < \infty,$$ \noindent and let $\{N_t, - \infty < t < \infty \}$ be a zero-mean R-valued process with $$\phi_N(\nu) = {1\over 1 + (2 \pi \nu)^2}, - \infty < \nu < \infty .$$ \noindent Assume that $E[X_t N_s] = 0 0 \forall $ s, t, and define $$Y_t = X_t + N_t, ~- \infty < t < \infty .$$ \itemitem{(i)} Find the linear least-square estimator, $\tilde X_t$, of $X_t$ given $\{Y_s, s\leq t\}$ (i.e., $\tilde X_t$ is the output of a linear filter driven by the observations). \medskip \itemitem{(ii)}Find the linear least-square estimator of $X_{t+\alpha}, \alpha > 0$, given $\{Y_s, s \leq t\}$. \medskip \itemitem{(b)} Consider the hypotheses $$\left.\eqalign{H_1 : r(t)&= A~sin(\omega_c t+\theta) + n(t)\cr H_0 : r(t)&= n(t)\cr}\right\} 0\leq t\leq T$$ \noindent where $\omega_c$ is a constant, $\theta$ is a r.v. uniformly distributed on $(0, 2\pi)$, and n(t) is a WGN with spectral density ${N_0 \over 2}$. For the sake of convenience, you may assume that $\omega_c T = k \pi$, k an integer. \medskip \noindent Let A be a discrete r.v. with P (A=0) = 1-p, and p(A+A$_0$) = p, p$\epsilon$ [0,1]. Determine the LRT using the Neyman-Pearson criterion. Can $q^2 = [\int_0^T~r(t)sin~\omega_c tdt[^2 + [\int_0^T~r(t)\cos~\omega_c$ $tdt]^2$ be used as a test-statistic? \vfill \eject \hfill Spring 85 \bigskip\noindent \underbar {Problem 5} \medskip\noindent Consider the on-dimensional system $$X_{k+1} = X_k + bW_k~,~ k = 0, 1,\ldots$$ \noindent where the observation process is $$Z_k = X_k + W_k~.$$ \noindent Here, $\{W_k\}$ is a sequence of zero mean Gaussian r.v.'s with $E\{W_k W_\ell\} = Q_k \delta_{k\ell}$. Let $Q_0 \sim N(0,P_0)$ and assume that $cov~[X_0,W_k] = 0 \forall k;$ b is a known constant. \medskip\noindent Define $Z^k \buildrel \Delta \over = (Z_0, \ldots, Z_k).$ \medskip \itemitem{(a)} Find a recursive scheme for generating the linear least-squares estimate $\tilde E[X_k/Z^k]]$. \medskip \itemitem{(b)} Find a recursive scheme for the linear least squares estimate $\tilde E[X_{k=N}/Z^k]$, where N is a given positive integer. \vfill \eject Here starts year 1987 \hfill Spring 87 \bigskip\noindent 1. ~ Let $\{\phi_1, \phi_2,\ldots ,\phi_n\}$ be continuous mappings [0,1] $\rightarrow \R$ and let $\{U_1, U_2,\ldots ,U_n\}$ be zero-mean $\R$-valued RV's with finite second moments. Assume the mappings\break $\{\phi_1,\phi_2,\ldots ,\phi_n\}$ to be orthonormal and the RV's $\{U_1,U_2,\ldots , U_n\}$ to be uncorrelated with $$E[U_kU_{\ell}] = \sigma_k^2\delta_{k\ell}~~k,\ell = 1,2,...,n$$ \noindent where $0 < \sigma_n^2 < \sigma_{n-1}^2 < \ldots < \sigma_1^2 < \infty.$ \itemitem{(1.a)}Compute the correlation kernel $R(\cdot , \cdot)$ of the $\R$-valued process $\{V(t), 0 \leq t \leq 1\}$ defined by $$V(t) = \sum_{k=1}^n U_{k \phi k} (t), ~~ 0\leq t \leq 1$$ \noindent and decide whether the process is mean-square continuous. (5 pts.) \medskip \itemitem{(1.b)}For every $\phi$ in $L^2([0,1])$, evaluate $R\phi$ where $R$ denotes the integral operator induced by the correlation kernel computed in (1.a) (5 pts.). \medskip \itemitem{(1.c)}Find the positive eigenvalues of $R$, their multiplicity and their corresponding eigenvectors (suitably orthonormalized). (5 pts.). \medskip \itemitem{(1.d)}Find the Karhunen-Loeve expansion of the process $\{V(t), 0\leq t \leq 1\}$ (5 pts.). \vfill \eject \hfill Spring 87 \bigskip\noindent 2. ~(Continued) From now on, assume the RV's $\{U_1, U_2,\ldots ,U_n\}$ to be jointly Gaussian and specialize the family $\{\phi_1, \phi_2,\ldots , \phi_n\}$ to $$\phi_k(t) = \sqrt 2~cos(k\pi t), ~~k = 1, 2, \ldots , n$$ Let $s$ be another continuous mapping [0, 1]$\rightarrow \R$, and consider the following hypothesis testing problem. \bigskip \settabs 8\columns \+&$H_1$:Z(t)=&s(t)+V(t),~~& $0 \leq t \leq 1$\cr \medskip \+&$H_0$:Z(t)=&V(t),~~& $0\leq t \leq 1$\cr \medskip \noindent where the hypothesis RV H is $\{0,1\}$-valued and independent of the RV's $\{U_1, U_2,\ldots ,U_n\}$. Let 0 $< p < 1$ be its known prior, i.e., $$P[H = 1] = p = 1 - P[H = 0].$$ \itemitem{(2.a)}Is the process $\{V(t), 0 \leq t\leq 1 \}$ Gaussian? Explain your answer (5 pts.). \medskip \itemitem{(2.b)}Solve the Bayesian problem under the probability of error criterion when the signal $s$ is given by $$s(t) = cos~{\left (\pi t\over 2\right)},~~0 \leq t \leq 1.$$ \noindent (10 pts.). \medskip \itemitem{(2.c)}Solve the Bayesian problem under the probability of error criterion when the signal $s$ is given by $$s(t) = \sqrt 3 \left[1 - 2 sin^2 {\left(\phi t \over 2\right)}\right],~~ 0\leq t \leq 1.$$ \noindent (10 pts.). \vfill \eject \hfill 87 \bigskip\noindent 3. ~Consider the hypotheses $\{H_\theta, \theta \epsilon \R^n\}$ on an $\R^n$-valued RV Z, with $$H_\theta: ~Z \sim N(\theta, R)$$ \noindent where $R > 0$. \medskip \itemitem{3.a)}Compute the likelihood ratio $L(\cdot; \theta)$ for testing the simple hypothesis $H_0$ against its simple alternative $H_\theta$ for some $\theta \not = 0$ in $\R^n$. Determine the form of the corresponding Neyman-Person test of size $\alpha$ and find an expression for its power $\beta (\alpha), 0 \leq \alpha \leq 1$ (5 pts.). \medskip \itemitem{}Consider now the composite hypothesis testing problem of deciding the simple hypothesis $H_0 : Z \sim N(0, R)$ against its composite alternative $H_1: Z \sim N(\theta, R), \theta \epsilon \Theta_1$, where $\Theta_1$ is a subset of $\R^n$ that does not contain 0. \medskip \itemitem{(3.b)}Assume $\theta$ to be a RV with $\Theta_1 = \R^n - \{0\}$. Under the assumption that $\theta \sim N(0,P)$ with $P > 0,$ compute the corresponding likelihood ratio and determine the likelihood ratio tests that are based on it (5 pts.). $$\int~ exp \left[\nu \prime \theta - {1\over 2} ~\theta \prime S^{-1} \theta \right ] d\theta = sqrt {(2 \pi) det S} ~exp \left[{1\over 2} \nu \prime S^{-1}\nu \right]$$ \noindent for all $\nu$ in $\R^n$. \itemitem{(3.c)}Assume $\theta$ to be an unknown constant. Decide whether a UMP test of size $\alpha$ exists when \medskip \itemitem{(1)}: $n$ = 1 and $\Theta_1 ~ = ~\R ~ - ~\{0\}$ (5 pts.) \smallskip \itemitem{(2)}: $n$ = 1 and $\Theta_1 ~ = ~(0, + \infty)$ (5 pts.) \smallskip \itemitem{(3)}: $n >$ and $\Theta_1 ~ = ~\R^n ~- ~\{0\}$ (5 pts.) \medskip \itemitem{(3.d)}(Continued) Find the generalized likelihood ratio $L_g(\cdot)$ when $\Theta_1 = \R^n - \{0\}$ and construct the likelihood ratio tests that are based on it. (5 pts.). \vfill \eject \hfill Spring 87 \bigskip\noindent 4. ~Consider the following binary hypothesis testing problem, where under each hypothesis, the $n$ observations $\{Z_1,\ldots ,Z_n\}$ are $\{0,1\}$-valued and mutually independent, with $$P[Z_i = 0\mid H = h] = \alpha_h, ~~i = 1, \ldots , n$$ \noindent for all h = 0, 1. Here $0 < \alpha _0 < alpha _1$ for sake of definiteness. \medskip \itemitem{(4.a)} Find the corresponding likelihood ratio and show that all the relevant information is contained in the RV $L_n$ defined by $$L_n ~: = ~\sum_{i = 1}^n~Z_i.$$ \itemitem{}Express the likelihood ratio tests in terms of it (5 pts.). \medskip \itemitem{(4.b)} Find an example of a test which is not a likelihood ratio test (5 pts.). \medskip \itemitem{(4.c)} Assume the hypothesis RV H to have a known prior distribution, say $$P[H = 1] = p = 1 - P[H = 0]$$ \noindent for some $0 < p < 1$. Find the test that minimizes the probability of error criterion, and find an expression for the value of the minimum probability of error (10 pts.). \medskip \itemitem{(4.d)} Solve the minimax version of the problem posed in (4.c) (5 pts.). \vfill \eject \noindent 1. ~Consider the following simple estimation problem where the observation RV $Z$ takes value in a countable subset $\{ a_1,a_2,\ldots \}$ of the real line $\R$ and the parameter $\theta$ to be estimated is binary-valued, with $\theta$ either $\theta$ = 0 or $\theta$ = 1. For $\theta$ = 0, 1, pose $$P_\theta [Z = a] = p_theta(a), ~~a = a_1, a_2,\ldots$$ \noindent and assume for sake of simplicity that $$p_\theta(a) > 0, ~a = a_1, a_2, \ldots$$ \bigskip \itemitem{(1.a)}Show that the likelihood ratio L($\cdot$) defined by $$L(a) = {p_1(a)\over p_0(a)}, ~~a = a_1, a_2, \ldots$$ \medskip\noindent is a sufficient statistic for estimating $\theta$ on the basis of $Z$. (10 pts.) \bigskip \itemitem{(1.b)} What are the implications of this fact for the binary hypothesis testing problem naturally associated with the problem formulation given here? Does it explain some of the results obtained earlier in the course? (5 pts.). \vfill \eject \noindent 2. ~Consider the following situation where the $\R$-valued RV $\theta$ (with finite second-order moment) is observed in additive noise, i.e., the observation RV's $\{Z(t), t = 0, 1\ldots\}$ are given by $$Z(t) = \theta + V(t), ~~t=0, 1\ldots$$ \noindent with $\{V(t), t = 0, 1\ldots\}$ denoting the noise sequence. Throughout it is assumed that \bigskip \itemitem{(1)} The noise sequence $\{V(t), t = 0, 1\ldots\}$ is an $\R$-valued white noise sequence with $$E[V(t)] = 0~\rm {and}~~E[V(t)V(s)] = \sigma^2 \delta(t,s)$$ \noindent for all $s, t = 0, 1, \ldots$ \bigskip \itemitem{(2)} The parameter $\theta$ and the noise sequence $\{V(t), t = 0, 1\ldots \}$ are uncorrelated. \bigskip \itemitem{}For all $t = 0, 1, \ldots$, the estimates $\theta_1 (t)$ and $\theta_2 (t)$ of the parameter $\theta$ on the basis of $Z^t :=(Z(0), \ldots , Z(t)$ are defined by $$\theta_1(t): = \rm {LMMSE~estimate~of}~ \theta \rm {~on~the~basis~of}~Z^t$$ \noindent and $$\theta_2 (t): = {1\over t + 1}~\sum_{r=0}^t~Z(r),$$ \noindent and for convenience denote the corresponding estimation errors by $\tilde \theta_i(t) = \theta - \theta_i(t), ~~i = 1, 2~~~~~~~~~~t = 0, 1...$ \itemitem{(2.a)} By making use only of the Orthogonality Principle, argue that in general $\theta_1(t)\not = \theta_2(t)$ (10 pts.). \bigskip \itemitem{(2.b)} In each case, find recursive equations for the propagation of the estimates $\{\theta_i(t), t = 0, 1, \ldots\}$ (i = 1, 2) (20 pts.). \vfill \eject \itemitem{(2.c)}In each case, find recursive equations for the propagation of the error variances $\{P_i(t), t = 0, 1\ldots \}$ where $$P_i(t): = E[\mid \theta - \theta_i(t)\mid^2], ~~i = 1, 2~~~~~~~~~~ t = 0,1\ldots$$ \noindent (10 pts.). \bigskip \itemitem{(2.d)}In each case, study the limit behavior of $\{P_i(t), t = 0, 1\ldots\}$ as $ t \uparrow \infty$, and determine the consistency of the estimates $\{\theta_i(t), t = 0, 1 \ldots \}$ ~~(10 pts.). \vfill \eject \noindent 3. ~Consider the situation where the observation RV's $\{Z_1, Z_2\ldots, \}$ form an i.i.d. sequence of Gaussian RV's with zero mean and unknown standard deviation $ \sigma > 0$. For notational convenience pose $ r = \sigma^2$. \bigskip \itemitem{(3.a)} Find the Fisher information matrix $M^{(n)}(\sigma)$ for the problem of estimating $\sigma > 0$ on the basis of the observations RV's $\{Z_1,\ldots, Z_n\}$. ~~(5 pts.). \bigskip \itemitem{(3.b)} Compute the Maximum Likelihood (ML) estimator $\sigma_n(\cdot)$ of $\sigma$ on the basis of the observations RV's $\{Z_1, \ldots, Z_n\}$. \bigskip \itemitem{(3.c)} Decide whether this ML estimator is (i) efficient, (ii) unbiased, (iii) asymptotically unbiased. ~~(15 pts.) \bigskip \itemitem{(3.d)} Investigate the consistency properties of this family of estimators $\{\sigma_n\}_1^\infty$~~(5 pts.) \bigskip \itemitem{(3.e)} Show that if $r_n$ denotes the Maximum Likelihood (ML) estimator $r$ on the basis of the observations RV's $\{Z_1, \ldots, \}$, then $$\sigma_n ~=~\sqrt {r_n}$$ \itemitem{} (5 pts.) \vfill \eject \noindent 1. ~An $\R_+$-valued RV $Z$ is said to admit a Rayleigh probability distribution with parameter $\theta > 0$ if $$P[Z \leq z]~=~ F_{\theta}(z) ~=~\int_{-\infty}^z~(x)dx$$ \noindent where $$f_{\theta}(z) ~=~\left\{\vbox{\halign{$\displaystyle{#}$\hfil&\hfil$ \displaystyle {#}$\cr ({z\over \theta}) \exp[-{z^2\over 2\theta}]&\quad if z~ \geq 0;\cr 0&\quad otherwise.\cr}}\right.$$ \noindent Throughout, let $\{Z_1,\ldots , Z_n\}$ be i.i.d. RV's with common Rayleigh distribution $F_\theta(\cdot)$ for some parameter $\theta$. \bigskip \itemitem{(1.a)}Find the Fisher information matrix $M^{(n)}(\theta)$ for the problem of estimating $\theta > 0$ on the basis of the observation RV's $\{Z_1, \ldots ,Z_n\}$. \bigskip \itemitem{(1.b)}Compute the Maximum Likelihood (ML) estimator $\theta_n (\cdot)$ of $\theta$ on the basis of the observations RV's $\{Z_1, \ldots ,Z_n\}$. \bigskip \itemitem{(1.c)} Decide whether this ML estimator is (i) efficient, (ii), (unbiased), (iii) asymptotically efficient, (iv) asymptotically unbiased. \bigskip \itemitem{(1.d)}Investigate the consistency properties of this family of estimators $\{\theta_n\}_1^\infty$. \bigskip\noindent 2. ~An off-on keyeing system is a digital communication system that conveys a digital zero by transmitting a sinusoidal waveform of known amplitude and frequency, and conveys a digital zero by transmitting nothing. The receiver observes the transmitted signal in the presence of additive noise, and the case of fading channels, the attenuation in the received signal can be modelled as a RV. The problem of identifying the nature of the digital signal can then be viewed as a composite hypothesis testing problem. The model adopted here is standard, namely \settabs 8\columns \+&$H_1$:\quad Z(t)= ~&Acos($\omega$ t) +~~&V(t),&0$\leq t\leq T$\cr \+&$H_0$:\quad Z(t)=~~ &&V(t),&0$\leq t \leq T$\cr \medskip\noindent where \bigskip \itemitem{(1)}: The fixed frequency $\omega$ has the form $\omega : = {2n\pi \over T}$ for some known positive integer $n$. \medskip \itemitem{(2)}: Under both hypothesis, the noise process $\{V(t), 0\leq t \leq T\}$ is a zero-mean GWN process with power spectral density ${N_0 \over 2}$. \medskip \itemitem{(3)}: Under $H_1$, the noise process $\{V(t), 0\leq t \leq T\}$ and the amplitude $A$ are statistically independent, and $A$ has a Rayleigh probability distribution with parameter $a$, i.e. $A$ has a probability density function $f_A(\cdot)$ given by $$f_A (A) = \left\{\vbox{\halign{$\displaystyle{#}$\hfil&\hfil$\displaystyle {#}$\cr ({a\over\alpha}) \exp [-{a^2\over 2\alpha}]&\quad if a \geq 0,\cr 0&\quad otherwise\cr}}\right.$$ \noindent for some $\alpha ~> 0.$ \bigskip \itemitem{(2.a)} Compute the likelihood ratio function for testing the simple null hypothesis $H_0$ against its composite alternative $H_1$ in terms of known distribution functions. \medskip \itemitem{(2.b)}Write down the corresponding ratio tests, and identify an appropriate test statistic. \vfill \eject \noindent 3. ~Let $\{\phi_1, \phi_2,\ldots, \phi_n\}$ be continuous mapping $\R$-valued RV's with finite second moments. Assume the mappings $\{\phi_1, phi_2,\ldots ,\phi_n\}$ to be orthonormal and the RV's $\{U_1,U_2,\ldots , U_n\}$ to be jointly Gaussian with $$E[U_k U_\ell] = \sigma_k^2 \delta_{k\ell}~~k, \ell = 1, 2, \ldots, n$$ \noindent where $0 < \sigma_n^2 < \sigma_n{n-1}^2 < ... <\sigma_1^2 < \infty.$ \bigskip Let $s$ be known continuous signal $[0,T]\rightarrow \R$. The signal $s$ is first amplitude-modulated and then sent over some noisy channel additively corrupted by some colored noise process $\{V(t), 0\leq \leq T\}$ given by $$V(t) = \sum_{k-1}^n~U_k \phi_k (t), ~~0, \leq t \leq T.$$ \noindent The resulting observations process $\{ Z(t), 0 \leq t \leq T\}$ is then given by $$Z (t) = A\cdot s(t) + V(t), ~~0 \leq t \leq T$$ where $A \not = 0$ is some unknown yet constant modulation factor. \bigskip \itemitem{(3.a)} Study the general problem of providing a Maximum Likelihood estimate $A_{ML}$ for $A$ on the basis of the observations process $\{Z(t), 0\leq t \leq T\}$. Determine the corresponding Carmer-Rao bound. Also investigate whether this ML estimate $A_{ML}$ is (i) unbiased, (ii) efficient. \medskip \itemitem{}With $T = 1$, specialize the family $\{\phi_1, phi_2, ldots, \phi_n\}$ to $$\phi_k(t) = \sqrt 2~cos (k\pi t), k = 1, 2, ldots, n$$ \itemitem{(3.b)}Apply the results of (3.1) when the signal $s$ is given by $$s(t) = cos({\pi t\over 2}), ~~0, \leq t \leq 1.$$ \itemitem{3.c)} Apply the results of (3.1) when the signal $s$ is given by $$s(t) = \sqrt 3 \bigl[1 - 2 sin^2 \bigl({\pi t\over 2\bigr)} \bigr],~ 0 \leq t \leq 1.$$ \vfill \eject \noindent 5. ~Consider the hypotheses $\{H_\theta, \theta \geq 0\}$ on an $\R_+$-valued RV Z, with $$H_\theta : ~~Z\sim R(\theta)$$ \noindent where $R(\theta)$ denotes the Rayleigh distributed with parameter $\theta > 0$. \bigskip \itemitem{(5.a)} Compute the likelihood ratio for testing the simple (null) hypothesis $H_\theta$ against its simple alternative $H_\theta$ for some $\theta \geq 0$ and $\sigma \geq 0.$ \medskip \itemitem{(5.b)}(Continued) Determine the form of the corresponding Neyman-Pearson test of size $\alpha$ and find an explicit expression for its power function $\beta(\alpha), 0 \leq \alpha \leq 1.$ \medskip \itemitem{(5.c)}(Continued) Determine a closed-form expression for the ROC curve (i.e., $f: [0, 1]\rightarrow [0,1]$ such that $_{PD} = f({_PF}))$. \medskip \itemitem{}Consider now the composite hypothesis testing problem of deciding the simple hypothesis $H_0: \theta = \theta_0$ against its composite alternative $H_1 : \theta \in \ominus_1, where \theta_0 > 0$ and $\ominus_1$ is a subset of ($0, + \infty$). Assume $\theta$ to be an unknown constant. \bigskip \itemitem{(5.d)} Decide whether a UMP test of size $\alpha$ exists when \bigskip \item\item{(1)}$\ominus_1 = (0, \theta_0) \cup (\theta_0, + \infty)$ \medskip \item\item{(2)}$\ominus_1 = (0, \theta_0)$ \medskip \item\item{(3)}$\ominus_1 = (\theta_0, + \infty)$ \bigskip \itemitem{(5.e)}(Continued) Find the generalized likelihood ratio $L_g(\cdot)$ in the cases (1) - (3), and construct the likelihood ratio tests that are based on it. \end