Return to NN.tex CVS log

Up to [Nicadd] / ttbar / p20_taujets_note

Annotation of ttbar/p20_taujets_note/NN.tex, revision 1.2

1.1       uid12904    1: \newpage
1.2     ! uid12904    2: %ttttt
1.1       uid12904    3: \section{\label{sub:NN}Neural Network Analysis}
                      4: 
                      5: \subsection{\label{sub:Variables}Variables for NN training}
                      6: 
                      7: \noindent Following the same procedure as in the previous analysis, we determine the content
                      8: of signal and background in the preselected sample, increase signal/background rate
                      9: and from this, measure the cross-section.
                     10: The procedure adopted in the p17 analysis was feed a set of topological variables into an
                     11: artificial neural network in order to provide the best possible separation between
                     12: signal and background. As before, the criteria for choosing such variables were: power of discrimination
                     13: and $\tau$-uncorrelated variables. The set is presented below:
                     14: 
                     15: \begin{itemize}
                     16: \item \textit{\textbf{$H_{T}$}} - the scalar sum of all jet's $p_{T}$ (here and below including $\tau$ lepton candidates). 
                     17: 
                     18: \item \textit{\textbf{$\not\!\! E_{T}$ significance}} - As being the variable that provides the best signal-background
                     19: separation we decided to optimize it.
                     20: 
                     21: \item \textit{\textbf{Aplanarity}} \cite{p17topo} - the normalized momentum tensor is defined as
                     22: 
                     23: \begin{center}
                     24: \begin{equation}
                     25: {\cal M} = \frac{\sum_{o}p^{o}_{i}p^{o}_{j}}{\sum_{o}|\overrightarrow{p^{o}}|}
                     26: \label{tensor}
                     27: \end{equation}
                     28: \end{center}
                     29: 
                     30: \noindent where $\overrightarrow{p^{0}}$ is  the momentum-vector of a reconstructed object $o$
                     31: and $i$ and $j$ are cartesian coordinates. From the diagonalization of $\cal M$ we find three eigenvalues
                     32: $\lambda_{1}\geq\lambda_{2}\geq\lambda_{3}$ with the constraint $\lambda_{1} + \lambda_{2} + \lambda_{3} = 1$.
                     33: The aplanarity {\cal A} is given by {$\cal A$} = $\frac{3}{2}\lambda_{3}$ and measures the flatness of an event.
                     34: Hence, it is defined in the range $0 \leq {\cal M} \leq 0.5$. Large values of {$\cal A$} correspond to more spherical events,
                     35: like $t\bar{t}$ events for instance, since they are typical of decays of heavy objects. On the other hand,
                     36: both QCD and $W + \mbox{jets}$ events are more planar since jets in these events are primarily due to
                     37: initial state radiation.
                     38: 
                     39: \item \textit{\textbf{Sphericity}} \cite{p17topo} - being defined as {$\cal S$} = $\frac{3}{2}(\lambda_{2} + \lambda_{3})$,
                     40: and having a range $0 \leq {\cal S} \leq 1.0$, sphericity is a measure of the summed $p^{2}_{\perp}$ with 
                     41: respect to the event axis. In this sense a 2-jets event corresponds to {$\cal S$} $\approx 0$ and an isotropic event
                     42: {$\cal S$} $\approx 1$. $t\bar{t}$ events are very isotropic as they are typical of the decays of heavy objects
                     43: and both QCD and $W + \mbox{jets}$ events are less isotropic due to the fact that jets in these events come 
                     44: primarily from initial state radiation.
                     45: 
                     46: \item \textit{\textbf{Top and $W$ mass likelihood}} - a $\chi^{2}$-like variable. 
                     47: $L\equiv\left(\frac{M_{3j}-m_{t}}{\sigma_{t}}\right)^{2}+\left(\frac{M_{2j}-M_{W}}{\sigma_{W}}\right)^{2}$,
                     48: where $m_{t}, M_{W},\sigma_{t},\sigma_{W}$ are top and W masses (172.4
                     49: GeV and 81.02 GeV respectively) and resolution values (19.4 GeV and 8.28
                     50: GeV respectively). $M_{3j}$ and $M_{2j}$ are invariant masses composed
                     51: of the jet combinations. We choose combination that minimizes $L$. 
                     52: 
                     53: \item \textit{\textbf{Centrality}}, defined as $\frac{H_{T}}{H_{E}}$ , where $H_{E}$ is sum
                     54: of energies of the jets. 
                     55: \item \textit{\textbf{$\cos(\theta*)$}} -  The angle between the beam axis and the 
                     56: highest-$p_T$ jet in the rest frame of all the jets in the event.
                     57: \item \textit{\textbf{$\sqrt(s)$}} - The invariant mass of all jets and $\tau$s in the event.
                     58: 
                     59: \end{itemize}
                     60: 
                     61: The chosen variables are in the end a consequence of the method employed  in this
                     62: analysis: use events from the QCD-enriched 
                     63: loose-tight sample to model QCD events in the signal-rich sample, and use
                     64: a b-tag veto sample as an independent control sample to check the validity of such 
                     65: background modeling.
                     66: 
                     67: %\clearpage
                     68: 
                     69: \subsection{\label{sub:NN-variables}Topological NN}
                     70: For training the Neural Network we used the Multilayer Perceptron algorithm, as described in 
                     71: \cite{MLPfit}. As explained before in Section \ref{sub:Results-of-the}, the first 1400000 
                     72: events in the ``loose-tight'' sample were used as background 
                     73: for NN training for taus types 1 and 2, and the first 600000 of the same sample for NN training for type 3 taus.
                     74: This means that different tau types are being treated separately in the topological NN.
                     75: In both cases 1/3 of the Alpgen sample of $t\bar{t} \rightarrow \tau +jets$ was used for NN training and 2/3 of it
                     76: for the measurement.
                     77: When doing the measurement later on (Section \ref{sub:xsect}) we pick the tau with the highest $NN(\tau)$
                     78: in the signal sample as the tau cadidate at same time that taus in the loose-tight sample are picked at
                     79: random since all of them are regarded as fake taus by being below the cut $NN(\tau)$ = 0.7. By doing this 
                     80: we expect to avoid any bias when selecting real taus for the measurement.
                     81: Figures \ref{fig:nnout_type2_training} and \ref{fig:nnout_type3_training} show the 
                     82: effect of each of the chosen the topological event NN input variables on the final output.
                     83: 
                     84: Figures \ref{fig:nnout_type2} and \ref{fig:nnout_type3} show the NN output as a result of the training
                     85: described above. It is evident from both pictures that high values of NN correspond to 
                     86: the signal-enriched region. 
                     87: 
                     88: 
                     89: 
                     90: \begin{figure}[h]
                     91: \includegraphics[scale=0.6]{plots/SetI_NNout_SM_type2_tauQCD.eps}
                     92: \caption{Training of topological Neural Network output for type 1 and 2 $\tau$ channel. 
                     93: Upper left: relative impact of each of the input variables; upper right: topological structure;
                     94: lower right: final signal-background separation of the method; lower left: convergence curves.}
                     95: \label{fig:nnout_type2_training}
                     96: \end{figure}
                     97: 
                     98: \newpage
                     99: 
                    100: 
                    101: \begin{figure}[h]
                    102: \includegraphics[scale=0.6]{plots/SetI_NNout_SM_type3_tauQCD.eps}
                    103: \caption{Training of topological Neural Network output for type 3 $\tau$ channel. 
                    104: Upper left: relative impact of each of the input variables; upper right: topological structure;
                    105: lower right: final signal-background separation of the method; lower left: convergence curves.}
                    106: \label{fig:nnout_type3_training}
                    107: \end{figure}
                    108: 
                    109: \begin{figure}[h]
                    110: \includegraphics[scale=0.5]{CONTROLPLOTS/Std_TypeI_II/nnout.eps}
                    111: \caption{The topological Neural Network output for type 1 and 2 $\tau$ channel}
                    112: \label{fig:nnout_type2}
                    113: \end{figure}
                    114: 
                    115: \newpage
                    116: 
                    117: \begin{figure}[t]
                    118: \includegraphics[scale=0.5]{CONTROLPLOTS/Std_TypeIII/nnout.eps}
                    119: \caption{The topological Neural Network output for type 3 $\tau$ channel}
                    120: \label{fig:nnout_type3}
                    121: \end{figure}
                    122: 
                    123: 
                    124: \subsection{\label{sub:NN-optimization}NN optimization}
                    125: One difference between this present analysis and the previous p17 is that we performed a NN optimization along with a
                    126: $\not\!\! E_{T}$ significance optimization. Previously a cut of $>$ 3.0 was applied to $\not\!\! E_{T}$ significance 
                    127: at the preselection stage and then it was included as one of the variables for NN training. This time as we 
                    128: chose to optimize it, since it is still a good variable to provide signal-background discrimination (Figure \ref{fig:metl_note}).
                    129: It is important to stress out that after the optimization we performed the 
                    130: analysis with the optimized $\not\!\! E_{T}$ significance cut
                    131: applied when doing both $\tau$ and b ID (Section \ref{sub:Results-of-the}), therefore 
                    132: after the preselection where no $\not\!\! E_{T}$ significance cut was applied. 
                    133: We then went back anp reprocessed (preselected) all MC samples with the optimized cut. Both results,
                    134: with $\not\!\! E_{T}$ significance applied during and after preselectio were identical. 
                    135: We then chose to present this analyis with this cut applied at the preselection 
                    136: level in order to have a consistent cut flow throughout the analysis(Section \ref{sub:Preselection}).
                    137: 
                    138: 
                    139: \begin{figure}[h]
                    140: \includegraphics[scale=0.5]{plots/metl_allEW.eps}
                    141: \caption{$\not\!\! E_{T}$ significance distribution for signal and backgrounds.}
                    142: \label{fig:metl_note}
                    143: \end{figure}
                    144: 
                    145: 
                    146: \newpage
                    147: 
                    148: Below we describe how we split this part of the analysis into two parts:
                    149: 
                    150: \begin{enumerate}
                    151: \item {\bf Set optimization:} We applied an arbitrary cut on $\not\!\! E_{T}$ significance of $\geq$ 4.0 and 
                    152: varied the set of varibles going into NN training
                    153: \item {\bf $\not\!\! E_{T}$ significance optimization:} After chosing the best set based on the lowest RMS, 
                    154: we then varied the $\not\!\! E_{T}$ significance cut
                    155: \end{enumerate}
                    156: 
                    157: For this part of the analysis we present the sets of variables that were taken into account to perform the NN traning
                    158: \begin{itemize}
                    159:  \item \textit{\textbf{Set I}} : {$H_{T}$},  aplan (aplanarity), sqrts ($\sqrt{s}$)
                    160:  \item \textit{\textbf{Set II}} : {$H_{T}$},  aplan, cent (centrality)
                    161:  \item \textit{\textbf{Set III}} : {$H_{T}$},  aplan, spher (spherecity)
                    162:  \item \textit{\textbf{Set IV}} : {$H_{T}$}, cent, spher
                    163:  \item \textit{\textbf{Set V}} : aplan, cent, spher
                    164:  \item \textit{\textbf{Set VI}} : {$H_{T}$}, aplan, sqrts, spher
                    165:  \item \textit{\textbf{Set VII}} : {$H_{T}$}, aplan, sqrts, cent
                    166:  \item \textit{\textbf{Set VIII}} : {$H_{T}$}, aplan, sqrts, costhetastar ($cos(\theta^{*})$
                    167:  \item \textit{\textbf{Set IX}} : {$H_{T}$}, aplan, sqrts, cent, spher
                    168:  \item \textit{\textbf{Set X}} : {$H_{T}$}, aplan, sqrts, cent, costhetastar
                    169:  \item \textit{\textbf{Set XI}} : {$H_{T}$}, aplan, sqrts, spher, costhetastar
                    170:  \item \textit{\textbf{Set XII}} : metl, {$H_{T}$}, aplan, sqrts
                    171:  \item \textit{\textbf{Set XIII}} : metl, {$H_{T}$}, aplan, cent
                    172:  \item \textit{\textbf{Set XIV}} : metl, {$H_{T}$}, aplan, spher
                    173:  \item \textit{\textbf{Set XV}} : metl, {$H_{T}$}, cent, spher
                    174:  \item \textit{\textbf{Set XVI}} : metl, {$H_{T}$}, aplan
                    175:  \item \textit{\textbf{Set XVII}} : metl, {$H_{T}$}, sqrts
                    176:  \item \textit{\textbf{Set XVIII}} : metl, aplan, sqrts
                    177:  \item \textit{\textbf{Set XIX}} : metl, {$H_{T}$}, cent
                    178:  \item \textit{\textbf{Set XX}} : metl, {$H_{T}$}, aplan, sqrts, cent
                    179:  \item \textit{\textbf{Set XXI}} : metl, {$H_{T}$}, aplan, cent, spher
                    180:  \item \textit{\textbf{Set XXII}} : metl, {$H_{T}$}, aplan, sqrts, spher
                    181:  \item \textit{\textbf{Set XXIII}} : metl, {$H_{T}$}, aplan, sqrts, costhetastar
                    182:  \item \textit{\textbf{Set XXIV}} : metl, sqrts, cent, spher, costhetastar
                    183:  \item \textit{\textbf{Set XXV}} : metl, {$H_{T}$}, cent, spher, costhetastar
                    184:  \item \textit{\textbf{Set XXVI}} : metl, aplan, cent, spher, costhetastar
                    185:  \item \textit{\textbf{Set XXVII}} : metl, {$H_{T}$}, aplan, cent, costhetastar
                    186:  \item \textit{\textbf{Set XXVIII}} : {$H_{T}$}, aplan, topmassl
                    187:  \item \textit{\textbf{Set XXIX}} : {$H_{T}$}, aplan, sqrts, topmassl
                    188:  \item \textit{\textbf{Set XXX}} : {$H_{T}$}, aplan, sqrts, cent, topmassl
                    189:  \item \textit{\textbf{Set XXXI}} : {$H_{T}$}, aplan, sqrts, costhetastar, topmassl
                    190:  \item \textit{\textbf{Set XXXII}} : metl, {$H_{T}$}, topmassl, aplan, sqrts
                    191:  \item \textit{\textbf{Set XXXIII}} : metl, spher, costhetastar, aplan, cent
                    192: % \item \textit{\textbf{Set XXXIV}} : metl, spher, sqrts, topmassl, ktminp
                    193:  \end{itemize}
                    194: 
                    195: The criteria used for making a decision on which variable should be used follow:
                    196: \begin{itemize}
                    197:  \item No more than 5 variables to keep NN simple and stable. More variables leads to instabilities (different 
                    198:  result after each retraining) and require larger training samples.
                    199: % \item We want to use $metl$ (\met significance) variable, since it's the one providing best discrimination.
                    200:  \item We do not want to use highly correlated variables in same NN.
                    201: % \item We can not use tau-based variables. 
                    202:  \item We want to use variables with high discriminating power.
                    203: \end{itemize}
                    204: 
                    205: In order to make the decision about which of these 11 choices is the optimal we created an ensemble of 
                    206: 20000 pseudo-datasets each containing events randomly (according to a Poisson distribution) picked 
                    207: from QCD, EW and $\ttbar$ templates. Each of these datasets was treated like real data, meaning applying all 
                    208: the cuts and doing the shape fit of event topological NN. QCD templates for fit were made from the same 
                    209: ``loose-tight $\tau$ sample'' from which the QCD component of the ``data'' was drawn. 
                    210: The figure of merit chosen is given by Equation \ref{merit} below:
                    211: 
                    212: \begin{equation}
                    213: f = \displaystyle \frac{(N_{fit} - N_{true})}{N_{true}}
                    214: \label{merit}
                    215: \end{equation}
                    216: 
                    217: 
                    218: \noindent where $N_{fit}$ is the number of $t\bar{t}$ pairs given by 
                    219: the fit and $N_{true}$ is the number of $t\bar{t}$ pairs from the Poisson distribution. 
                    220: In both Set and $\not\!\! E_{T}$ significance optimization, the lowest RMS was used 
                    221: to caracterize which configuration is the best in each case.
                    222: 
                    223: 
                    224: The plots showing results concerning the set optimizations are found in Appendix \ref{app:set_opt} and are summarized 
                    225: in Table \ref{setopt_table} below, where each RMS and mean are shown.
                    226: For NN training is standard to choose the number of hidden nodes as being twice 
                    227: the number the number of variables used for the training. The parenthesis after each set ID show the number of 
                    228: hidden nodes in NN training.
                    229: 
                    230: \begin{table}[htbp]
                    231: \begin{tabular}{|c|r|r|r|} \hline
                    232: Set of variables  & \multicolumn{1}{c|}{RMS}    & \multicolumn{1}{c|}{mean} \\ \hline
                    233: 
                    234: \hline
                    235: 
                    236: 
                    237: Set1(6)      &  \multicolumn{1}{c|}{0.1642}   &  \multicolumn{1}{c|}{0.0265}\\ \hline
                    238: 
                    239: Set2(6)      &  \multicolumn{1}{c|}{0.1840}     &  \multicolumn{1}{c|}{0.0054}\\ \hline
                    240: 
                    241: Set3(6)      &  \multicolumn{1}{c|}{0.1923}   &  \multicolumn{1}{c|}{0.0060}\\ \hline
                    242: 
                    243: Set4(6)      &  \multicolumn{1}{c|}{0.1978}   &  \multicolumn{1}{c|}{0.0175}\\ \hline
                    244: 
                    245: Set5(6)      &  \multicolumn{1}{c|}{0.2385}     &  \multicolumn{1}{c|}{0.0022}\\ \hline
                    246: 
                    247: Set6(8)     &  \multicolumn{1}{c|}{0.1687}   &  \multicolumn{1}{c|}{0.0115}\\ \hline
                    248: 
                    249: Set7(8)     &  \multicolumn{1}{c|}{0.1667}   &  \multicolumn{1}{c|}{0.0134}\\ \hline
                    250: 
                    251: Set8(10)     &  \multicolumn{1}{c|}{0.1668}     &  \multicolumn{1}{c|}{0.0162}\\ \hline
                    252: 
                    253: Set9(10)     &  \multicolumn{1}{c|}{0.1721}     &  \multicolumn{1}{c|}{0.0102}\\ \hline
                    254: 
                    255: Set10(10)     &  \multicolumn{1}{c|}{0.1722}     &  \multicolumn{1}{c|}{0.0210}\\ \hline
                    256: 
                    257: Se11(10)      &  \multicolumn{1}{c|}{0.1716}   &  \multicolumn{1}{c|}{0.0180}\\ \hline
                    258: 
                    259: Set12(8)     &  \multicolumn{1}{c|}{0.1662}     &  \multicolumn{1}{c|}{0.0039}\\ \hline
                    260: 
                    261: Set13(8)     &  \multicolumn{1}{c|}{0.1819}     &  \multicolumn{1}{c|}{0.0018}\\ \hline
                    262: 
                    263: Set14(8)     &  \multicolumn{1}{c|}{0.1879}     &  \multicolumn{1}{c|}{0.0019}\\ \hline
                    264: 
                    265: Set15(8)     &  \multicolumn{1}{c|}{0.1884}     &  \multicolumn{1}{c|}{-0.0004}\\ \hline
                    266: 
                    267: Set16(6)     &  \multicolumn{1}{c|}{0.1912}     &  \multicolumn{1}{c|}{0.0034}\\ \hline
                    268: 
                    269: Set17(6)     &  \multicolumn{1}{c|}{0.1768}     &  \multicolumn{1}{c|}{0.0074}\\ \hline
                    270: 
                    271: Set18(6)     &  \multicolumn{1}{c|}{0.2216}     &  \multicolumn{1}{c|}{-0.0030}\\ \hline
                    272: 
                    273: Set19(6)     &  \multicolumn{1}{c|}{0.1921}     &  \multicolumn{1}{c|}{0.0015}\\ \hline
                    274: 
                    275: Set20(10)     &  \multicolumn{1}{c|}{0.1620}     &  \multicolumn{1}{c|}{0.0262}\\ \hline
                    276: 
                    277: Set21(10)     &  \multicolumn{1}{c|}{0.1753}     &  \multicolumn{1}{c|}{0.0010}\\ \hline
                    278: 
                    279: Set22(10)     &  \multicolumn{1}{c|}{0.1646}     &  \multicolumn{1}{c|}{0.0086}\\ \hline
                    280: 
                    281: Set23(10)     &  \multicolumn{1}{c|}{0.1683}     &  \multicolumn{1}{c|}{0.0132}\\ \hline
                    282: 
                    283: Set24(10)     &  \multicolumn{1}{c|}{0.2053}     &  \multicolumn{1}{c|}{0.0122}\\ \hline
                    284: 
                    285: Set25(10)     &  \multicolumn{1}{c|}{0.1906}     &  \multicolumn{1}{c|}{0.0038}\\ \hline
                    286: 
                    287: Set26(10)     &  \multicolumn{1}{c|}{0.2130}     &  \multicolumn{1}{c|}{0.0028}\\ \hline
                    288: 
                    289: Set27(10)     &  \multicolumn{1}{c|}{0.1859}     &  \multicolumn{1}{c|}{0.0004}\\ \hline
                    290: 
                    291: Set28(6)     &  \multicolumn{1}{c|}{0.1910}     &  \multicolumn{1}{c|}{-0.0022}\\ \hline
                    292: 
                    293: Set29(8)     &  \multicolumn{1}{c|}{0.1587}     &  \multicolumn{1}{c|}{0.0214}\\ \hline
                    294: 
                    295: Set30(10)     &  \multicolumn{1}{c|}{0.1546}     &  \multicolumn{1}{c|}{0.0148}\\ \hline
                    296: 
                    297: Set31(10)     &  \multicolumn{1}{c|}{0.1543}     &  \multicolumn{1}{c|}{0.0203}\\ \hline
                    298: 
                    299: Set32(10)     &  \multicolumn{1}{c|}{0.1468}     &  \multicolumn{1}{c|}{0.0172}\\ \hline
                    300: 
                    301: Set33(10)     &  \multicolumn{1}{c|}{0.2201}     &  \multicolumn{1}{c|}{0.0081}\\ \hline
                    302: 
                    303: %Set34(10)     &  \multicolumn{1}{c|}{0.1955}     &  \multicolumn{1}{c|}{0.0184}\\ \hline
                    304: \end{tabular}
                    305: \caption{Results for set optimization part whit $\not\!\! E_{T}$ significance $>$ 4.0 applied to all sets.
                    306: The number in parenthesis refers to number of hidden nodes in each case.}
                    307: \label{setopt_table} 
                    308: \end{table}
                    309: 
                    310: From Table \ref{setopt_table} we see that Set I has the lowest RMS, thus we chose it
                    311: as the set to be used in $\not\!\! E_{T}$ significance optimization part, whose results are
                    312: shown in Appendix \ref{app:metl_opt} and then summarized in Table \ref{metlopt_table} below
                    313: 
                    314: \begin{table}[htbp]
                    315: \begin{tabular}{|c|r|r|r|} \hline
                    316: Set of variables  & $\not\!\! E_{T}$ significance cut & RMS    & \multicolumn{1}{c|}{mean} \\ \hline
                    317: 
                    318: \hline
                    319: 
                    320: 
                    321: %Set6(10)      &  \multicolumn{1}{c|}{1.0} &  \multicolumn{1}{c|}{0.2611}   \\ \hline
                    322: 
                    323: %Set6(10)      &  \multicolumn{1}{c|}{1.5} &  \multicolumn{1}{c|}{0.2320}   \\ \hline
                    324: 
                    325: %Set6(10)      &  \multicolumn{1}{c|}{2.0} &  \multicolumn{1}{c|}{0.2102}   \\ \hline
                    326: 
                    327: %Set6(10)      &  \multicolumn{1}{c|}{2.5} &  \multicolumn{1}{c|}{0.2021}   \\ \hline
                    328: 
                    329: Set32(10)      &  \multicolumn{1}{c|}{3.0} &  \multicolumn{1}{c|}{0.1507}   &  \multicolumn{1}{c|}{0.0157}\\ \hline
                    330: 
                    331: Set32(10)      &  \multicolumn{1}{c|}{3.5} &  \multicolumn{1}{c|}{0.1559}   &  \multicolumn{1}{c|}{0.0189}\\ \hline
                    332: 
                    333: Set32(10)     &  \multicolumn{1}{c|}{4.0} &  \multicolumn{1}{c|}{0.1468}   &  \multicolumn{1}{c|}{0.0172}\\ \hline
                    334: 
                    335: Set32(10)     &  \multicolumn{1}{c|}{4.5} &  \multicolumn{1}{c|}{0.1511}   &  \multicolumn{1}{c|}{0.0153}\\ \hline
                    336: 
                    337: Set32(10)     &  \multicolumn{1}{c|}{5.0} &  \multicolumn{1}{c|}{0.1552}   &  \multicolumn{1}{c|}{0.0205}\\ \hline
                    338: 
                    339: %Set6(10)     &  \multicolumn{1}{c|}{5.5} &  \multicolumn{1}{c|}{0.4008}   \\ \hline
                    340: \end{tabular}
                    341: \caption{Results for $\not\!\! E_{T}$ significance optimization part when varying the $\not\!\! E_{T}$ significance cut
                    342: The number in parenthesis refers to number of hidden nodes in each case.}
                    343: \label{metlopt_table} 
                    344: \end{table}
                    345: 
                    346: 
                    347: 
                    348: Combined results from Tables \ref{setopt_table} and \ref{metlopt_table} show that the best configuration found
                    349: was Set I  with $\not\!\! E_{T}$ significance $\geq$ 4.0. Therefore, this was the 
                    350: configuration used to perform the cross-section measurement.Figure \ref{fig:METsig_RMS} shows the variation of the RMS as function 
                    351: of the $\not\!\! E_{T}$ significance we applied.
                    352: 
                    353: \begin{figure}[b]
                    354: \includegraphics[scale=0.4]{plots/METsig-RMS.eps}
                    355: \caption{Plot of RMS as a function the $\not\!\! E_{T}$ significance applied}
                    356: \label{fig:METsig_RMS}
                    357: \end{figure}
                    358: 
                    359: 
                    360: \clearpage
                    361: 
                    362: 
                    363: In order to check the validity of our emsemble tests procedure, it is instructive to plot both the 
                    364: distribution of the predicted number of $t\bar{t}$ and what is called ``pull'', defined in Equation  
                    365: \ref{pull} below:
                    366: 
                    367: \begin{equation}
                    368: p = \displaystyle \frac{(N_{fit}-N_{true})}{\sigma_{fit}}
                    369: \label{pull}
                    370: \end{equation}
                    371: 
                    372: \noindent where $\sigma_{fit}$ is the error on the number of $t\bar{t}$ pairs given by the fit.
                    373: 
                    374: Figures \ref{fig:gaus_ttbar} and \ref{fig:pull} show both beforementioned distributions.
                    375: 
                    376: From Figure \ref{fig:gaus_ttbar} we see a good agreement between the number of $t\bar{t}$ pairs
                    377: initially set in the ensemble and the measured value. And Figure \ref{fig:pull} shows a nice gaussian
                    378: curve, that indicates a good behaviour of the fit uncertainties in the ensembles.
                    379: 
                    380: \begin{figure}[t]
                    381: \includegraphics[scale=0.5]{plots/gaus_ttbar.eps}
                    382: \caption{Distribution of the output ``measurement'' for an ensemble with 116.9 $\ttbar$ events.}
                    383: \label{fig:gaus_ttbar}
                    384: \end{figure}
                    385: 
                    386: \begin{figure}[t]
                    387: \includegraphics[scale=0.5]{plots/pull1-40.eps}
                    388: \caption{The ensemble test's pull.}
                    389: \label{fig:pull}
                    390: \end{figure}
                    391: 
                    392: 
                    393: %\newpage
                    394: 
                    395: 
                    396: 
                    397: 
                    398: 
                    399: \clearpage
                    400: