Return to NN.tex CVS log

Up to [Nicadd] / ttbar / p20_taujets_note

Annotation of ttbar/p20_taujets_note/NN.tex, revision 1.1

1.1     ! uid12904    1: \newpage
        !             2: 
        !             3: \section{\label{sub:NN}Neural Network Analysis}
        !             4: 
        !             5: \subsection{\label{sub:Variables}Variables for NN training}
        !             6: 
        !             7: \noindent Following the same procedure as in the previous analysis, we determine the content
        !             8: of signal and background in the preselected sample, increase signal/background rate
        !             9: and from this, measure the cross-section.
        !            10: The procedure adopted in the p17 analysis was feed a set of topological variables into an
        !            11: artificial neural network in order to provide the best possible separation between
        !            12: signal and background. As before, the criteria for choosing such variables were: power of discrimination
        !            13: and $\tau$-uncorrelated variables. The set is presented below:
        !            14: 
        !            15: \begin{itemize}
        !            16: \item \textit{\textbf{$H_{T}$}} - the scalar sum of all jet's $p_{T}$ (here and below including $\tau$ lepton candidates). 
        !            17: 
        !            18: \item \textit{\textbf{$\not\!\! E_{T}$ significance}} - As being the variable that provides the best signal-background
        !            19: separation we decided to optimize it.
        !            20: 
        !            21: \item \textit{\textbf{Aplanarity}} \cite{p17topo} - the normalized momentum tensor is defined as
        !            22: 
        !            23: \begin{center}
        !            24: \begin{equation}
        !            25: {\cal M} = \frac{\sum_{o}p^{o}_{i}p^{o}_{j}}{\sum_{o}|\overrightarrow{p^{o}}|}
        !            26: \label{tensor}
        !            27: \end{equation}
        !            28: \end{center}
        !            29: 
        !            30: \noindent where $\overrightarrow{p^{0}}$ is  the momentum-vector of a reconstructed object $o$
        !            31: and $i$ and $j$ are cartesian coordinates. From the diagonalization of $\cal M$ we find three eigenvalues
        !            32: $\lambda_{1}\geq\lambda_{2}\geq\lambda_{3}$ with the constraint $\lambda_{1} + \lambda_{2} + \lambda_{3} = 1$.
        !            33: The aplanarity {\cal A} is given by {$\cal A$} = $\frac{3}{2}\lambda_{3}$ and measures the flatness of an event.
        !            34: Hence, it is defined in the range $0 \leq {\cal M} \leq 0.5$. Large values of {$\cal A$} correspond to more spherical events,
        !            35: like $t\bar{t}$ events for instance, since they are typical of decays of heavy objects. On the other hand,
        !            36: both QCD and $W + \mbox{jets}$ events are more planar since jets in these events are primarily due to
        !            37: initial state radiation.
        !            38: 
        !            39: \item \textit{\textbf{Sphericity}} \cite{p17topo} - being defined as {$\cal S$} = $\frac{3}{2}(\lambda_{2} + \lambda_{3})$,
        !            40: and having a range $0 \leq {\cal S} \leq 1.0$, sphericity is a measure of the summed $p^{2}_{\perp}$ with 
        !            41: respect to the event axis. In this sense a 2-jets event corresponds to {$\cal S$} $\approx 0$ and an isotropic event
        !            42: {$\cal S$} $\approx 1$. $t\bar{t}$ events are very isotropic as they are typical of the decays of heavy objects
        !            43: and both QCD and $W + \mbox{jets}$ events are less isotropic due to the fact that jets in these events come 
        !            44: primarily from initial state radiation.
        !            45: 
        !            46: \item \textit{\textbf{Top and $W$ mass likelihood}} - a $\chi^{2}$-like variable. 
        !            47: $L\equiv\left(\frac{M_{3j}-m_{t}}{\sigma_{t}}\right)^{2}+\left(\frac{M_{2j}-M_{W}}{\sigma_{W}}\right)^{2}$,
        !            48: where $m_{t}, M_{W},\sigma_{t},\sigma_{W}$ are top and W masses (172.4
        !            49: GeV and 81.02 GeV respectively) and resolution values (19.4 GeV and 8.28
        !            50: GeV respectively). $M_{3j}$ and $M_{2j}$ are invariant masses composed
        !            51: of the jet combinations. We choose combination that minimizes $L$. 
        !            52: 
        !            53: \item \textit{\textbf{Centrality}}, defined as $\frac{H_{T}}{H_{E}}$ , where $H_{E}$ is sum
        !            54: of energies of the jets. 
        !            55: \item \textit{\textbf{$\cos(\theta*)$}} -  The angle between the beam axis and the 
        !            56: highest-$p_T$ jet in the rest frame of all the jets in the event.
        !            57: \item \textit{\textbf{$\sqrt(s)$}} - The invariant mass of all jets and $\tau$s in the event.
        !            58: 
        !            59: \end{itemize}
        !            60: 
        !            61: The chosen variables are in the end a consequence of the method employed  in this
        !            62: analysis: use events from the QCD-enriched 
        !            63: loose-tight sample to model QCD events in the signal-rich sample, and use
        !            64: a b-tag veto sample as an independent control sample to check the validity of such 
        !            65: background modeling.
        !            66: 
        !            67: %\clearpage
        !            68: 
        !            69: \subsection{\label{sub:NN-variables}Topological NN}
        !            70: For training the Neural Network we used the Multilayer Perceptron algorithm, as described in 
        !            71: \cite{MLPfit}. As explained before in Section \ref{sub:Results-of-the}, the first 1400000 
        !            72: events in the ``loose-tight'' sample were used as background 
        !            73: for NN training for taus types 1 and 2, and the first 600000 of the same sample for NN training for type 3 taus.
        !            74: This means that different tau types are being treated separately in the topological NN.
        !            75: In both cases 1/3 of the Alpgen sample of $t\bar{t} \rightarrow \tau +jets$ was used for NN training and 2/3 of it
        !            76: for the measurement.
        !            77: When doing the measurement later on (Section \ref{sub:xsect}) we pick the tau with the highest $NN(\tau)$
        !            78: in the signal sample as the tau cadidate at same time that taus in the loose-tight sample are picked at
        !            79: random since all of them are regarded as fake taus by being below the cut $NN(\tau)$ = 0.7. By doing this 
        !            80: we expect to avoid any bias when selecting real taus for the measurement.
        !            81: Figures \ref{fig:nnout_type2_training} and \ref{fig:nnout_type3_training} show the 
        !            82: effect of each of the chosen the topological event NN input variables on the final output.
        !            83: 
        !            84: Figures \ref{fig:nnout_type2} and \ref{fig:nnout_type3} show the NN output as a result of the training
        !            85: described above. It is evident from both pictures that high values of NN correspond to 
        !            86: the signal-enriched region. 
        !            87: 
        !            88: 
        !            89: 
        !            90: \begin{figure}[h]
        !            91: \includegraphics[scale=0.6]{plots/SetI_NNout_SM_type2_tauQCD.eps}
        !            92: \caption{Training of topological Neural Network output for type 1 and 2 $\tau$ channel. 
        !            93: Upper left: relative impact of each of the input variables; upper right: topological structure;
        !            94: lower right: final signal-background separation of the method; lower left: convergence curves.}
        !            95: \label{fig:nnout_type2_training}
        !            96: \end{figure}
        !            97: 
        !            98: \newpage
        !            99: 
        !           100: 
        !           101: \begin{figure}[h]
        !           102: \includegraphics[scale=0.6]{plots/SetI_NNout_SM_type3_tauQCD.eps}
        !           103: \caption{Training of topological Neural Network output for type 3 $\tau$ channel. 
        !           104: Upper left: relative impact of each of the input variables; upper right: topological structure;
        !           105: lower right: final signal-background separation of the method; lower left: convergence curves.}
        !           106: \label{fig:nnout_type3_training}
        !           107: \end{figure}
        !           108: 
        !           109: \begin{figure}[h]
        !           110: \includegraphics[scale=0.5]{CONTROLPLOTS/Std_TypeI_II/nnout.eps}
        !           111: \caption{The topological Neural Network output for type 1 and 2 $\tau$ channel}
        !           112: \label{fig:nnout_type2}
        !           113: \end{figure}
        !           114: 
        !           115: \newpage
        !           116: 
        !           117: \begin{figure}[t]
        !           118: \includegraphics[scale=0.5]{CONTROLPLOTS/Std_TypeIII/nnout.eps}
        !           119: \caption{The topological Neural Network output for type 3 $\tau$ channel}
        !           120: \label{fig:nnout_type3}
        !           121: \end{figure}
        !           122: 
        !           123: 
        !           124: \subsection{\label{sub:NN-optimization}NN optimization}
        !           125: One difference between this present analysis and the previous p17 is that we performed a NN optimization along with a
        !           126: $\not\!\! E_{T}$ significance optimization. Previously a cut of $>$ 3.0 was applied to $\not\!\! E_{T}$ significance 
        !           127: at the preselection stage and then it was included as one of the variables for NN training. This time as we 
        !           128: chose to optimize it, since it is still a good variable to provide signal-background discrimination (Figure \ref{fig:metl_note}).
        !           129: It is important to stress out that after the optimization we performed the 
        !           130: analysis with the optimized $\not\!\! E_{T}$ significance cut
        !           131: applied when doing both $\tau$ and b ID (Section \ref{sub:Results-of-the}), therefore 
        !           132: after the preselection where no $\not\!\! E_{T}$ significance cut was applied. 
        !           133: We then went back anp reprocessed (preselected) all MC samples with the optimized cut. Both results,
        !           134: with $\not\!\! E_{T}$ significance applied during and after preselectio were identical. 
        !           135: We then chose to present this analyis with this cut applied at the preselection 
        !           136: level in order to have a consistent cut flow throughout the analysis(Section \ref{sub:Preselection}).
        !           137: 
        !           138: 
        !           139: \begin{figure}[h]
        !           140: \includegraphics[scale=0.5]{plots/metl_allEW.eps}
        !           141: \caption{$\not\!\! E_{T}$ significance distribution for signal and backgrounds.}
        !           142: \label{fig:metl_note}
        !           143: \end{figure}
        !           144: 
        !           145: 
        !           146: \newpage
        !           147: 
        !           148: Below we describe how we split this part of the analysis into two parts:
        !           149: 
        !           150: \begin{enumerate}
        !           151: \item {\bf Set optimization:} We applied an arbitrary cut on $\not\!\! E_{T}$ significance of $\geq$ 4.0 and 
        !           152: varied the set of varibles going into NN training
        !           153: \item {\bf $\not\!\! E_{T}$ significance optimization:} After chosing the best set based on the lowest RMS, 
        !           154: we then varied the $\not\!\! E_{T}$ significance cut
        !           155: \end{enumerate}
        !           156: 
        !           157: For this part of the analysis we present the sets of variables that were taken into account to perform the NN traning
        !           158: \begin{itemize}
        !           159:  \item \textit{\textbf{Set I}} : {$H_{T}$},  aplan (aplanarity), sqrts ($\sqrt{s}$)
        !           160:  \item \textit{\textbf{Set II}} : {$H_{T}$},  aplan, cent (centrality)
        !           161:  \item \textit{\textbf{Set III}} : {$H_{T}$},  aplan, spher (spherecity)
        !           162:  \item \textit{\textbf{Set IV}} : {$H_{T}$}, cent, spher
        !           163:  \item \textit{\textbf{Set V}} : aplan, cent, spher
        !           164:  \item \textit{\textbf{Set VI}} : {$H_{T}$}, aplan, sqrts, spher
        !           165:  \item \textit{\textbf{Set VII}} : {$H_{T}$}, aplan, sqrts, cent
        !           166:  \item \textit{\textbf{Set VIII}} : {$H_{T}$}, aplan, sqrts, costhetastar ($cos(\theta^{*})$
        !           167:  \item \textit{\textbf{Set IX}} : {$H_{T}$}, aplan, sqrts, cent, spher
        !           168:  \item \textit{\textbf{Set X}} : {$H_{T}$}, aplan, sqrts, cent, costhetastar
        !           169:  \item \textit{\textbf{Set XI}} : {$H_{T}$}, aplan, sqrts, spher, costhetastar
        !           170:  \item \textit{\textbf{Set XII}} : metl, {$H_{T}$}, aplan, sqrts
        !           171:  \item \textit{\textbf{Set XIII}} : metl, {$H_{T}$}, aplan, cent
        !           172:  \item \textit{\textbf{Set XIV}} : metl, {$H_{T}$}, aplan, spher
        !           173:  \item \textit{\textbf{Set XV}} : metl, {$H_{T}$}, cent, spher
        !           174:  \item \textit{\textbf{Set XVI}} : metl, {$H_{T}$}, aplan
        !           175:  \item \textit{\textbf{Set XVII}} : metl, {$H_{T}$}, sqrts
        !           176:  \item \textit{\textbf{Set XVIII}} : metl, aplan, sqrts
        !           177:  \item \textit{\textbf{Set XIX}} : metl, {$H_{T}$}, cent
        !           178:  \item \textit{\textbf{Set XX}} : metl, {$H_{T}$}, aplan, sqrts, cent
        !           179:  \item \textit{\textbf{Set XXI}} : metl, {$H_{T}$}, aplan, cent, spher
        !           180:  \item \textit{\textbf{Set XXII}} : metl, {$H_{T}$}, aplan, sqrts, spher
        !           181:  \item \textit{\textbf{Set XXIII}} : metl, {$H_{T}$}, aplan, sqrts, costhetastar
        !           182:  \item \textit{\textbf{Set XXIV}} : metl, sqrts, cent, spher, costhetastar
        !           183:  \item \textit{\textbf{Set XXV}} : metl, {$H_{T}$}, cent, spher, costhetastar
        !           184:  \item \textit{\textbf{Set XXVI}} : metl, aplan, cent, spher, costhetastar
        !           185:  \item \textit{\textbf{Set XXVII}} : metl, {$H_{T}$}, aplan, cent, costhetastar
        !           186:  \item \textit{\textbf{Set XXVIII}} : {$H_{T}$}, aplan, topmassl
        !           187:  \item \textit{\textbf{Set XXIX}} : {$H_{T}$}, aplan, sqrts, topmassl
        !           188:  \item \textit{\textbf{Set XXX}} : {$H_{T}$}, aplan, sqrts, cent, topmassl
        !           189:  \item \textit{\textbf{Set XXXI}} : {$H_{T}$}, aplan, sqrts, costhetastar, topmassl
        !           190:  \item \textit{\textbf{Set XXXII}} : metl, {$H_{T}$}, topmassl, aplan, sqrts
        !           191:  \item \textit{\textbf{Set XXXIII}} : metl, spher, costhetastar, aplan, cent
        !           192: % \item \textit{\textbf{Set XXXIV}} : metl, spher, sqrts, topmassl, ktminp
        !           193:  \end{itemize}
        !           194: 
        !           195: The criteria used for making a decision on which variable should be used follow:
        !           196: \begin{itemize}
        !           197:  \item No more than 5 variables to keep NN simple and stable. More variables leads to instabilities (different 
        !           198:  result after each retraining) and require larger training samples.
        !           199: % \item We want to use $metl$ (\met significance) variable, since it's the one providing best discrimination.
        !           200:  \item We do not want to use highly correlated variables in same NN.
        !           201: % \item We can not use tau-based variables. 
        !           202:  \item We want to use variables with high discriminating power.
        !           203: \end{itemize}
        !           204: 
        !           205: In order to make the decision about which of these 11 choices is the optimal we created an ensemble of 
        !           206: 20000 pseudo-datasets each containing events randomly (according to a Poisson distribution) picked 
        !           207: from QCD, EW and $\ttbar$ templates. Each of these datasets was treated like real data, meaning applying all 
        !           208: the cuts and doing the shape fit of event topological NN. QCD templates for fit were made from the same 
        !           209: ``loose-tight $\tau$ sample'' from which the QCD component of the ``data'' was drawn. 
        !           210: The figure of merit chosen is given by Equation \ref{merit} below:
        !           211: 
        !           212: \begin{equation}
        !           213: f = \displaystyle \frac{(N_{fit} - N_{true})}{N_{true}}
        !           214: \label{merit}
        !           215: \end{equation}
        !           216: 
        !           217: 
        !           218: \noindent where $N_{fit}$ is the number of $t\bar{t}$ pairs given by 
        !           219: the fit and $N_{true}$ is the number of $t\bar{t}$ pairs from the Poisson distribution. 
        !           220: In both Set and $\not\!\! E_{T}$ significance optimization, the lowest RMS was used 
        !           221: to caracterize which configuration is the best in each case.
        !           222: 
        !           223: 
        !           224: The plots showing results concerning the set optimizations are found in Appendix \ref{app:set_opt} and are summarized 
        !           225: in Table \ref{setopt_table} below, where each RMS and mean are shown.
        !           226: For NN training is standard to choose the number of hidden nodes as being twice 
        !           227: the number the number of variables used for the training. The parenthesis after each set ID show the number of 
        !           228: hidden nodes in NN training.
        !           229: 
        !           230: \begin{table}[htbp]
        !           231: \begin{tabular}{|c|r|r|r|} \hline
        !           232: Set of variables  & \multicolumn{1}{c|}{RMS}    & \multicolumn{1}{c|}{mean} \\ \hline
        !           233: 
        !           234: \hline
        !           235: 
        !           236: 
        !           237: Set1(6)      &  \multicolumn{1}{c|}{0.1642}   &  \multicolumn{1}{c|}{0.0265}\\ \hline
        !           238: 
        !           239: Set2(6)      &  \multicolumn{1}{c|}{0.1840}     &  \multicolumn{1}{c|}{0.0054}\\ \hline
        !           240: 
        !           241: Set3(6)      &  \multicolumn{1}{c|}{0.1923}   &  \multicolumn{1}{c|}{0.0060}\\ \hline
        !           242: 
        !           243: Set4(6)      &  \multicolumn{1}{c|}{0.1978}   &  \multicolumn{1}{c|}{0.0175}\\ \hline
        !           244: 
        !           245: Set5(6)      &  \multicolumn{1}{c|}{0.2385}     &  \multicolumn{1}{c|}{0.0022}\\ \hline
        !           246: 
        !           247: Set6(8)     &  \multicolumn{1}{c|}{0.1687}   &  \multicolumn{1}{c|}{0.0115}\\ \hline
        !           248: 
        !           249: Set7(8)     &  \multicolumn{1}{c|}{0.1667}   &  \multicolumn{1}{c|}{0.0134}\\ \hline
        !           250: 
        !           251: Set8(10)     &  \multicolumn{1}{c|}{0.1668}     &  \multicolumn{1}{c|}{0.0162}\\ \hline
        !           252: 
        !           253: Set9(10)     &  \multicolumn{1}{c|}{0.1721}     &  \multicolumn{1}{c|}{0.0102}\\ \hline
        !           254: 
        !           255: Set10(10)     &  \multicolumn{1}{c|}{0.1722}     &  \multicolumn{1}{c|}{0.0210}\\ \hline
        !           256: 
        !           257: Se11(10)      &  \multicolumn{1}{c|}{0.1716}   &  \multicolumn{1}{c|}{0.0180}\\ \hline
        !           258: 
        !           259: Set12(8)     &  \multicolumn{1}{c|}{0.1662}     &  \multicolumn{1}{c|}{0.0039}\\ \hline
        !           260: 
        !           261: Set13(8)     &  \multicolumn{1}{c|}{0.1819}     &  \multicolumn{1}{c|}{0.0018}\\ \hline
        !           262: 
        !           263: Set14(8)     &  \multicolumn{1}{c|}{0.1879}     &  \multicolumn{1}{c|}{0.0019}\\ \hline
        !           264: 
        !           265: Set15(8)     &  \multicolumn{1}{c|}{0.1884}     &  \multicolumn{1}{c|}{-0.0004}\\ \hline
        !           266: 
        !           267: Set16(6)     &  \multicolumn{1}{c|}{0.1912}     &  \multicolumn{1}{c|}{0.0034}\\ \hline
        !           268: 
        !           269: Set17(6)     &  \multicolumn{1}{c|}{0.1768}     &  \multicolumn{1}{c|}{0.0074}\\ \hline
        !           270: 
        !           271: Set18(6)     &  \multicolumn{1}{c|}{0.2216}     &  \multicolumn{1}{c|}{-0.0030}\\ \hline
        !           272: 
        !           273: Set19(6)     &  \multicolumn{1}{c|}{0.1921}     &  \multicolumn{1}{c|}{0.0015}\\ \hline
        !           274: 
        !           275: Set20(10)     &  \multicolumn{1}{c|}{0.1620}     &  \multicolumn{1}{c|}{0.0262}\\ \hline
        !           276: 
        !           277: Set21(10)     &  \multicolumn{1}{c|}{0.1753}     &  \multicolumn{1}{c|}{0.0010}\\ \hline
        !           278: 
        !           279: Set22(10)     &  \multicolumn{1}{c|}{0.1646}     &  \multicolumn{1}{c|}{0.0086}\\ \hline
        !           280: 
        !           281: Set23(10)     &  \multicolumn{1}{c|}{0.1683}     &  \multicolumn{1}{c|}{0.0132}\\ \hline
        !           282: 
        !           283: Set24(10)     &  \multicolumn{1}{c|}{0.2053}     &  \multicolumn{1}{c|}{0.0122}\\ \hline
        !           284: 
        !           285: Set25(10)     &  \multicolumn{1}{c|}{0.1906}     &  \multicolumn{1}{c|}{0.0038}\\ \hline
        !           286: 
        !           287: Set26(10)     &  \multicolumn{1}{c|}{0.2130}     &  \multicolumn{1}{c|}{0.0028}\\ \hline
        !           288: 
        !           289: Set27(10)     &  \multicolumn{1}{c|}{0.1859}     &  \multicolumn{1}{c|}{0.0004}\\ \hline
        !           290: 
        !           291: Set28(6)     &  \multicolumn{1}{c|}{0.1910}     &  \multicolumn{1}{c|}{-0.0022}\\ \hline
        !           292: 
        !           293: Set29(8)     &  \multicolumn{1}{c|}{0.1587}     &  \multicolumn{1}{c|}{0.0214}\\ \hline
        !           294: 
        !           295: Set30(10)     &  \multicolumn{1}{c|}{0.1546}     &  \multicolumn{1}{c|}{0.0148}\\ \hline
        !           296: 
        !           297: Set31(10)     &  \multicolumn{1}{c|}{0.1543}     &  \multicolumn{1}{c|}{0.0203}\\ \hline
        !           298: 
        !           299: Set32(10)     &  \multicolumn{1}{c|}{0.1468}     &  \multicolumn{1}{c|}{0.0172}\\ \hline
        !           300: 
        !           301: Set33(10)     &  \multicolumn{1}{c|}{0.2201}     &  \multicolumn{1}{c|}{0.0081}\\ \hline
        !           302: 
        !           303: %Set34(10)     &  \multicolumn{1}{c|}{0.1955}     &  \multicolumn{1}{c|}{0.0184}\\ \hline
        !           304: \end{tabular}
        !           305: \caption{Results for set optimization part whit $\not\!\! E_{T}$ significance $>$ 4.0 applied to all sets.
        !           306: The number in parenthesis refers to number of hidden nodes in each case.}
        !           307: \label{setopt_table} 
        !           308: \end{table}
        !           309: 
        !           310: From Table \ref{setopt_table} we see that Set I has the lowest RMS, thus we chose it
        !           311: as the set to be used in $\not\!\! E_{T}$ significance optimization part, whose results are
        !           312: shown in Appendix \ref{app:metl_opt} and then summarized in Table \ref{metlopt_table} below
        !           313: 
        !           314: \begin{table}[htbp]
        !           315: \begin{tabular}{|c|r|r|r|} \hline
        !           316: Set of variables  & $\not\!\! E_{T}$ significance cut & RMS    & \multicolumn{1}{c|}{mean} \\ \hline
        !           317: 
        !           318: \hline
        !           319: 
        !           320: 
        !           321: %Set6(10)      &  \multicolumn{1}{c|}{1.0} &  \multicolumn{1}{c|}{0.2611}   \\ \hline
        !           322: 
        !           323: %Set6(10)      &  \multicolumn{1}{c|}{1.5} &  \multicolumn{1}{c|}{0.2320}   \\ \hline
        !           324: 
        !           325: %Set6(10)      &  \multicolumn{1}{c|}{2.0} &  \multicolumn{1}{c|}{0.2102}   \\ \hline
        !           326: 
        !           327: %Set6(10)      &  \multicolumn{1}{c|}{2.5} &  \multicolumn{1}{c|}{0.2021}   \\ \hline
        !           328: 
        !           329: Set32(10)      &  \multicolumn{1}{c|}{3.0} &  \multicolumn{1}{c|}{0.1507}   &  \multicolumn{1}{c|}{0.0157}\\ \hline
        !           330: 
        !           331: Set32(10)      &  \multicolumn{1}{c|}{3.5} &  \multicolumn{1}{c|}{0.1559}   &  \multicolumn{1}{c|}{0.0189}\\ \hline
        !           332: 
        !           333: Set32(10)     &  \multicolumn{1}{c|}{4.0} &  \multicolumn{1}{c|}{0.1468}   &  \multicolumn{1}{c|}{0.0172}\\ \hline
        !           334: 
        !           335: Set32(10)     &  \multicolumn{1}{c|}{4.5} &  \multicolumn{1}{c|}{0.1511}   &  \multicolumn{1}{c|}{0.0153}\\ \hline
        !           336: 
        !           337: Set32(10)     &  \multicolumn{1}{c|}{5.0} &  \multicolumn{1}{c|}{0.1552}   &  \multicolumn{1}{c|}{0.0205}\\ \hline
        !           338: 
        !           339: %Set6(10)     &  \multicolumn{1}{c|}{5.5} &  \multicolumn{1}{c|}{0.4008}   \\ \hline
        !           340: \end{tabular}
        !           341: \caption{Results for $\not\!\! E_{T}$ significance optimization part when varying the $\not\!\! E_{T}$ significance cut
        !           342: The number in parenthesis refers to number of hidden nodes in each case.}
        !           343: \label{metlopt_table} 
        !           344: \end{table}
        !           345: 
        !           346: 
        !           347: 
        !           348: Combined results from Tables \ref{setopt_table} and \ref{metlopt_table} show that the best configuration found
        !           349: was Set I  with $\not\!\! E_{T}$ significance $\geq$ 4.0. Therefore, this was the 
        !           350: configuration used to perform the cross-section measurement.Figure \ref{fig:METsig_RMS} shows the variation of the RMS as function 
        !           351: of the $\not\!\! E_{T}$ significance we applied.
        !           352: 
        !           353: \begin{figure}[b]
        !           354: \includegraphics[scale=0.4]{plots/METsig-RMS.eps}
        !           355: \caption{Plot of RMS as a function the $\not\!\! E_{T}$ significance applied}
        !           356: \label{fig:METsig_RMS}
        !           357: \end{figure}
        !           358: 
        !           359: 
        !           360: \clearpage
        !           361: 
        !           362: 
        !           363: In order to check the validity of our emsemble tests procedure, it is instructive to plot both the 
        !           364: distribution of the predicted number of $t\bar{t}$ and what is called ``pull'', defined in Equation  
        !           365: \ref{pull} below:
        !           366: 
        !           367: \begin{equation}
        !           368: p = \displaystyle \frac{(N_{fit}-N_{true})}{\sigma_{fit}}
        !           369: \label{pull}
        !           370: \end{equation}
        !           371: 
        !           372: \noindent where $\sigma_{fit}$ is the error on the number of $t\bar{t}$ pairs given by the fit.
        !           373: 
        !           374: Figures \ref{fig:gaus_ttbar} and \ref{fig:pull} show both beforementioned distributions.
        !           375: 
        !           376: From Figure \ref{fig:gaus_ttbar} we see a good agreement between the number of $t\bar{t}$ pairs
        !           377: initially set in the ensemble and the measured value. And Figure \ref{fig:pull} shows a nice gaussian
        !           378: curve, that indicates a good behaviour of the fit uncertainties in the ensembles.
        !           379: 
        !           380: \begin{figure}[t]
        !           381: \includegraphics[scale=0.5]{plots/gaus_ttbar.eps}
        !           382: \caption{Distribution of the output ``measurement'' for an ensemble with 116.9 $\ttbar$ events.}
        !           383: \label{fig:gaus_ttbar}
        !           384: \end{figure}
        !           385: 
        !           386: \begin{figure}[t]
        !           387: \includegraphics[scale=0.5]{plots/pull1-40.eps}
        !           388: \caption{The ensemble test's pull.}
        !           389: \label{fig:pull}
        !           390: \end{figure}
        !           391: 
        !           392: 
        !           393: %\newpage
        !           394: 
        !           395: 
        !           396: 
        !           397: 
        !           398: 
        !           399: \clearpage
        !           400: