NICADD Compute Cluster

As of 2018, the NICADD cluster provides 700 processor slots (1.8-2.6 GHz) under the HT CONDOR batch system and 200 TB of shared disk space, with instant access to CERN, FERMILAB and OSG software libraries. The system also serves NICADD's data acquisition servers and desktops for collecting and analyzing test data for MU2E, DUNE, and beam experiments. Detailed hardware description.

Alma Linux 8 upgrade (2023)

Alma Linux 8.x installed in January 2024; software packages are in sync with the NIU Metis system.

Access to the cluster login nodes

  • ssh -Y username@nodename.nicadd.niu.edu

All login and compute nodes are running Alma Linux 8.X, HT-Condor 10.0.2 batch system, CVMFS (and thus cvmfs-based tools of ATLAS, CMS, and DUNE), and provide local ROOT and custom compiler/linker tools via modules:

  • ROOT: module load root/root-6.26.06-gcc-11.3.0-cuda-11.8-pythia8307-el8
  • GCC+OPENMPI+CUDA: module load openmpi/openmpi-4.1.1-gcc-11.3.0-cuda-11.8-el8
  • INTEL-ONEAPI: module load intel/intel-oneapi-2022.1.2-el8

Nodes configuration

  • Interactive login nodes (1 Gbit/s public uplink, accessible via ssh)
    • t3int0.nicadd.niu.edu (Alma8) - ATLAS T3 login node
    • hpcm.nicadd.niu.edu (Alma8) - Beam group login node
    • cms1.nicadd.niu.edu (Alma8) - CMS login/CUDA developer node

Note:: for wireless access within NIU please use the "NIUwireless" network (the "NIUguest" blocks ssh ports).

  • HTCONDOR batch system (1-4 Gbit/s internal network)
    • Alma Linux 8.X, 1.5-2.5 GB of RAM per batch slot, up to 128 GB for special tasks
    • Nodes: pt3wrk0-pt3wrk4 (T3 ATLAS), phpc0-phpc2 (beam), pcms0-pcms6 (CMS), pnc0-pnc3 (shared)
  • InfiniBand subluster (Alma Lnux 8.X, ibhpc0 - ibhpc2)

A closer look at NICADD computing (slides).

Disk space configuration

  • /home/$USER - user's home area, 5GB initial quota
  • /bdata, /xdata - shared data partitions (80 TB each), for code development and analysis projects
  • /nfs/work/$nodename - local nodes scratches (1-4 TB), for batch jobs use

Backup policy.

We only provide the previous day's backup of /the home area. We strongly encourage users to use the GIT system for any code development and frequently backup essential data to remote locations.


Page last modified on January 24, 2024, at 11:15 AM EST

|--Created: Sergey A. Uzunyan -|- Site Config --|