System Overview


Overview

  • ~ 250 Teraflops Peak computing performance
  • Master/Management nodes
    • 2 Master Nodes in High Availability mode
    • 4 Login Nodes: 2 CPU only, 1 GPU based and 1 MIC based
    • 1 Management Node
  • Total 162 compute Nodes
    • 126 nodes containing 2 Intel Xeon E5-2680 v3, 12 Core, 2.5 GHz processors and 64 GB RAM per node
    • 4 High memory compute noded with 512 GB RAM per node
    • 16 Nodes containing 2 NVIDIA Tesla k40 (GPGPU) per node
    • 16 Nodes containing 2 Intel Xeon Phi 7120 (MIC) per node
  • 1 Mellanox FDR (56Gbps) 324 port chassis switch as primary high speed interconnect
  • 300TB Storage with 15GB/s write throughput based on lustre parallel file system
  • Stoftware Stack includes: CentOS 6.6, Intel Parallel Studio 2016, GNU compilers, Intel MPSS, CUDA, Mellanox OFED, Luster, SLURM Resource Manager & Scheduler and Bright Cluster Manager.

Schematic Diagram

Compute without any Accelerator

  • 126 nodes
  • 3024 cores
  • 2 x Intel Xeon E5-2680 v3, 12-core, 2.5 GHz processors per node
  • 64 GB of physical memory per node
  • Compute power of 121 Tflops

High Memory Compute Nodes without any Accelerator

  • 4 nodes
  • 96 cores
  • 2 x Intel Xeon E5-2680 v3, 12-core, 2.5 GHz processors per node
  • 512 GB of physical memory per node
  • Compute power of 3.8 Teraflops

Compute Nodes with GPU

  • 16 nodes
  • 384 cores
  • 2 x Intel Xeon E5-2680 v3, 12-core, 2.5 GHz processors per node
  • 64 GB of physical memory per node
  • GPU accelerator 2 x NVIDIA Tesla K40 per node
  • Compute power of 60 Tfops/s

Compute Nodes with Xeon Phi

  • 16 nodes
  • 384 cores
  • 2 x Intel Xeon E5-2680 v3, 12-core, 2.5 GHz processors per node
  • 64 GB of physical memory per node
  • MIC accelerator 2x Intel Xeon Phi 7120 per node
  • Compute power of 47.36 Tfops/s

Software Stack

File System

  • Home
    • 100TB lustre based Storage
    • 30GB default quota
  • Scratch
    • 10GB/sec write throughput
    • Users are recommended to use this file-system during execution of their job
    • They must transfer back their data to home file-system
  • Archive
    • Policy based movement of Home file-system data to archive filesystem 16 nodes