Connecting to the HPC Cluster

There are several ways to interact with the Greene HPC cluster. Similar to other Linux clusters, the most common method of connection is via a Command Line Interface (CLI). A CLI is a program that allows you to create and delete files, run programs, and navigate through directories and files via a typed prompt. On Mac, the built-in CLI application is called Terminal. While Windows 11 machines support a Linux Subsystem, which allows for similar functionality, a popular tool used to connect to a Linux server is a free application called PuTTY.

tip

If you'd prefer to access the HPC cluster via a web gateway instead, please refer to the section describing Open OnDemand (OOD).

The following sections will outline basic ways to connect to the Greene cluster. Access to the clusters is primarily handled via the Secure Shell Protocol, or ssh. Below we outline ways to connect via ssh on Mac, Linux, and Windows machines.

Quick Start HPC Connection Guide

Configuring Your SSH Client

To connect to HPC systems, it's important to configure your machine's SSH client. For Linux and Mac machines, the configuration file is the ~/.ssh/config file on your computer. These are the basic lines that should be added to your ~/.ssh/config file:

Host greene.hpc.nyu.edu dtn.hpc.nyu.edu gw.hpc.nyu.edu
  StrictHostKeyChecking no
  ServerAliveInterval 60
  ForwardAgent yes
  UserKnownHostsFile /dev/null
  LogLevel ERROR
 
Host hpcgwtunnel
  HostName gw.hpc.nyu.edu
  ForwardX11 no
  StrictHostKeyChecking no
  LocalForward 8027 greene.hpc.nyu.edu:22
  UserKnownHostsFile /dev/null
  User <Your NetID>

Host greene
  HostName localhost
  Port 8027
  ForwardX11 yes
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  LogLevel ERROR
  User <Your NetID>

Setting Up an SSH Config File on Windows

For Windows, you can now follow a similar configuration setup. Using Powershell you can use the following to create and open a Windows SSH config file:

# Create the config file with Powershell
New-Item -Path $HOME\.ssh\config -ItemType File

# Open config File with Notepad
C:\WINDOWS\System32\notepad.exe $HOME\.ssh\config

Configuring the file above will allow you to use the cmd terminal to connect to Greene.

Avoiding Man in the Middle Warning.

If you see this warning:

warning

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.

Do not be alarmed - this is an issue that occurs because the cluster has multiple login nodes (log-1, log-2, and log-3) that greene.hpc.nyu.edu resolves to.

To avoid this warning, you can add these lines to your SSH configuration file. Open ~/.ssh/config and place the following lines in it:

tip

This segment, referenced in the recommended config above will nullify the error:

Host greene.hpc.nyu.edu dtn.hpc.nyu.edu gw.hpc.nyu.edu
  StrictHostKeyChecking no
  ServerAliveInterval 60
  ForwardAgent yes
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  LogLevel ERROR

The above will also fix SSH timeout errors by extending the ServerAliveInterval argument.

Connecting to the NYU Network

To access the HPC systems, you must either be on the NYU secure network (such as the NYU WiFi or a wired connection from an NYU office) or use a method of accessing it, such as the NYU VPN or the HPC Gateway servers.

Connecting to the HPC clusters from outside NYU Network

VPN (Preferred)

Set up your computer to use the NYU VPN. Once you've created a VPN connection, you can proceed as if you were connected to the NYU net

Gateway

Go through our gateway servers (example below). Gateways are designed to support only a very minimal set of commands and their only purpose is to let users connect HPC systems without needing to first connect to the VPN

Command Line Interface (with a Terminal)

Mac & Linux Access

VPN Connection

If you're on the VPN or the main NYU network on campus you can SSH directly to greene like so:

ssh <Your NetID>@greene.hpc.nyu.edu

Gateway / SSH Tunneling Option

To connect to the gateway servers, after you've configured your ~/.ssh/config file, simply open a terminal application and follow these steps:

1. Open a terminal
2. Connect to the gateway via the ssh command:

ssh hpcgwtunnel

3. Authenticate with your password and Duo MFA

4. Open a second terminal

5. Connect to the Greene login nodes via the ssh command:

ssh greene

6. Authenticate

After typing in your password you will be logged in to the cluster.

Windows CMD

Windows 11 users have several options. First, the CMD program should contain an ssh client, allowing you to log into the HPC systems the same way as with a Linux terminal.

Windows WSL2

If you run Windows 10, you can install WSL, and then install Ubuntu or other Linux distribution (for example, from Microsoft Store). You will have a fully functional Ubuntu with terminal and can connect to cluster using instructions provided above for Linux/Mac users.

Instructions on WSL installation can be found here: https://docs.microsoft.com/en-us/windows/wsl/install-win10

tip

• One of many options to get terminal that support tabs, etc. is to install 'Windows Terminal' from Microsoft Store.
• If you are using WSL 2 (Windows subsystem for Linux), you may not be able to access internet when Cisco AnyConnect VPN, installed from exe file, is activated. A potential solution: uninstall Cisco AnyConnect and install AnyConnect using Microsoft Store, and then setup new VPN connection using settings described on IT webpage.

Additional Tips

Below are some additional options and tips for connecting to the HPC resources.

Setting up SSH Keys

Instead of typing your password every time you need to log in, you can also specify an ssh key.

• Only do that on the computer you trust

• Generate ssh key pair (terminal in Linux/Mac or cmd/WSL in Windows): https://www.ssh.com/ssh/keygen/

• Note the path to ssh key files. Don't share key files with anybody - anybody with this key file can login to your account

• Log into cluster using regular login/password and then add the content of generated public key file (the one with .pub) to $HOME/.ssh/authorized_keys on cluster

• Next time you will log into cluster no password will be required

For additional recommendations on how to configure your SSH sessions, see the ssh configuring and x11 forwarding page.

PuTTY (Only for Windows)

There are many SSH clients for Windows OS, but we recommend using PuTTY SSH if you have not already. Once it is installed, launch PuTTY and configure new session "Session" category as in the screenshot below:

Here we are instructing PuTTY to connect to host gw.hpc.nyu.edu on port 22 using SSH protocol (note, that this interface allows you to save this connection configuration for future). Just like for Linux and Mac users, if you are connecting from the outside of NYU network, you need to go through the gateway servers.

Once you click "Open", a terminal window with prompt for password will pop up. Enter your NetID password and you should be authorized on the gateway server. Gateways are designed to support only a very minimal set of commands and their only purpose it to let users access HPC systems. Once you are there type in an ssh command that will let you connect to Greene cluster :

# Gateway Login
ssh gw.hpc.nyu.edu

A new command line interface window will open up that prompts you for your password on the gateway server, from there you can connect to Greene by entering the following:

ssh greene.hpc.nyu.edu