Triton QuickStart Guide

Before you get started:

  • Make sure you understand our core Policies.

  • You need to be a member of a Triton project which has one of triton_faculty, triton_student or triton_education resource type.

  • Make sure you connect to the UM network (on campus or via VPN).

Basic Concepts

home directory vs. scratch directory (scratch space)

Each user will have a home directory on Triton located at /home/<caneid> as the working directory for submitting and running jobs. It is also for installing user software and libraries that are not provided as system utilities. Home directory contains an allocation of 250GB per user.

Each project group will have a scratch directory located at /scratch/projects/<project_name> for holding the input and output data. You can have some small and intermediate data in your home directory, but there are benefits to put data in the scratch directory:

  1. everyone in the group can share the data

  2. the scratch directory is larger (usually 2T, and you can require more)

  3. the scratch directory will be faster

login node vs. compute node

You can think of the login node as the “user interface” to the whole Triton system. When you connect to Triton and run commands on the command line, you are actually doing things on the login node.

When you submit jobs using bsub, Triton’s job scheduler will look for the compute nodes that satisfy your resource request and assign your code to the nodes to run. You do not have direct access to the compute nodes yourself.

Basic Steps

Here are the basic steps to run a simple Python script on Triton. In this example, the user has CaneID abc123 and is a member of Triton project xyz. You need to replace these with your own CaneID and Triton project name.

1. Preparing the code you would like to run

Editing the code

You can edit the code written in any programming language on your local computer. The example.py here is written in Python.

import matplotlib.pyplot as plt
import time

start = time.time()

X, Y = [], []

# read the input data from the scratch directory
# remember to replace xyz with your project name
for line in open('/scratch/xyz/data.txt', 'r'):
    values = [float(s) for s in line.split()]
    X.append(values[0])
    Y.append(values[1])

plt.plot(X, Y)

# save the output data to the scratch directory
# remember to replace xyz with your project name
plt.savefig('/scratch/xyz/data_plot.png')

# give you some time to monitor the submitted job
time.sleep(120)

elapsed = (time.time() - start)

print(f"The program lasts for {elapsed} seconds.")

Transfering the code to your Triton home directory

After editing the code, you need to transfer it from the local computer to your Triton home directory. You can do it with a file transfer tool such as FileZilla GUI application and scp command-line utility.

If using FileZilla, you need to put sftp://t2.idsc.miami.edu in the Host field, fill in the Username and Password fields with your CaneID and the associated password, and leave the Port field blank. By clicking the check mark icon in the menu bar, you will connect to Triton and the Remote site on the right will be your Triton home directory by default. Then, you can change the Local site on the left to the directory holding example.py and transfer the file by dragging it from left to right.

If using scp, you need to type, assuming origin is the absolute path that specifies the directory on your local computer holding example.py, scp origin/example.py abc123@t2.idsc.miami.edu:/home/abc123, not forgetting to put your CaneID in place of abc123, and then following the prompt for the associated password.

After that, the file will be located at /home/abc123/example.py on Triton for user abc123.

2. Preparing the input data

Getting the input data

In this example, you prepare the data.txt file as your input data on the local computer.

0  0
1  1
2  4
4 16
5 25
6 36

Transferring the input data to your project scratch directory on Triton

You can use FileZilla or scp to transfer the input data to /scratch/projects/xyz/data.txt on Triton. You need to replace xyz with your project name.

3. Installing dependent libraries on Triton

Logging in to Triton

You can use Terminal on a Mac or PuTTY on a Windows machine to log in to Triton via SSH Protocol.

If using Terminal on Mac, you can run the command ssh abc123@t2.idsc.miami.edu (remember to replace abc123 with your CaneID) and follow the instruction to type your password.

If using PuTTY, you need to put t2.idsc.miami.edu in the Host Name field, leave 22 in the Port field, and select SSH as the Connection type, then press Open. After that, you can follow the instruction to type your password.

At this point, you should be able to see the Triton welcome message and [abc123@mgt3.summit ~]$ which indicates you have logged in to the Triton login node and at the home directory ~.

If you are new to Linux, you can check our Linux Guides.

Installing software/libraries needed for the code

In the example, you will need the Python interpreter and Python packages to run the code. Also, for Python it is better to set up different environments for different projects to avoid conflictions of packages.

On Triton, you can use the system-installed Anaconda to do the Python environment set up:

[abc123@mgt3.summit ~]$ module load python/2.7.15-anaconda2-5.3.0
[abc123@mgt3.summit ~]$ conda create -n example_env python=3.8 matplotlib

4. Preparing the job script

Editing the job script

The job script is important. It tells the job scheduler how much resources your job needs, where to find the dependent software or libraries, and how the job should be run.

You can edit the example_script.job file to make example.py run on a Triton compute node.

#!/bin/bash
#BSUB -J example_job
#BSUB -o example_job%J.out
#BSUB -P xyz
#BSUB -n 1
#BSUB -R "rusage[mem=128M]"
#BSUB -q normal
#BSUB -W 00:10

module load python/2.7.15-anaconda2-5.3.0
conda activate example_env
cd ~
python example.py

  • #BSUB -J example_job specifies the name of the job.

  • #BSUB -o ~/example_job%J.out The line gives the path and name for the standard output file. It contains the job report and any text you print out to the standard output. %J in the name of the file will be replaced by the unique job id.

  • #BSUB -P xyz specifies the project (remember to replace xyz with your project name).

  • #BSUB -q normal specifies which queue you are submitting the job to. Most of the “normal” jobs running on Triton will submit to the normal queue.

  • #BSUB -n 1 requests 1 CPU core to run the job. Since the example job is simple, 1 CPU core will be enough. You can request up to 40 cores from one computing node on Triton for non-distributed jobs.

  • #BSUB -R "rusage[mem=128M]" requests 128 megabytes memory to run the job. Since the example job is simple, 128 megabytes memory will be enough. You can request up to ~250 gigabytes memory from one computing node on Triton.

  • #BSUB -W 00:10 requests 10 minutes to run the job. If you do not put this line, the default time limit is 1 day and the maximum time you can request is 7 days.

  • module load python/2.7.15-anaconda2-5.3.0 loads the Anaconda module on Triton.

  • conda activate example_env activates the Conda environment you created which contains the dependent Python package for the job.

  • cd ~ goes to the home directory where example.py is located.

  • python example.py runs example.py

Transferring the job script to your Triton home directory

You can use FileZilla or scp to transfer the job script to /home/abc123/example.job on Triton. You need to replace abc123 with your CaneID.

5. Submitting and monitoring the job

Job submission

[abc123@mgt3.summit ~]$ bsub < example_script.job

Job monitoring

While the job is submitted, you can use bjobs to check the status.

[abc123@mgt3.summit ~]$ bjobs

When the job is running you will see:

JOBID   USER    STAT  QUEUE      FROM_HOST   EXEC_HOST   JOB_NAME   SUBMIT_TIME
594966  abc123  RUN   normal     mgt3      t031        *ample_job Mar 25 11:43

If the job has finished you will see:

No unfinished job found

6. Checking the job output

Standard output file

This is the file you specify with #BSUB -o in your job script. In this example, after the job is finished, the standard output file example_job594966.out will be placed in the directory you submit the job, you can locate it to a different directory by giving the path. 594966 is the job id which is unique for each submitted job.

At the end of this file, you can see the report which gives the CPU time, memory usage, run time, etc., for the job. It could guide you to estimate the resources to request for the future jobs. Also, you can see the text you ask to print (to the stardard output) in example.py.

------------------------------------------------------------

Successfully completed.

Resource usage summary:

    CPU time :                                   8.89 sec.
    Max Memory :                                 51 MB
    Average Memory :                             48.50 MB
    Total Requested Memory :                     128.00 MB
    Delta Memory :                               77.00 MB
    Max Swap :                                   -
    Max Processes :                              4
    Max Threads :                                5
    Run time :                                   123 sec.
    Turnaround time :                            0 sec.

The output (if any) follows:

The program lasts for 120.23024702072144 seconds.

Output data

After the job is done, you will find the output data which is the png file saved in the scratch space. In this example, it is /scratch/projects/xyz/data_plot.png.

Transferring output file to local computer

You can view the output plot using any image viewer software on your local computer. To transfer the output file from Triton to your local computer, you can use FileZilla to drag the file from right to left, which transfers it, or you can use scp by typing, in the terminal on your local computer (assuming your CaneID is abc123, and destination is the absolute path that specifies the directory on the local computer to which you intend to move the file), scp abc123@t2.idsc.miami.edu:/scratch/projects/xyz/data_plot.png destination and following the prompt to provide a password.

7. Chao

Logging out from Triton on the command-line interface

[abc123@mgt3.summit ~]$ exit

Disconnecting from Triton on ``FileZilla``

On FileZilla, you can click on the x icon in the menu bar to disconnect from Triton.