Versions Compared

Old Version 53

changes.mady.by.user Kimberly Thomas

Saved on

compared with

New Version Current

changes.mady.by.user Cheung, Wilson

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Complex Machine Learning workflows are supported through terminal/SSH logins and a full Linux/Ubuntu CUDA development suite. Users may install additional library packages (e.g. conda/pip, CRAN) as needed, or can opt to replace the default environment entirely by launching their own custom Docker containers. High High-speed cluster-local storage houses workspaces and common training corpora (e.g. CIFAR, ImageNet).

...

The are two ways in which to access the Research Cluster - via ssh SSH or via the datahubDatahub.

Expand
titleAccessing the Research Cluster via SSH

First, login via SSH to the “dsmlpresearchcluster-login.ucsd.edu" Linux server login node using your UC San Diego Active Directory (AD) username (with ‘@dsmlp‘@researchcluster-login.ucsd.edu’) and password.  After logging in, you will be in a login node for the Research Cluster and should not perform any computation in the login node.

Login step-by-step guidance:

  • Open command line interface - known as the 'Terminal' for MacOS and 'Command Prompt' for Windows.

    • Enter command ‘ssh ADusername@dsmlp

    in the login node.

    Login step-by-step guidance:

    1. Open command line interface - known as the 'Terminal' for MacOS and 'Command Prompt' for Windows.

    2. Enter command ‘ssh ADusername@researchcluster-login.ucsd.edu'.

      1. If the researchcluster-login.ucsd.edu login node is currently down/unavailable, you may TEMPORARILY use the “dsmlp-login.ucsd.edu” login node instead. The dsmlp-login.ucsd.edu login node is intended for instructional class work use by students as its first priority, so it’s recommended to always use the researchcluster-login.ucsd.edu

      '

      1. for research work.

    You may be asked a question after entering your username. Select 'yes’ to continue connecting.

    1. Enter your password. Note: Your password will not display as you enter it.

    2. A successful login will display your last login information. For example, ‘Last login: Thu Aug 3 10:25:19 2023 from 137.110.14.162’. Note that you are now in the Login Node. 

    IMPORTANT: DO NOT RUN JOBS IN THE LOGIN NODE. ! Jobs must only be run in a launched container. Follow the guidance in the next section (Launching a Container), before running your compute jobs. 

    Expand
    titleAccessing the Research Cluster via the datahub

    1. Login page: https://datahub.ucsd.edu/hub/login .

      • Enter UCSD email address and AD password. You will be sent a push to confirm via DUODuo.

    2. Select the your chosen notebook environment. 

      • Research Cluster users will have a choice of multiple environments to select.

      • If joining a PI/lab specific environment, your may only see the name of your PI/lab’s environment.

      • The ‘public’ folder will include storage that is shared and where datasets can be stored for all to access.

    3. Next, you’ll be directed to your environment and see the Jupyter Notebook interface.

    4. Click ‘New’ and select the kernel you wills to start up.

    5. When done using the datahub, select ‘logout’ to terminate kernels and end session.

    ...

    Expand
    titleStandard launch scripts

    The standard launch scripts are predefined meaning they have specific RAM and CPU (and/or GPU) configurations. Other launch scripts are available at /softwareopt/common64launch-sh/dsmlp/bin/ .

    Launch Script

    Description

    #GPU

    #CPU

    RAM

    Container Image(s)

    launch-scipy-ml.sh

    Python 3, PyTorch, TensorFlow

    0

    2

    8

    ucsdets/scipy-ml-notebook:2020.2.9

    launch-scipy-ml-gpu.sh

    Python 3, PyTorch, TensorFlow

    1

    4

    16

    ucsdets/scipy-ml-notebook:2020.2.9

    launch-datascience.sh

    Python 3, Datascience, R

    0

    2

    8

    ucsdets/datascience-notebook:2020.2-stable

    launch-rstudio.sh

    R-Studio

    1

    4

    16

    ucsdets/datascience-rstudio:latest

    Standard images with ‘pytorch’ include the GNU Screen utility, which may be used to manage multiple terminal sessions in a window-like manner.  

    ...

    To access the web interface tool, users are directed to sign-in at https://datahub.ucsd.edu/ (or via selecting the login button up top).

    ...

    Expand
    titleLaunching a Jupyter Notebook

    Click on the "Log In" button above, or visit https://datahub.ucsd.edu/ and sign in with your UC San Diego Google account and password.
    (Note: only '@ucsd.edu' addresses are currently accepted, not departmental or divisional addresses such as '@eng.ucsd.edu' or '@physics.ucsd.edu'.)

    Click the    button.

    Select a software and hardware configuration via the "Spawner options" page:

    Open a blank Python 3 notebook:

    When your work is complete, please shut down your Notebook via the Control Panel's "Stop my Server" option:

    ...