Skip to content

Pre-requisites for Deploying NAI

In this part of the lab we will prepare pre-requisites for LLM application on GPU nodes.

The following is the flow of the applications lab:

stateDiagram-v2
    direction LR

    state PreRequisites {
        [*] --> CreateFilesShare  
        CreateFilesShare --> PrepareHuggingFace
        PrepareHuggingFace --> [*]
    }


    [*] --> PreRequisites
    PreRequisites --> DeployNAI : next section
    DeployNAI --> TestNAI
    TestNAI --> [*]

Prepare the following pre-requisites needed to deploy NAI on target kubernetes cluster.

Create Nutanix Files Storage Class

We will create Nutanix Files storage class which will be used to create a pvc that will store the LLama-3-8B model files.

  1. In Prism Central, choose Files from the menu
  2. Choose the file server (e.g. labFS)
  3. Click on Shares & Exports
  4. Click on +New Share or Export

  5. Fill the details of the Share

    • Name - model_share
    • Description - for NAI model store
    • Share path - leave blank
    • Max Size - 10 GiB (adjust to the model file size)
    • Primary Protocol Access - NFS

  6. Click Next and make sure Enable compression in checked

  7. Click Next

  8. In NFS Protocol Access, choose the following:

    • Authentication - System
    • Default Access (for all clients) - Read-Write
    • Squash - Root Squash

    Note

    Consider changing access options for Production environment

  9. Click Next

  10. Confirm the share details and click on Create

Create the Files Storage Class

  1. Run the following command to check K8S status of the nkpdev cluster

    kubectl get nodes

  2. In VSC Explorer, click on New File and create a config file with the following name:

    nai-nfs-storage.yaml

    Add the following content and replace the nfsServerName with the name of the Nutanix Files server name .

    Finding nfsServerName and nfsServer fqdn

    apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nai-nfs-storage
parameters:
  nfsPath: <nfs-path>
  nfsServer: <nfs-server>
  storageType: NutanixFiles
provisioner: csi.nutanix.com
reclaimPolicy: Delete
volumeBindingMode: Immediate

    apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nai-nfs-storage
parameters:
  nfsPath: /model_share
  nfsServer: labFS.ntnxlab.local
  storageType: NutanixFiles
provisioner: csi.nutanix.com
reclaimPolicy: Delete
volumeBindingMode: Immediate

  3. Create the storage class

    kubectl apply -f nai-nfs-storage.yaml

  4. Check storage classes in the cluster for the Nutanix Files storage class

    kubectl get storageclass

    kubectl get storageclass

NAME                       PROVISIONER                     RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
dkp-object-store           kommander.ceph.rook.io/bucket   Delete          Immediate              false                  28h
nai-nfs-storage            csi.nutanix.com                 Delete          Immediate              true                   24h
nutanix-volume (default)   csi.nutanix.com                 Delete          WaitForFirstConsumer   false                  28h

Request Access to Model on Hugging Face

Follow these steps to request access to the meta-llama/Meta-Llama-3.1-8B-Instruct model:

LLM Recommendation

From testing google/gemma-2-2b-it model is quicker to download and obtain download rights, than meta-llama/Meta-Llama-3.1-8B-Instruct model.

Feel free to use the google/gemma-2-2b-it model if necessary. The procedure to request access to the model is the same.

  1. Sign in to your Hugging Face account:

  2. Navigate to the model page:

  3. Request access:

    • On the model page, you will see a section or button labeled Request Access (this is usually near the top of the page or near the "Files and versions" section).
    • Click Request Access.

  4. Complete the form:

    • You may be prompted to fill out a form or provide additional details about your intended use of the model.
    • Complete the required fields and submit the request.

  5. Wait for approval:

    • After submitting your request, you will receive a notification or email once your access is granted.
    • This process can take some time depending on the approval workflow.

Once access is granted, there will be an email notification.

Note

Email from Hugging Face can take a few minutes or hours before it arrives.

Create a Hugging Face Token with Read Permissions

Follow these steps to create a Hugging Face token with read permissions:

  1. Sign in to your Hugging Face account:

  2. Access your account settings:

    • Click on your profile picture in the top-right corner.
    • From the dropdown, select Settings.

  3. Navigate to the "Access Tokens" section:

    • In the sidebar, click on Access Tokens.
    • You will see a page where you can create and manage tokens.

  4. Create a new token:

    • Click the New token button.
    • Enter a name for your token (i.e., read-only-token).

  5. Set token permissions:

    • Under the permissions dropdown, select Read. For Example: hf-token

  6. Create and copy the token:

    • After selecting the permissions, click Create.
    • Your token will be generated and displayed only once, so make sure to copy it and store it securely.

Use this token for accessing Hugging Face resources with read-only permissions.

Prepare NAI Docker Download Credentials

All NAI Docker images will be downloaded from the public Docker Hub registry. In order to download the images, you will need to logon to Nutanix Portal - NAI and create a Docker ID and access token.

  1. Login to Nutanix Portal - NAI using your credentials
  2. Click on Generate Access Token option
  3. Copy the generated Docker ID and access token to a safe place as we will need it for the Deploy NAI section.

Warning

Currently there are issues with the Nutanix Portal to create a Docker ID and access token. This will be fixed soon.

Click on the Manage Access Token option and use the credentials listed there until the Nutanix Portal is fixed.