Prerequisites
Before installation, ensure your device is supported. You can reference our Supportability Matrix.
Step 1: Get Access to Latent AI Packages
Tip
You can make
LICENSE_KEY
andLEIP_WORKSPACE
persist for every newbash
session by placing the variables in~/.bashrc
.
In order to install packages or pull any of our Docker images, you’ll need to create a personal access token. To do so, follow these steps:
- Login to Latent AI Artifact repository.
- Click the
Sign-in
link in the upper right. - Select
Sign-in with SSO
. - Enter your access credentials.
- Click the
- Create your Personal Access Token.
- Click on your profile in the upper right.
- Select
User Token
on the left navigation. - Select the
Access User Token
button. - View your user token name and user token passcode.
- Export your user token name and passcode.
REPOSITORY_TOKEN_NAME=<token_name> REPOSITORY_TOKEN_PASS=<token_value>
Step 2: Install LRE on Target Hardware
Bug
There’s a current issue with
pip install pylre[liblre]
installingpylre==0.0.3
. You still needpylre[liblre]
regardless of this.As a workaround, install
pylre[liblre]
first, then runpip install pylre==1.0.0
.
Proceed with installation on your edge device by opening up a terminal session and following the steps below.
- Add the Latent AI packaged repository.
sudo wget -qO - https://public.latentai.io/add_apt_repository | sudo bash
- Run
sudo apt update
. - Install LRE. If your application is using Python, make sure you install PyLRE as well.
pip install --extra-index-url=https://$REPOSITORY_TOKEN_NAME:$REPOSITORY_TOKEN_PASS@repository.latentai.com/repository/pypi/simple pylre[liblre]
Step 3: A Simple PyLRE Test
You can do a simple PyLRE test with the model to confirm the model binary can perform inference:
import numpy as np
import torch as T
from pylre import LatentRuntimeEngine
from pathlib import Path
# Load runtime
lre = LatentRuntimeEngine("modelLibrary.so")
lre.set_model_precision("int8")
print(lre.get_metadata())
# Call runtime
input_data = np.asarray(np.random.rand(1,3,512,512)).astype(np.float32)
outputs = lre.infer(input_data)
output_torch = T.from_dlpack(lre.get_outputs()[0])
print(output_torch)