Artifacts Download
feature in TrueFoundry which automatically provisions a volume, downloads the model onto the volume, mounts the volume and also handles upgrades safely. The way this works during the first deployment is:
Download Path Environment Variable
you specified while configuring the Download and Cache Artifact
.
You can do one of these two things, Get it directly in your code using os.environ.get()
or Pass it as a command-line argument if your service needs it that way.
MODEL_ID_1
as theDownload Path Environment Variable
, to download model.joblib
file, you can access the cached model like this:
vllm/vllm-openai:v0.2.6
docker image. The following takes the model and tokenizer as a command-line argument.
You can pass the Download Path Environment Variable
in the arguments.