Deploying sample MNIST model with AWS Multi-Model Server
model/mnist.py
: The pytorch model definition.model/mnist_cnn.pt
: The trained pytorch model checkpoint.model/mnist_handler.py
: Contains the main handler that runs the inference.requirements.txt
: Contains the dependencies.config.properties
: Contains the configuration for the model server.handle
that takes in data
and context
as inputs and returns the inference output.
The function in our code orchestrates the other 4 functions:
initialize
: Loads the model and any other resources needed for inference.preprocess
: Preprocesses the input data.inference
: Runs the inference.postprocess
: Postprocesses the output data..mar
file
mnist.mar
file.
Log the MAR Model To Model Registry
mnist.mar
file to the model registry. You can follow the guide here to log the model to the registry.mnist.mar
file.Push the code to a Git repository or directly deploy from local machine
MODEL_DIR
environment variable where the model will be downloaded to.Download Model from Model Registry in the deployment configuration
MODEL_DIR
environment variable to the deployed service.Add the model id and revision from HuggingFace Hub in Artifacts Download
sectionView the deployment, logs and metrics