Autoscaling now available for model serving and deployments

4 years ago   •   2 min read

By David

We're pleased to announce the release of autoscaling for model serving and deployments on Paperspace Gradient.

Autoscaling is an essential MLOps tool for adjusting compute resources or instances dedicated to a machine learning workload dynamically.

The benefits of autoscaling are multiple:

  • Automatic rather than manual instance allocation
  • Tight control of costs since instances provision / deprovision automatically
  • High availability due to automatic instance failover

To autoscale a deployment on Gradient, follow these steps:

Step 0 - Create a deployment

Step 1 - Choose a model to deploy

Step 2 - Select a recommended container or a custom container

Step 3 - Select a cluster and a machine

Step 4 - Give the deployment a name

Step 5 - Enable autoscaling and provide target autoscaling parameters

Step 6 - Provide a point of entry to the container (optional) and then create your deployment!

At this point you should now have a brand new autoscaled deployment building in your deployments console! Congratulations!

For more information on autoscaling deployments and model serving, read the docs.

If you have any questions feel free to drop us a note!

Spread the word

Keep reading