how to deploy custom container in Vertex AI as endpoint

Question

i am trying to deploy custom container in vertex ai as endpoint ( REST URL or API) , i am able to build the docker image successfully , but not able to deploy the model as endpoint , from the logs also i am not able to understand whats the error.

below is my predict.py , dockerfile and deployment script

import numpy as np

def predict(data):
  # Example prediction function
  print (data)
  #result = np.sum(data, axis=1)  # Example: Sum along axis 1
  results = [3,4,5,6]
  results_array = np.array(results)
  print ({"predictions": results_array.tolist()})
  return ({"predictions": results_array.tolist()})
  predict([3,4,6])

dockerfile

FROM python:3.10
WORKDIR /code

COPY requirements.txt requirements.txt
COPY model.pkl model.pkl
RUN pip install --upgrade pip
RUN pip --version
RUN pip install -r requirements.txt
COPY . .
CMD ["python3", "predict.py"]

Deployment script

import os
import google.cloud.aiplatform as aiplatform

# Set your GCP project ID, location, and model name
project = "project ID"
location = "us-central1"
model_name = "testing_3"

# Initialize the API client
aiplatform.init(project=project, location=location)

# Define the container image URI
container_image_uri = "image URI"

# Create a custom container prediction model
model = aiplatform.Model.upload(
display_name=model_name,
serving_container_image_uri=container_image_uri,
 )

print ("deploying the model now")
try:
  # Deploy the model
  endpoint = model.deploy(machine_type="n1-standard-4")
  print(endpoint)
except Exception as e:
  print(f"Error deploying the model: {e}")

i have also try to run the deployed docker image and its running without any error , only i am not able to deploy the endpoint

Can anyone help me here?

Answer 1

I'm always starting with an official base image from GCP like so:

FROM gcr.io/deeplearning-platform-release/pytorch-gpu.2-0.py310:latest

This one is with GPU support but there are others: https://console.cloud.google.com/gcr/images/deeplearning-platform-release/GLOBAL

With those base images you don't need to care about GCP SDK and its dependencies.

Answer 2

It is possible that you are running out of resources hence having an error when deploying your custom container. Kindly check your quota and limits, at the same time, take a look at the custom container requirements.

I found a similar question, posted some months ago and I believe this will help you resolve the problem.