I'm trying to run Gemini pro as suggested in Google Docs of VertexAI: https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/gemini-pro

import vertexai
from vertexai.preview.generative_models import GenerativeModel, ChatSession

# TODO(developer): Update and un-comment below lines
project_id = "<my_project_id>"
location = "us-central1"
vertexai.init(project=project_id, location=location)

model = GenerativeModel("gemini-1.0-pro")
chat = model.start_chat(response_validation=False)

def get_chat_response(chat: ChatSession, prompt: str):
    response = chat.send_message(prompt)
    return response.text

for prompt in prompt_list:
  print(get_chat_response(chat, prompt))

I've modified only by adding response_validation=False.

I run this code in a loop for 4,5K prompts form prompt_list. It works for around 150 prompts then it breaks with the following error:

2024-03-11 08:04:47 [error    ] Error in LLM request gender: 400 Request contains an invalid argument.
Traceback (most recent call last):
  File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
    return callable_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/grpc/_channel.py", line 1176, in __call__
    return _end_unary_response_blocking(state, call, False, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/grpc/_channel.py", line 1005, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.INVALID_ARGUMENT
    details = "Request contains an invalid argument."
    debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B2a00:1450:401b:80e::200a%5D:443 {created_time:"2024-03-11T08:04:47.917745+01:00", grpc_status:3, grpc_message:"Request contains an invalid argument."}"

The above exception was the direct cause of the following exception
...
 File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py", line 727, in send_message
    return self._send_message(
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py", line 814, in _send_message
    response = self._model._generate_content(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py", line 435, in _generate_content
    gapic_response = self._prediction_client.generate_content(request=request)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/google/cloud/aiplatform_v1beta1/services/prediction_service/client.py", line 2080, in generate_content
    response = rpc(
               ^^^^
  File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Request contains an invalid argument.
  1. How can I overcome this error?
  2. Can this be that I'm located in EU and using the US location?

I've removed validation.

0

There are 0 best solutions below