I'm trying to run Gemini pro as suggested in Google Docs of VertexAI: https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/gemini-pro
import vertexai
from vertexai.preview.generative_models import GenerativeModel, ChatSession
# TODO(developer): Update and un-comment below lines
project_id = "<my_project_id>"
location = "us-central1"
vertexai.init(project=project_id, location=location)
model = GenerativeModel("gemini-1.0-pro")
chat = model.start_chat(response_validation=False)
def get_chat_response(chat: ChatSession, prompt: str):
response = chat.send_message(prompt)
return response.text
for prompt in prompt_list:
print(get_chat_response(chat, prompt))
I've modified only by adding response_validation=False.
I run this code in a loop for 4,5K prompts form prompt_list. It works for around 150 prompts then it breaks with the following error:
2024-03-11 08:04:47 [error ] Error in LLM request gender: 400 Request contains an invalid argument.
Traceback (most recent call last):
File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
return callable_(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/grpc/_channel.py", line 1176, in __call__
return _end_unary_response_blocking(state, call, False, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/grpc/_channel.py", line 1005, in _end_unary_response_blocking
raise _InactiveRpcError(state) # pytype: disable=not-instantiable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = "Request contains an invalid argument."
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B2a00:1450:401b:80e::200a%5D:443 {created_time:"2024-03-11T08:04:47.917745+01:00", grpc_status:3, grpc_message:"Request contains an invalid argument."}"
The above exception was the direct cause of the following exception
...
File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py", line 727, in send_message
return self._send_message(
^^^^^^^^^^^^^^^^^^^
File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py", line 814, in _send_message
response = self._model._generate_content(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/vertexai/generative_models/_generative_models.py", line 435, in _generate_content
gapic_response = self._prediction_client.generate_content(request=request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/google/cloud/aiplatform_v1beta1/services/prediction_service/client.py", line 2080, in generate_content
response = rpc(
^^^^
File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
return wrapped_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/l028/.pyenv/versions/playground_env/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InvalidArgument: 400 Request contains an invalid argument.
- How can I overcome this error?
- Can this be that I'm located in EU and using the US
location?
I've removed validation.