tls: failed to verify certificate: x509: certificate signed by unknown authority when I use text2vec-openai module in weaviate docker container

Question

When I execute below code in operate_weaviate container(python:latest), text2vec-openai module(in weaviate) respond error message. Please tell me the way to fix this error.

■Code

import weaviate

# establish connection
client = weaviate.Client(
    url = "http://xxxxx:8080",
    additional_headers = {
        "X-OpenAI-Api-Key": os.environ.get("OPENAI_APIKEY")
    }
)

# create weaviate class
class_obj = {
    "class": "ManaminaArticle",
    "vectorizer": "text2vec-openai",
    "properties": [
        {
            "dataType": ["text"],
            "name": "sentence",
            "description": "sentence of article",
            "moduleConfig": {
                "text2vec-openai": {
                    "vectorizePropertyName": False,
                    "skip": False,
                }
            }
        }, ...
    ]
}

client.schema.create_class(class_obj)

# batch upload to weaviate
with client.batch as batch:
    batch.batch_size=100

    for i, data in enumerate(manamina_article):
        sentences = split_sentence(data["contents"], 200)

        for sentence in sentences:
            properties = {
                "title": data["title"],
                "url": 'https://manamina.valuesccg.com/articles/' + data["slug"],
                "sentence": sentence,
            }

            client.batch.add_data_object(properties, "ManaminaArticle")

■response(error message)

{'error': [{'message': 'update vector: send POST request: Post "https://api.openai.com/v1/embeddings": tls: failed to verify certificate: x509: certificate signed by unknown authority'}]}

■environment info docker-compose.yml

version: '3.4'
services:
  weaviate:
    command:
    - --host
    - 0.0.0.0
    - --port
    - '8080'
    - --scheme
    - http
    image: semitechnologies/weaviate:1.23.0
    ports:
    - 8080:8080
    - 50051:50051
    volumes:
    - weaviate_data:/var/lib/weaviate
    restart: on-failure:0
    environment:
      OPENAI_APIKEY: ${OPENAI_KEY}
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'text2vec-openai'
      ENABLE_MODULES: 'text2vec-openai,ref2vec-centroid,qna-openai,generative-openai'
      CLUSTER_HOSTNAME: 'node1'
    networks:
      - gpt-nw

  operate_weaviate:
    build: .
    container_name: operate_weaviate
    volumes:
      - ./xxxx:/xxxx
    working_dir: /xxxx
    tty: true
    stdin_open: true
    environment: 
      - CHOKIDAR_USEPOLLING=true
    networks:
      - gpt-nw

volumes:
  weaviate_data:

networks:
  gpt-nw:
    external: true

Dockerfile(for operate_weaviate container)

FROM python:latest

ENV CI=true
ENV http_proxy ${proxy}
ENV https_proxy ${proxy}

# Install ping
RUN set -x \
    && apt update -y \
    && apt upgrade -y \
    && apt install -y iputils-ping net-tools \
    && rm -rf /var/lib/apt/lists/*

COPY ./xxxx /xxxx
COPY ./xxxx/requirements.txt .

WORKDIR /xxxx

RUN pip install -r requirements.txt

We tried below command but there was nothing to improve.

1. install ca-certificates

$ apt-get install -y ca-certificates openssl

2. import open AI root CA using browser and set environment variable "REQUESTS_CA_BUNDLE"

# add below sentence before data insertion process
os.environ["REQUESTS_CA_BUNDLE"] = "../OpenAI_Root.crt"

The weaviate module doesn't seem to use Python's "requests" module internally.

Answer 1

I think your ENV http proxy have some problem

Because "https://api.openai.com/v1/embeddings" certificate is correct

FROM python:latest

ENV CI=true
ENV http_proxy ${proxy}
ENV https_proxy ${proxy}

Check your http proxy or https_proxy server is working properly.

If not using, just get rid of it

Also I'm confused which part is requesting to openapi, and where your proxy is used