Improve Django filter preformance for all fields searching special ManyToMany

Question

I am trying to use django-filter to create a search for all fields. Here is my code:

from django.db.models import Q, CharField, TextField, ForeignKey, IntegerField, Func, Value
def get_all_fields_q_object(model, search_value, exclude_fields=None, prefix=None):
    q_object = Q()
    exclude_fields = exclude_fields or []

    for field in model._meta.get_fields():
        if field.name in exclude_fields:
            continue
        lookup_field_name = f"{prefix}__{field.name}" if prefix else field.name
        if isinstance(field, (CharField, TextField)):
            q_object |= Q(**{f"{lookup_field_name}__icontains": search_value})
        elif isinstance(field, ForeignKey):
            related_model = field.related_model
            related_q_object = get_all_fields_q_object(related_model, search_value, exclude_fields=exclude_fields, prefix=lookup_field_name)
            q_object |= related_q_object
        elif isinstance(field, IntegerField):
            try:
                int_value = int(search_value)
                q_object |= Q(**{lookup_field_name: int_value})
            except ValueError:
                pass
        elif isinstance(field, ArrayField):
            q_object |= Q(**{f'{lookup_field_name}__icontains': search_value})
        # # Add more field types as needed...

    return q_object

It works for integer, char, foreignkey and array at this moment. But now the model also added ManytoMany fields. I would like to show result based on string icontains. So I handled them like foreignkeyfields:

...
elif isinstance(field, ManyToManyField):
            related_model = field.related_model
            related_q_object = get_all_fields_q_object(related_model, search_value, exclude_fields=exclude_fields, prefix=lookup_field_name)
            q_object |= related_q_object

Although it works, but the result returns way too slow. Things are:

I assume manytomany fields take time to loop the search

I think about two ways : 1. convert manytomany fields object to a string only use PKname or 2. another search implement to django like SQLAlchemy? But I am not sure in the right direction...Any one has idea to improve it? Thank you!