Our team has been undergoing optimizing our Django DRF / Graphene stack for latency we can't quite understand the source of our slowdowns
Our Stack:
- Python 3.11
- Django 4.2.8
- DRF 3.14.0
- Django Graphene 3.1.5
We've done the 'basic' optimizations of
- implementing prefetch / select related
- removed all n+1 generated SQL queries
And of course, scrounged the internet looking for answers, and saw the ModelSerializer issue posted here: https://hakibenita.com/django-rest-framework-slow
Which as I understand it is 'mostly' fixed in the latest Django?
We've manually optimized our QuerySets . SQL, and got our worst case SQL latency is around 200ms, most in the 10-100 range.
However, our TTFB, is anywhere from 1 to 3 seconds as measured in dev tools on a browser performing a graphiQL query, as well as in our test suites
Here are some numbers from GraphiQL Console:
One GQL QUery generates 6 optimzied SQL Queries resulting in a response of a 1.4 MB JSon file which has a heavy amount of nested of elements (necessary for our workload)
Django Debug Toolbar numbers:
- 101.9ms for SQL access
Chrome Dev Tools
- 6 SQL Querues 56.44 ms (not sure how it knows this?)
- 44us to send request
- 937 ms waiting for server response
- 8.48 ms to download
- Total ~1 second for a 100ms DB query!
To try to figure out the source which at this point I presume is Serializers or internal JSON creation? Are serializers THIS slow?
How would folks go about optimizing the TTFB given the 10 to 1 discrepancy of DB perf to App Server perf?
Thank you!
Here's a PySpy profile : https://www.dropbox.com/scl/fi/61eo71pyb2hvvbqsttol4/profile-gunicorn-with-signed-url.svg?rlkey=j7ggax4k3f4r6clxodr8kwt0r&dl=0
- S3 signed URL Hash creation takes roughly 30% of the time (we generate of lot of signed URLS)
- JSON response construction is a measly 1-2% at most
- The rest seems pretty squarely to fall into Django / Graphene SQL compilation and planning.
- The Profile is for the degenerate case I mention above, 100ms SQL total, but roughly 1 second Django Runtime usage.