Performance: LDPViewSet can prefetch the appropriate fields
In my performance testing on 30/10/2020, I found that by using Django's prefetch_related utility during the LDPViewSet.get_queryset
with the right fields, I was able to reduce the serialization time on /users/
of 9,000 resources from avg. 22s to avg. 14.5s, and reduce the number of database hits from around 9,000 to < 50 (figure 8)
The issue then is to select the appropriate fields automatically (as doing it manually reintroduces an old issue of tight-coupling between DjangoLDP packages)
The optimum fields for /users/
was:
('projects', 'projects__user', 'projects__project', 'circles', 'jobOffers', 'inbox', 'groups', 'skills', 'communities', 'communities__user', 'communities__community', 'chatProfile', 'profile', 'account')
Which can be broken up into the sub-groups
Foreign keys: a priori we can fetch all of these without any significant overhead
('chatProfile', 'profile', 'account')
Through-model foreign keys: likewise these can be fetch-all, and automatically deduced from the through model, to a certain depth
('projects__user', 'projects__project', 'communities__user', 'communities__community')
Selected nested-fields: this is the tricky part. We're after all the nested fields that will be serialized
Implementation
LDPViewSet
should generate (via a new method) automatically the prefetch fields described above
- working with
cls.depth
- (unit) testing