Performance investigation
Summary
We need to improve as much as possible the performance on Startinblox so that Orbit is usable enough to be sold eveywhere.
Lead Dev(s)
@matthieu @calummackervoy @balessan
Startup time
- Configuration of Skypack
- Time before authentication
- caching looks ok
Data loading time
- Projects/circles/users loaded one by one on Orbit
- How can we use server side pagination? framework/sib-core#447 (closed)
- Total data transfer on startup looks ok
- Server response time on these requests looks ok
- Memory usage before component activation looks ok (60MB)
- How can we deal with federations? What's the maximum reasonable number of servers?
Component specific performances
-
Delay before first display of directory not acceptable.
Too many requests-> https://git.startinblox.com/blox/profile-directory/issues/7 - Memory usage of solid-xmpp-chat not acceptable
Other core specific issues
- framework/sib-core#694 -> Insights related
- framework/sib-core#611 (closed)
- framework/sib-core#618 (closed)
- framework/sib-core#616 (closed)
- framework/sib-core#594 (closed) -> Priority one
- framework/sib-router#62 (closed)
Related issues in Djangoldp-Packages
- djangoldp-packages/djangoldp#269 (closed)
- djangoldp-packages/djangoldp#382
- djangoldp-packages/djangoldp#285
- djangoldp-packages/django-webidoidc-provider#15 (closed)
- djangoldp-packages/djangoldp#299 (closed)
Related issues in Blox or App repositories
- applications/hubl#800
- https://git.startinblox.com/applications/hubl/issues/1038
- components/solid-xmpp-chat#260 (closed)
Specifications related
Customers waiting for it
- FNK
- HD
- ESS
- All of them, maybe
Related issues in the associated project (if any)
Expected delivery date
- 7th, May
Next steps
- Test in some apps
- FNK
- Coopstarter
- Happy Dev Website?
Related milestones
- Milestone in sib-core
- Milestone in customer project
/cc @balessan
Activity
@balessan for the related issues, I think https://git.startinblox.com/applications/sib-app/issues/382 is linked to the Auth pb, not performance. I don't know if we want to follow this, but https://git.startinblox.com/applications/sib-app/issues/403 was part of the performance problem and is done now :)
assigned to @matthieu
@balessan do we agree that framework/sib-core#447 (closed) will be treated in a next release?
If yes, we have a proposal for framework/sib-core#594 (closed). We need to test it in real apps
Yep I do agree for the #447
For #594 the thing if you are comfortable for that would be to push that to beta already and propose the projects team to test it.
First thing could be to validate that with @calummackervoy before.
-
I don't feel comfortable enough, as we may want to revert some changes due to the visual changes I described here: framework/sib-core#594 (comment 26175)
@calummackervoy, do you have a setup which allows you to test the sib-app with a specific branch of the core? Do you need help for that?
marked the checklist item framework/sib-core#611 (closed) as completed
marked the checklist item https://git.startinblox.com/applications/sib-app/issues/417 as completed
-
Some updates on this:
The remaining issue related to performances in applications is applications/hubl#399 (closed)
It has been improved already by:
It still need a few more to be fixed, but they will be treated later:
- framework/sib-core#618 (closed) -> new rendering system + widget refactoring
- framework/sib-core#594 (closed) -> new rendering system
- framework/sib-router#62 (closed) -> router SEO refactoring
mentioned in issue framework/sib-core#594 (closed)
-
Hello @balessan due date here is still on May 7th. I'm guessing this is outdated :) . Any news on when this is going to move forward?
-
Hello @rachel : I confirm that this is outdated. Then, performance will be one of our important topic this summer. We are launching the investigation backend side this afternoon and @matthieu already produced some benchmarks and ideas client side.
So, looking forward for some improvements in a near future.
I recently completed some basic performance tests on DjangoLDP, and whilst I was able to make one minor improvement to performance (and I'm sure there are others), I found that the serialization overall grows linearly (which I think is reasonable for this kind of operation)
We discussed this in the stand-up this morning, I also found that pagination made a massive improvement on the results. DjangoLDP support for pagination already exists, but as collecting federated resources and filtering are currently provided by the front-end, pagination for performance has some major caveats. Obviously pulling in data from other servers would be slow as well
@balessan mentioned that we could have indexing servers and @matthieu raised the point about SPARQL/triplestore systems, which pull federated resources together on the query-level. I'm aware Mastodon completes federation server-side as well
As much as pagination would be a great improvement to our architecture, I think it's a totally different topic from performance. I think we should keep our focus on having acceptable rendering times. It looks like @balessan 's tests were far from it, and I doubt pagination is a solution to that.
Indexing and sparql interface sound like long term goals and I don't think they can be a solution in the short run.