Switch to Converse in headless mode

This discussion thread was created to discuss this extract from an email I sent

In my opinion, Prosody and ConverseJS are the correct choices for a scalable and user friendly chat solution that is required for SiB App. However, after working on this project for a few months, fixing a few issues and getting to understand the overall chat/group-chat architecture, it is my humble opinion that we are not using the full potential ConverseJS and inherently creating a bottleneck that is responsible for some of the issues I have seen.

When you have web front-end team and you are just need a JavaScript library with a good API to handle the XMPP plumbing, then you should use ConverseJS in headless mode. Use plugins to handle any custom extensions you add to XMPP.

IF you are happy with the ConverseJS UI (based on Backbone) and want to reuse the views, then you should set an appropriate view mode and use plugins to customise the views as well as handle custom extensions you add to XMPP.

The view mode is very important it determines the chat use case you are trying to implement. ConverseJS offers the following :

• Overlay: a full chat and group-chat overlayed on a web page in the style of Facebook. To be used when chat is supporting something else like social media, public web site or a web application.

• Full-screen: a full chat and group-chat in a standalone window/browser tab in the style of Slack

• Embedded: to be used for a single chat or a groupchat on a web page between a web page visitor and an agent or support group somewhere else using ConverseJS in overlay or full-screen mode.

The current design of SiB App is using ConverseJS in embedded view mode. The implication is that chat and group-chats are handled like a modal dialog. Only one can be opened and in focus. All activity to others is suspended until user selects them.

The constant opening and closing the chat views, the overhead of fetching chat history several times as well as making users leave/join chat rooms does have a performance hit on the overall user experience and will be a major bottleneck to scaling the application up into many users.

If we can afford a redesign now, then I suggest we choose between ConverseJS in headless mode or Converse with the full-screen mode.

With headless, SiB App is responsible for all the front-end UI. We would have to implement Converse chatbox view as an internal chat component is the same style.

With full-screen, we would have to develop a plugin to override the behaviour of ConverseJS views and a provide a CSS file to re-skin the views to the look and feel of SiB App.

Response to the following question

Could you please list the issues you think it would solve?

The main issue it will solve is performance. As mentioned in my post, the re-loading of chat history every time a user switched between chat/group-chat conversations has an impact on the server, network and client. It is more efficient to load chat history once at startup and add to the chat history with incremental chat meaasages.

Issues around unread messages, missing chat history should be easier to deal with if users remain in a chat room until they leave the web app or explicitly leave the room.

More importantly is the fact that the embedded mode of Converse was designed to handle a single open chatbox or chatroom view. Events occuring in other chats or groupchats on the SiB App UI could be missed.

Response to this question

My fear is that we scramble in that direction just because things don't work without anybody knowing why at the moment, and we end up having a lesser version of what we currently have with no improvement regarding performance.

Moving from enbedded mode to headless is a big move. It is like moving your familiy to a new neighborhood because a new child is on the way and your 3 bedroom house is not big enough :-)

SiB App will become responsible for ALL the UI aspects of chat. Converse will only give you a very rich high level API for handling the XMPP messages.

In any case, please don't take my word for it. Confirm this with @jcbrand

Response to this comment

Do keep in mind that the average user of the app will belong to ~100 MUC. The singleton pattern doesn't sound stupid to me, if that is what we are trying to avoid.

I am interpreting this to mean that the user base will be in the region of a 100 users per chat room. If so then providing that the users will spend most of their time in a single chat room with little or no private chat with each other, then I agree with you, embedded mode is perfect for this use case.

That is what we do at igniterealtime.org. Everything happens in a single chat room called open_chat

Thank you for this explanation. The idea would be to leave the MUC connections open for all chatrooms then? My fear is that this won't be sustainable for 100+ MUC, which our users will have. Any thoughts on this? It might not be what you have in mind though when you say It is more efficient to load chat history once at startup and add to the chat history with incremental chat meaasages. If you can reassure me a bit here, that's cool.

unread messages are normally dealt with outside of ConverseJS. I believe @jbpasquier and @sylvain came up with a solution validated by @MattJ if I am correct. I let them enlighten us.

Events occuring in other chats or groupchats on the SiB App UI could be missed: Such as? :-)

By Alexandre on 2020-02-13T15:07:35 (imported from GitLab project)

Ah ah ah, if that message was supposed to get me worried about the move: job done! :-D

3 kids?! I hope @jcbrand confirm headless is not that bad... :-D

By Alexandre on 2020-02-13T15:09:54 (imported from GitLab project)

Hmmm... I think I really meant that each of our users will be part of ~100 MUC. Another way to say it if you will is that the user will have ~100 MUC on her left hand side menu. The biggest MUC will accommodate ~500 users.

That's why I tend to think the singleton is smart, because MUCs will be quite active and expensive to keep open I think, and having only one open at the time sounds efficient. Maybe we can optimize the history of "already open MUCs" by storing it on the client side and fetching only the missing part of it when accessing it again. Maybe that is what you have in mind when you think about the headless mode. I must say I'm sceptical this will make a big difference in terms of performance, and I also don't believe it will avoid the history problems we are facing but... I'm actually ready to trust your advice and @jcbrand's on this. We will do what you guys recommand. I'd just like to make sure we really think things through before diving in this adventure.

By Alexandre on 2020-02-13T15:16:40 (imported from GitLab project)

I'm not really expert in the innards of converse.js, but if by "singleton" the XMPP connection is being started from scratch on each navigation, that's a bit weird.

But there is not right/wrong answer here - one is more efficient if the user mostly stays in a single MUC, the other is more efficient if they switch around a lot.

Our users do indeed switch around a lot. We keep on switching every 30s I'd say when we are on the chat.

By Alexandre on 2020-02-13T15:21:36 (imported from GitLab project)

In my personal experience of using e.g. Slack, I also switch around a lot.

My "XMPP expert" instinct here would be also to keep the connection alive and simply remain joined to every room. There are many options for optimisation if any issues ever showed up (like just staying connected to the last 10 circles viewed, for example).

But I don't think you have to do it this way right now. It depends how comfortable you folk are with developing a chat UI from scratch if you go with headless mode. I don't get involved in web dev these days, but I know I wouldn't want that task :)

However I also defer to Dele + JC's advice when it comes to the best way to use Converse.

According to Dele's comment, the primary reason to switch is to gain performance (presumably on switching chats). So I think you would need to compare today's performance against the development cost of switching everything to use the headless view mode.

Roger. Thanks for this comment. It helps me to think clearly about what our options are.

I think the biggest pains we are facing are:

Maybe headless would solve them all but it kind of feels like black magic to me. @dele, @jcbrand your expertise will settle the question I think.

By Alexandre on 2020-02-13T15:55:02 (imported from GitLab project)

but if by "singleton" the XMPP connection is being started from scratch on each navigation, that's a bit weird.

No. Connection is persistent. When a user recieves a notification of a new message and clicks on the circle or contact, the current chat/groupchat is closed, the new one opened and history fetched from server. If the user then returns to the previous conversation, the new one is closed and chat history for the previous is refetched.

"singleton" is a flag for Converse to enforce a policy for a modal single conversation for the connection that is auto-joined after the connection is made. It is mainly used to provide the "visitor chat side" of chat support for web site vistors.

Actually, @jbpasquier will have clearer ideas than me, but I think the unread system is actually based on converse.

By Sylvain Le Bon on 2020-02-13T16:29:53 (imported from GitLab project)

mentioned in issue #130 (closed)

By Jean-Baptiste Pasquier on 2020-02-18T16:28:09 (imported from GitLab project)

Closed in favor of https://git.happy-dev.fr/startinblox/components/sib-chat/issues/130

By Jean-Baptiste Pasquier on 2020-02-19T16:30:36 (imported from GitLab project)

closed

By Jean-Baptiste Pasquier on 2020-02-19T16:30:37 (imported from GitLab project)

Switch to Converse in headless mode

Designs

Child items ...

Activity