Chat history loss (archived messages)

Many one-to-one conversations history disappear again on Hubl. Here is one:

--

Chat history is still here, this issue is due to the performances issue, making converse unable to properly show the use history.

@jbpasquier @MattJ @XaviFP This should be panic on Earth :-)

I agreed with @jbpasquier to only work on estimated and approved issues. I'm not sure what to estimate for an issue like this without investigating it. What is the expected process for this?

Usually, provide an estimation for the investigation then for the resolution (or not, if you solve it while investigating).

On this case, flagged as bug-s1 (Unsure why?), you are able to rush on it without any delay or budget validation.

@MattJ @XaviFP Yes, sorry I thought it was clear but bug-s1 means we'd finance it because it is critical.

@jbpasquier Well... people can't talk with each other anymore when they lose their history, the messages they were supposed to read are not displayed, therefore the app literally ceases to function for them.

@alexbourlier From the issue, it stands for an history loss on one one-to-one conversation, I don't experience anything similar on my side. I have no clue if it impact other people or even if your contact also face it. Standing on that, I don't feel this issue as an s1, but you and @rachel are masters on this. :-)

He he, he doesn't trust me :-D When I say Many, I mean many.

Bérangère, Alice, Sylvain and myself are experiencing the problem, and that's only those who bothered reporting it to me.

Seems there has been a change in s3 configuration/usage for avatars:

GEThttps://happy-dev-rc.s3.eu-west-3.amazonaws.com/uvNHjPPBTpibWZwQJ/avatars/agLhdySm8rxDo3Bvy NoSuchBucketThe specified bucket does not existhappy-dev-rc95BEE3208E319003n7ZRZt6hL6erbIRbbmcgey/csiQTeBlnkSr49RyxKnFVRtfa1oUBYuX0qk+6ulc/ypwuiItukKY=

This might lead to errors while fetching those in the backend and consequently generating those 404 to the GET requests to https://api.community.startinblox.com/circles/XXX/

In case it helps I'm getting {"detail":"Not found.","@context":"https://cdn.happy-dev.fr/owl/hdcontext.jsonld"} on GET request to https://api.community.startinblox.com/circles/14/

Console errors are unrelated, as avatars on chat are served inline by the vcard.

I think I would hesitate to put it as s1, but as @alexbourlier is the master of budget I trust he is using s1 carefuly :) That being said, unless checking all new issues, there is no way any member of the team can currently know when an s1 is created. What about adding to the process that people get pinged? At least maybe JB + myself? So we can estimate if we need to communicate it more widlely?

Agreed

Ok, had a quick look into this just now at the request of @jbpasquier who is also experiencing the issue.

The good news is that history is there - Alex's archive spans from April to today, so it's not a question of it being lost or missing.

@jbpasquier reported the issue to me at 18:35 UTC, but I see no history queries from jbpasquier@community.startinblox.com since 14:51 UTC. That query returned 30 results from Prosody to Converse.js.

It again seems to me that there is some problem with Converse.js not reliably fetching history. We know one bug is when there are no displayable messages in the results (we have reduced many cases of this, but I can't guarantee it will never happen - undisplayable messages are a natural part of XMPP to ensure e.g. reliable delivery reports).

@XaviFP is this something you can reproduce, can you see if Converse.js is correctly querying when it should, displaying the result and requesting further results when necessary?

I don't know if it helps, but Bérengère from https://app.nouvelle-aquitaine.happy-dev.fr/ said today that she somehow recovered all her chat history with Sylvain from https://app.paris.happy-dev.fr/, which had disappeared for several days.

She didn't do anything special as far as she knows, it just magically reappared according to her.

Another example of history disappearance on https://community.startinblox.com/

@jbpasquier @XaviFP Where are we on this one guys?

I have the feeling we are unable to solve any bug in less than 1 or 2 months on the chat component. The same is happening with #161 (closed), and we are not releasing any feature requests in the meantime. That's not acceptable.

I understand we needed to update ConverseJS, I understand there are performance issues, I understand personal emergencies and there might be other things that I'm not aware of, but we can't afford to solve critical bugs in months, this needs to improve and by a significant factor. We are losing our best advocates of our federated app project because they doubt we'll be able to have this thing working one day. They kep experiencing the same bugs and they stop believing me we'll fix them one day. We won't go anywhere if we don't keep them on board, and I start to worry a lot that we will.

Can we have a call to discuss this? I'm available anytime, any day.

We need to be faster at solving bugs on the chat component otherwise we just won't make it through 2021.

Related to #211 (closed)

@jbpasquier I dared to switch this one back as a bug-s1, cf. the labels description. This is a critical broken feature with no workaround. I can't talk to @Cyrilthiriet anymore on Hubl and have to default to Whatsapp for doing so.

An F5 should clear your storage and restore your history, if it does not then it's a new issue.

In case it does not, could you try:

• Spam Cyril with new messages, at least enough to be able to scroll.
• Scroll up, then down, then up.
• Do you see your old messages back?

Ping @XaviFP

@jbpasquier Reloading or logging out and back in doesn't solve the problem.

If I scroll up to trigger the fetching of previous history, it doesn't bring it back. I'm only accessing new messages on the client side.

Worth a new issue then?

Definitely

@XaviFP Any idea of what could happen here?

Actually, I had created one already : #216 (closed)

Ahah, this one was closed in favor of the one we're at.

Ok, let's keep both open.

#199 -> Performance related

#216 (closed) -> ? History fetching issue, maybe. Need debugging.

It could be related to the cause of #193 (closed). There are some similar behaviors like the need of re-scrolling, after the last fix to avoid Converse from scrolling down after a history fetch.

Need to investigate further, but all these issues seem related to me. Or at least I believe are part of the same chain reaction.

marked #202 (closed) as a duplicate of this issue

mentioned in issue #202 (closed)

changed the description

mentioned in issue #216 (closed)

mentioned in issue #220 (closed)

changed milestone to %Xmpp Cyril

assigned to @manv

mentioned in issue #264 (closed)

mentioned in issue #196 (closed)

mentioned in issue #242 (closed)