Keep track of the known topics in store

[sm-sayedi]

#1508 but rebased on top of main with the review of @gnprice addressed.

(Thanks @PIG208 for all of your previous work in #1508)

Fixes: #1499

[chrisbobbe]

Thanks! Glad to be maintaining a data structure for this. 🙂

Comments below, from reading the implementation in the first commit (I haven't yet read the tests):

d6b2242 channel: Keep track of channel topics, and keep up-to-date with events

[chrisbobbe]

nit: too-long line; put comment on line above

[chrisbobbe]

Suggested change

	/// Pairs of the known topics and its latest message ID, in the given channel.
	/// The topics in the given channel, along with their latest message ID.

[chrisbobbe]

nit:

Suggested change

	/// Returns null if the data has never been fetched yet.
	/// Returns null if the data has not been fetched yet.

[chrisbobbe]

nit, omit needless words:

Suggested change

	/// The result is guaranteed to be sorted by [GetStreamTopicsEntry.maxId]
	/// descending, and the topics are guaranteed to be distinct.
	/// The result is sorted by [GetStreamTopicsEntry.maxId] descending,
	/// and the topics are distinct.

[chrisbobbe]

It seems worth highlighting in general that maxId might be…incorrect, basically. (Not just that two topics might have the same maxId.) Maybe something like:

(Also, isn't message deletion another reason maxId could be wrong?)

Suggested change

	/// In some cases, the same maxId affected by message moves can be present in
	/// multiple [GetStreamTopicsEntry] entries. For this reason, the caller
	/// should not rely on [getChannelTopics] to determine which topic the message
	/// is in. Instead, refer to [PerAccountStore.messages].
	/// See [handleUpdateMessageEvent] on how this could happen.
	/// Occasionally, [GetStreamTopicsEntry.maxId] will refer to a message
	/// that doesn't exist or is no longer in the topic.
	/// This happens when a topic's latest message is moved or deleted
	/// and we don't have enough information
	/// to replace [GetStreamTopicsEntry.maxId] accurately.
	/// (We don't keep a snapshot of all messages.)
	/// Use [PerAccountStore.messages] to check a message's topic accurately.

[chrisbobbe]

nit:

Suggested change

	final StreamMessage(:streamId, :topic) = event.message as StreamMessage;
	final StreamMessage(streamId: channelId, :topic) = event.message as StreamMessage;

[chrisbobbe]

I'm getting confused trying to parse this sentence. Instead, how about, inside the if just below this, say something like:

// The event raced with a message fetch.

[chrisbobbe]

How about:

Suggested change

	// If we don't already know about the list of topics of the channel this
	// message belongs to, we don't want to proceed and put one entry about the
	// topic of this message, otherwise [fetchTopics] and the callers of
	// [getChannelTopics] would assume that the channel only has this one topic
	// and would never fetch the complete list of topics for that matter.
	if (latestMessageIdsByTopic == null) return false;
	if (latestMessageIdsByTopic == null) {
	// We're not tracking this channel's topics yet.
	// We start doing that when [fetchTopics] is called,
	// and we fill in all the topics at that time.
	return false;
	}

[chrisbobbe]

With a switch/case on propagateMode, I think we can be less verbose in the comment, e.g.:

Suggested change

	final origLatestMessageIdsByTopics = _latestMessageIdsByChannelTopic[origStreamId];
	// We only handle the case where all the messages of [origTopic] are
	// moved to [newTopic]; in that case we can remove [origTopic] safely.
	// But if only one messsage is moved (`PropagateMode.changeOne`) or a few
	// messages are moved (`PropagateMode.changeLater`), we cannot do anything
	// about [origTopic] here as we cannot determine the new `maxId` for it.
	// (This is the case where there could be multiple channel-topic keys with
	// the same `maxId`)
	if (propagateMode == PropagateMode.changeAll
	&& origLatestMessageIdsByTopics != null) {
	shouldNotify = origLatestMessageIdsByTopics.remove(origTopic) != null;
	}
	final origLatestMessageIdsByTopics = _latestMessageIdsByChannelTopic[origStreamId];
	switch (propagateMode) {
	case PropagateMode.changeOne:
	case PropagateMode.changeLater:
	// We can't know the new `maxId` for the original topic.
	// Shrug; leave it unchanged. (See dartdoc of [getChannelTopics],
	// where we call out this possibility that `maxId` is incorrect.
	break;
	case PropagateMode.changeAll:
	if (origLatestMessageIdsByTopics != null) {
	origLatestMessageIdsByTopics.remove(origTopic);
	shouldNotify = true;
	}
	}

[chrisbobbe]

A bit easier to read, I think:

Suggested change

	final newLatestMessageIdsByTopics = _latestMessageIdsByChannelTopic[newStreamId];
	if (newLatestMessageIdsByTopics != null) {
	final movedMaxId = event.messageIds.max;
	if (!newLatestMessageIdsByTopics.containsKey(newTopic)
	|| newLatestMessageIdsByTopics[newTopic]! < movedMaxId) {
	newLatestMessageIdsByTopics[newTopic] = movedMaxId;
	shouldNotify = true;
	}
	}
	final newLatestMessageIdsByTopics = _latestMessageIdsByChannelTopic[newStreamId];
	if (newLatestMessageIdsByTopics != null) {
	final movedMaxId = event.messageIds.max;
	final currentMaxId = newLatestMessageIdsByTopics[newTopic];
	if (currentMaxId == null || currentMaxId < movedMaxId) {
	newLatestMessageIdsByTopics[newTopic] = movedMaxId;
	shouldNotify = true;
	}
	}

[sm-sayedi]

Thanks @chrisbobbe for the review. Pushed new revision.