fluffychat/lib/pangea/toolbar/message_selection_overlay.dart
ggurdin e8428783e6
Fluffychat merge 2 (#5590)
* build: Reenable shrink resources and minify in gradle

* build: (deps): bump image from 4.6.0 to 4.7.1

Bumps [image](https://github.com/brendan-duncan/image) from 4.6.0 to 4.7.1.
- [Changelog](https://github.com/brendan-duncan/image/blob/main/CHANGELOG.md)
- [Commits](https://github.com/brendan-duncan/image/commits)

---
updated-dependencies:
- dependency-name: image
  dependency-version: 4.7.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build: (deps): bump file_picker from 10.3.7 to 10.3.8

Bumps [file_picker](https://github.com/miguelpruivo/flutter_file_picker) from 10.3.7 to 10.3.8.
- [Release notes](https://github.com/miguelpruivo/flutter_file_picker/releases)
- [Changelog](https://github.com/miguelpruivo/flutter_file_picker/blob/master/CHANGELOG.md)
- [Commits](https://github.com/miguelpruivo/flutter_file_picker/compare/v10.3.7...v10.3.8)

---
updated-dependencies:
- dependency-name: file_picker
  dependency-version: 10.3.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* feat: Improved search

* build: Use matrix sdk vom pub.dev again

* chore: Follow up better search

* build: (deps): bump image from 4.7.1 to 4.7.2

Bumps [image](https://github.com/brendan-duncan/image) from 4.7.1 to 4.7.2.
- [Changelog](https://github.com/brendan-duncan/image/blob/main/CHANGELOG.md)
- [Commits](https://github.com/brendan-duncan/image/commits)

---
updated-dependencies:
- dependency-name: image
  dependency-version: 4.7.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Make cross signing self sign mandatory for bootstrap

* chore: Update user device keys before creating bootstrap

* fix: Better wait for secrets after verification bootstrap

* refactor: Remove native imaging and enable web worker

* refactor: Remove unused html onfocus streams

* build: (deps): bump flutter_foreground_task from 9.1.0 to 9.2.0

Bumps [flutter_foreground_task](https://github.com/Dev-hwang/flutter_foreground_task) from 9.1.0 to 9.2.0.
- [Changelog](https://github.com/Dev-hwang/flutter_foreground_task/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Dev-hwang/flutter_foreground_task/commits)

---
updated-dependencies:
- dependency-name: flutter_foreground_task
  dependency-version: 9.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(translations): Translated using Weblate (Uzbek)

Currently translated at 99.7% (823 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/uz/

* chore(translations): Translated using Weblate (Russian)

Currently translated at 99.8% (824 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/ru/

* chore(translations): Translated using Weblate (Norwegian Bokmål)

Currently translated at 90.9% (750 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/nb_NO/

* chore(translations): Translated using Weblate (Galician)

Currently translated at 100.0% (825 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/gl/

* chore(translations): Translated using Weblate (Basque)

Currently translated at 99.7% (823 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/eu/

* chore(translations): Translated using Weblate (Ukrainian)

Currently translated at 100.0% (825 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/uk/

* chore(translations): Translated using Weblate (Estonian)

Currently translated at 100.0% (825 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/et/

* chore(translations): Translated using Weblate (Dutch)

Currently translated at 100.0% (825 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/nl/

* chore(translations): Translated using Weblate (Russian)

Currently translated at 100.0% (825 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/ru/

* chore(translations): Translated using Weblate (Spanish)

Currently translated at 95.2% (788 of 827 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/es/

* chore(translations): Translated using Weblate (Spanish)

Currently translated at 96.3% (797 of 827 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/es/

* chore(translations): Translated using Weblate (Russian)

Currently translated at 100.0% (825 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/ru/

* chore(translations): Translated using Weblate (Russian)

Currently translated at 100.0% (825 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/ru/

* fix: Broken ruzzian plurals

* chore(translations): Translated using Weblate (Norwegian Bokmål)

Currently translated at 91.2% (753 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/nb_NO/

* chore(translations): Translated using Weblate (Bengali)

Currently translated at 4.5% (38 of 827 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/bn/

* chore(translations): Translated using Weblate (French)

Currently translated at 82.3% (679 of 825 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/fr/

* build: (deps): bump translations_cleaner from 0.0.5 to 0.1.0

Bumps [translations_cleaner](https://github.com/Chinmay-KB/translations_cleaner) from 0.0.5 to 0.1.0.
- [Changelog](https://github.com/Chinmay-KB/translations_cleaner/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Chinmay-KB/translations_cleaner/commits)

---
updated-dependencies:
- dependency-name: translations_cleaner
  dependency-version: 0.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(translations): Translated using Weblate (German)

Currently translated at 99.2% (821 of 827 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/de/

* chore(translations): Translated using Weblate (Estonian)

Currently translated at 100.0% (827 of 827 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/et/

* build: Bump version to 2.4.0

* build: (deps): bump sqflite_common_ffi from 2.3.6 to 2.3.7+1

Bumps [sqflite_common_ffi](https://github.com/tekartik/sqflite) from 2.3.6 to 2.3.7+1.
- [Commits](https://github.com/tekartik/sqflite/compare/sqflite_common_ffi_v2.3.6...sqflite_common_ffi/v2.3.7)

---
updated-dependencies:
- dependency-name: sqflite_common_ffi
  dependency-version: 2.3.7+1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(translations): Translated using Weblate (Czech)

Currently translated at 66.1% (547 of 827 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/cs/

* chore(translations): Translated using Weblate (Czech)

Currently translated at 72.7% (602 of 827 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/cs/

* chore(translations): Translated using Weblate (German)

Currently translated at 99.8% (826 of 827 strings)

Translation: FluffyChat/Translations
Translate-URL: https://hosted.weblate.org/projects/fluffychat/translations/de/

* chore: Add security.md file

* fix: Locale unlocalized strings

* build: (deps): bump matrix from 4.1.0 to 5.0.0

Bumps [matrix](https://github.com/famedly/matrix-dart-sdk) from 4.1.0 to 5.0.0.
- [Release notes](https://github.com/famedly/matrix-dart-sdk/releases)
- [Changelog](https://github.com/famedly/matrix-dart-sdk/blob/main/CHANGELOG.md)
- [Commits](https://github.com/famedly/matrix-dart-sdk/compare/v4.1.0...v5.0.0)

---
updated-dependencies:
- dependency-name: matrix
  dependency-version: 5.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: Notifications on web correctly managed when tab not focused

* chore: Add changelog for android

* chore: Remove duplicated localization

* fix: Sign in label

* chore: Versionize fcm shared isolate

* build: Remove unused packag

* build: (deps): bump package_info_plus from 8.3.1 to 9.0.0

Bumps [package_info_plus](https://github.com/fluttercommunity/plus_plugins/tree/main/packages/package_info_plus) from 8.3.1 to 9.0.0.
- [Release notes](https://github.com/fluttercommunity/plus_plugins/releases)
- [Commits](https://github.com/fluttercommunity/plus_plugins/commits/package_info_plus-v9.0.0/packages/package_info_plus)

---
updated-dependencies:
- dependency-name: package_info_plus
  dependency-version: 9.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* feat: Display particle animation on login page

* chore: Use fixed version of fcm shared isolate

* fix: apk crash on some platforms due new flutter version

* chore: Correct kotlin format

* fix iOS notifications

* fluffychat merge

* fluffychat merge

* fluffychat merge

* fluffychat merge

* fluffychat merge

* fluffychat merge

* add missing type annotations

* update matrix version

* fluffychat merge

* fluffychat merge

* fix notification on click actions

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Christian Kußowski <c.kussowski@famedly.com>
Co-authored-by: Krille-chan <christian-kussowski@posteo.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: BeMeritus <bemerituss@gmail.com>
Co-authored-by: Frank Paul Silye <frankps@gmail.com>
Co-authored-by: josé m. <correoxm@disroot.org>
Co-authored-by: xabirequejo <xabi.rn@gmail.com>
Co-authored-by: Ihor Hordiichuk <igor_ck@outlook.com>
Co-authored-by: Priit Jõerüüt <jrthwlate@users.noreply.hosted.weblate.org>
Co-authored-by: Jelv <post@jelv.nl>
Co-authored-by: Дмитрий Михирев <bizdelnick@gmail.com>
Co-authored-by: Kimby <kimbyqs@gmail.com>
Co-authored-by: Christian <christian-pauly@posteo.de>
Co-authored-by: Kom nake <kominak310@svcache.com>
Co-authored-by: hugues de keyzer <komputilisto@hugues.info>
Co-authored-by: nautilusx <translate@disroot.org>
Co-authored-by: Šebestová <ka.sebestova.cz@gmail.com>
2026-02-10 08:01:12 -05:00

327 lines
10 KiB
Dart

import 'dart:async';
import 'dart:developer';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:flutter/scheduler.dart';
import 'package:collection/collection.dart';
import 'package:matrix/matrix.dart' hide Result;
import 'package:fluffychat/config/app_config.dart';
import 'package:fluffychat/pages/chat/chat.dart';
import 'package:fluffychat/pangea/analytics_data/analytics_updater_mixin.dart';
import 'package:fluffychat/pangea/common/utils/error_handler.dart';
import 'package:fluffychat/pangea/events/event_wrappers/pangea_message_event.dart';
import 'package:fluffychat/pangea/events/event_wrappers/pangea_representation_event.dart';
import 'package:fluffychat/pangea/events/models/pangea_token_model.dart';
import 'package:fluffychat/pangea/events/models/pangea_token_text_model.dart';
import 'package:fluffychat/pangea/text_to_speech/text_to_speech_response_model.dart';
import 'package:fluffychat/pangea/text_to_speech/tts_controller.dart';
import 'package:fluffychat/pangea/toolbar/layout/message_selection_positioner.dart';
import 'package:fluffychat/pangea/toolbar/message_practice/practice_controller.dart';
import 'package:fluffychat/pangea/toolbar/reading_assistance/select_mode_buttons.dart';
import 'package:fluffychat/pangea/toolbar/reading_assistance/select_mode_controller.dart';
import 'package:fluffychat/pangea/toolbar/reading_assistance/tokens_util.dart';
import 'package:fluffychat/pangea/toolbar/token_rendering_mixin.dart';
import 'package:fluffychat/widgets/matrix.dart';
/// Controls data at the top level of the toolbar (mainly token / toolbar mode selection)
class MessageSelectionOverlay extends StatefulWidget {
final ChatController chatController;
final Event _event;
final Event? _nextEvent;
final Event? _prevEvent;
final PangeaToken? _initialSelectedToken;
final Timeline _timeline;
const MessageSelectionOverlay({
required this.chatController,
required Event event,
required PangeaToken? initialSelectedToken,
required Event? nextEvent,
required Event? prevEvent,
required Timeline timeline,
super.key,
}) : _initialSelectedToken = initialSelectedToken,
_nextEvent = nextEvent,
_prevEvent = prevEvent,
_event = event,
_timeline = timeline;
@override
MessageOverlayController createState() => MessageOverlayController();
}
class MessageOverlayController extends State<MessageSelectionOverlay>
with SingleTickerProviderStateMixin, AnalyticsUpdater, TokenRenderingMixin {
Event get event => widget._event;
PangeaTokenText? _selectedSpan;
ValueNotifier<PangeaToken?> selectedTokenNotifier = ValueNotifier(null);
List<PangeaTokenText>? _highlightedTokens;
double maxWidth = AppConfig.toolbarMinWidth;
late SelectModeController selectModeController;
ValueNotifier<SelectMode?> get selectedMode =>
selectModeController.selectedMode;
late PracticeController practiceController;
double? screenWidth;
/////////////////////////////////////
/// Lifecycle
/////////////////////////////////////
@override
void initState() {
super.initState();
selectModeController = SelectModeController(pangeaMessageEvent);
practiceController = PracticeController(pangeaMessageEvent);
_initializeTokensAndMode();
WidgetsBinding.instance.addPostFrameCallback(
(_) => widget.chatController.setSelectedEvent(event),
);
}
@override
void didChangeDependencies() {
super.didChangeDependencies();
final newWidth = MediaQuery.widthOf(context);
if (screenWidth != null && screenWidth != newWidth) {
widget.chatController.clearSelectedEvents();
return;
}
screenWidth = newWidth;
}
@override
void dispose() {
WidgetsBinding.instance.addPostFrameCallback(
(_) => widget.chatController.clearSelectedEvents(),
);
selectModeController.dispose();
practiceController.dispose();
selectedTokenNotifier.dispose();
super.dispose();
}
Future<void> _initializeTokensAndMode() async {
try {
if (pangeaMessageEvent.event.messageType != MessageTypes.Text) {
return;
}
RepresentationEvent? repEvent =
pangeaMessageEvent.messageDisplayRepresentation;
if (repEvent == null ||
(repEvent.event == null && repEvent.tokens == null)) {
repEvent = await _fetchNewRepEvent();
}
if (repEvent?.event != null) {
await repEvent!.requestTokens();
}
} catch (e, s) {
debugger(when: kDebugMode);
ErrorHandler.logError(
e: e,
s: s,
data: {"eventID": pangeaMessageEvent.eventId},
);
} finally {
_initializeSelectedToken();
if (mounted) setState(() {});
}
}
void _initializeSelectedToken() => widget._initialSelectedToken != null
? updateSelectedSpan(widget._initialSelectedToken!.text)
: null;
/////////////////////////////////////
/// State setting
/////////////////////////////////////
/// We need to check if the setState call is safe to call immediately
/// Kept getting the error: setState() or markNeedsBuild() called during build.
/// This is a workaround to prevent that error
@override
void setState(VoidCallback fn) {
// if (pangeaMessageEvent != null) {
// debugger(when: kDebugMode);
// modeLevel = toolbarMode.currentChoiceMode(this, pangeaMessageEvent!);
// } else {
// debugger(when: kDebugMode);
// }
final phase = SchedulerBinding.instance.schedulerPhase;
if (mounted &&
(phase == SchedulerPhase.idle ||
phase == SchedulerPhase.postFrameCallbacks)) {
// It's safe to call setState immediately
try {
super.setState(fn);
} catch (e, s) {
ErrorHandler.logError(
e: "Error calling setState in MessageSelectionOverlay: $e",
s: s,
data: {},
);
}
} else {
// Defer the setState call to after the current frame
WidgetsBinding.instance.addPostFrameCallback((_) {
try {
if (mounted) super.setState(fn);
} catch (e, s) {
ErrorHandler.logError(
e: "Error calling setState in MessageSelectionOverlay after postframeCallback: $e",
s: s,
data: {},
);
}
});
}
}
/// Update [selectedSpan]
void updateSelectedSpan(PangeaTokenText? selectedSpan) {
if (MatrixState.pangeaController.subscriptionController.isSubscribed ==
false) {
return;
}
if (selectedSpan == _selectedSpan) {
selectModeController.setPlayingToken(selectedToken?.text);
return;
}
_selectedSpan = selectedSpan;
selectedTokenNotifier.value = selectedToken;
selectModeController.setPlayingToken(selectedToken?.text);
if (selectedToken != null &&
selectModeController.selectedMode.value != SelectMode.audio) {
TtsController.tryToSpeak(
selectedToken!.text.content,
langCode: pangeaMessageEvent.messageDisplayLangCode,
);
}
if (!mounted) return;
if (selectedToken != null && isNewToken(selectedToken!)) {
final token = selectedToken!;
collectNewToken(
event.eventId,
"word-zoom-card-${token.text.uniqueKey}",
token,
Matrix.of(context).analyticsDataService,
roomId: event.room.id,
eventId: event.eventId,
).then((_) {
if (mounted) setState(() {});
});
return;
}
setState(() {});
}
PangeaMessageEvent get pangeaMessageEvent => PangeaMessageEvent(
event: widget._event,
timeline: widget._timeline,
ownMessage: widget._event.room.client.userID == widget._event.senderId,
);
PangeaToken? get selectedToken {
if (pangeaMessageEvent.isAudioMessage == true) {
final stt = pangeaMessageEvent.getSpeechToTextLocal();
if (stt == null || stt.transcript.sttTokens.isEmpty) return null;
return stt.transcript.sttTokens
.firstWhereOrNull((t) => isTokenSelected(t.token))
?.token;
}
return pangeaMessageEvent.messageDisplayRepresentation?.tokens
?.firstWhereOrNull(isTokenSelected);
}
/// If sentence TTS is playing a word, highlight that word in message overlay
void highlightCurrentText(int currentPosition, List<TTSToken> ttsTokens) {
final List<TTSToken> textToSelect = [];
// Check if current time is between start and end times of tokens
for (final TTSToken token in ttsTokens) {
if (token.endMS > currentPosition) {
if (token.startMS < currentPosition) {
textToSelect.add(token);
} else {
break;
}
}
}
if (const ListEquality().equals(textToSelect, _highlightedTokens)) return;
_highlightedTokens = textToSelect.isEmpty
? null
: textToSelect.map((t) => t.text).toList();
setState(() {});
}
Future<RepresentationEvent?> _fetchNewRepEvent() async {
final RepresentationEvent? repEvent =
pangeaMessageEvent.messageDisplayRepresentation;
if (repEvent != null) return repEvent;
final eventID = await pangeaMessageEvent
.requestRepresentationByDetectedLanguage();
if (eventID == null) return null;
final event = await widget._event.room.getEventById(eventID);
if (event == null) return null;
return RepresentationEvent(
timeline: pangeaMessageEvent.timeline,
parentMessageEvent: pangeaMessageEvent.event,
event: event,
);
}
void onClickOverlayMessageToken(PangeaToken token) =>
updateSelectedSpan(token.text);
/// Whether the given token is currently selected or highlighted
bool isTokenSelected(PangeaToken token) {
final isSelected =
_selectedSpan?.offset == token.text.offset &&
_selectedSpan?.length == token.text.length;
return isSelected;
}
bool isNewToken(PangeaToken token) =>
TokensUtil.isNewTokenByEvent(token, pangeaMessageEvent);
bool isTokenHighlighted(PangeaToken token) {
if (_highlightedTokens == null) return false;
return _highlightedTokens!.any(
(t) => t.offset == token.text.offset && t.length == token.text.length,
);
}
String tokenEmojiPopupKey(PangeaToken token) =>
"${token.uniqueId}_${event.eventId}_emoji_button";
@override
Widget build(BuildContext context) {
return MessageSelectionPositioner(
overlayController: this,
chatController: widget.chatController,
event: widget._event,
nextEvent: widget._nextEvent,
prevEvent: widget._prevEvent,
initialSelectedToken: widget._initialSelectedToken,
);
}
}