feat: fetch languages directly from CMS (#5764)

* feat: fetch languages directly from CMS

- Switch language_repo.dart to fetch from CMS REST API (public, no auth)
- Parse CMS paginated response format (docs[] envelope)
- Rename getLanguages URL to cmsLanguages in urls.dart
- Add 15 unit tests for CMS response parsing
- Add design docs: course-plans, layout instructions

* formatting

---------

Co-authored-by: ggurdin <ggurdin@gmail.com>
This commit is contained in:
wcjord 2026-02-23 10:21:10 -05:00 committed by GitHub
parent 9ada5f6748
commit 859cb78339
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 541 additions and 11 deletions

View file

@ -0,0 +1,161 @@
---
applyTo: "lib/pangea/course_plans/**,lib/pangea/course_creation/**,lib/pangea/course_chats/**"
---
# Course Plans — Client Design
Client-side loading, caching, and display of localized course content (plans, topics, activities, locations, media).
- **Choreo design doc**: [course-localization.instructions.md](../../../2-step-choreographer/.github/instructions/course-localization.instructions.md)
- **Activity system**: [conversation-activities.instructions.md](conversation-activities.instructions.md)
---
## 1. Data Flow Overview
```
CoursePlanModel (fetched first)
├── topicIds: List<String> ─── fetchTopics() ───► CourseTopicModel[]
│ ├── activityIds ─── fetchActivities() ► ActivityPlanModel[]
│ └── locationIds ─── fetchLocations() ─► CourseLocationModel[]
└── mediaIds: List<String> ─── fetchMediaUrls() ► CourseMediaResponse
```
Each level returns **IDs only**; the next level is fetched in a separate HTTP call. This is intentional — each level has its own cache and can be loaded independently.
---
## 2. Loading Pattern — `CoursePlanProvider` Mixin
[`CoursePlanProvider`](../../lib/pangea/course_plans/courses/course_plan_builder.dart) is a mixin used by course pages (`CourseChatsPage`, `SelectedCoursePage`, `PublicCoursePreview`). It orchestrates the multi-step loading:
| Method | What it fetches | When called |
|--------|----------------|-------------|
| `loadCourse(courseId)` | Course plan + media URLs | On page init |
| `loadTopics()` | ALL topics for the course + location media | Immediately after `loadCourse` completes |
| `loadActivity(topicId)` | Activities for ONE topic | On-demand (user taps a topic) |
| `loadAllActivities()` | Activities for ALL topics in parallel | Called from `chat_details.dart` on the course settings tab — **this is the performance bottleneck** |
### Current loading sequences
**`CourseChatsPage`** (main course view):
```
1. loadCourse(id) → POST /choreo/course_plans/localize
2. loadTopics() → POST /choreo/topics/localize (ALL topic IDs)
3. _loadTopicsMedia() → location media for each topic
```
Activities are NOT loaded here. They display only in the course settings tab.
**`ChatDetailsController._loadCourseInfo()`** (course settings/details tab):
```
1. loadCourse(id)
2. loadTopics()
3. loadAllActivities() ← loads ALL activities for ALL topics at once
```
This is the call path that creates CMS pressure. See [course-localization.instructions.md §5](../../../2-step-choreographer/.github/instructions/course-localization.instructions.md) for the full analysis.
---
## 3. Repos & Caching
Each content type has its own repo with `GetStorage`-backed caching (1-day TTL):
| Repo | Storage Key | Cache Check |
|------|------------|-------------|
| [`CoursePlansRepo`](../../lib/pangea/course_plans/courses/course_plans_repo.dart) | `"course_storage"``"${uuid}_${l1}"` | Single course only (TODO: batch) |
| [`CourseTopicRepo`](../../lib/pangea/course_plans/course_topics/course_topic_repo.dart) | `"course_topic_storage"``"${uuid}_${l1}"` | Checks cache first, fetches only missing topics |
| [`CourseActivityRepo`](../../lib/pangea/course_plans/course_activities/course_activity_repo.dart) | `"course_activity_storage"``"${id}_${l1}"` | Checks cache first, fetches only missing activities |
| `CourseLocationRepo` | `"course_location_storage"` | Locations for topics |
| `CourseLocationMediaRepo` | `"course_location_media_storage"` | Media URLs for locations |
| `CourseMediaRepo` | `"course_media_storage"` | Media URLs for course-level images |
### Deduplication
All repos use a `Completer`-based in-flight cache (`Map<String, Completer<T>>`) keyed by batch ID to prevent duplicate concurrent requests for the same data.
### Cache invalidation
`CoursePlansRepo.clearCache()` clears ALL six storage boxes. This is called when the global 1-day TTL expires (checked on next `get()`).
---
## 4. Models
### `CoursePlanModel`
Returned by `CoursePlansRepo.get()`. Contains:
- Course metadata (`title`, `description`, `targetLanguage`, `cefrLevel`, etc.)
- `topicIds: List<String>` — IDs only, not full topic objects
- `mediaIds: List<String>` — IDs only
- `loadedTopics` — reads from `CourseTopicRepo` cache synchronously
- `fetchTopics()` — async, calls choreo API for missing topics
### `CourseTopicModel`
Returned by `CourseTopicRepo.get()`. Contains:
- Topic metadata (`title`, `description`, `uuid`)
- `activityIds: List<String>` — IDs only
- `locationIds: List<String>` — IDs only
- `loadedActivities` — reads from `CourseActivityRepo` cache synchronously
- `fetchActivities()` — async, calls choreo API for missing activities
### `ActivityPlanModel`
Returned by `CourseActivityRepo.get()`. Full activity plan with all fields (title, description, learning_objective, instructions, topic, objective, roles, mode, etc.).
---
## 5. Key API Endpoints
| Client Call | Choreo Endpoint | What's Sent |
|-------------|----------------|-------------|
| `CoursePlansRepo.get()` | `POST /choreo/course_plans/localize` | 1 course ID + L1 |
| `CoursePlanModel.fetchTopics()` | `POST /choreo/topics/localize` | ALL topic IDs for the course + L1 |
| `CourseTopicModel.fetchActivities()` | `POST /choreo/activity_plan/localize` | ALL activity IDs for one topic + L1 |
---
## 6. Pages That Load Courses
| Page | Loading Behavior |
|------|-----------------|
| [`CourseChatsPage`](../../lib/pangea/course_chats/course_chats_page.dart) | `loadCourse()` then `loadTopics()` — shows course with topic list. No activities loaded. |
| [`ChatDetailsController`](../../lib/pages/chat_details/chat_details.dart) | `loadCourse()` then `loadTopics()` then **`loadAllActivities()`** — the only call site that eagerly loads all activities |
| [`SelectedCoursePage`](../../lib/pangea/course_creation/selected_course_page.dart) | `loadCourse()` then `loadTopics()` — course detail/preview |
| [`PublicCoursePreview`](../../lib/pangea/course_creation/public_course_preview.dart) | `loadCourse()` then `loadTopics()` — browsing public courses |
---
## 7. Topic Unlock Logic — `ActivitySummariesProvider`
[`ActivitySummariesProvider`](../../lib/pangea/course_plans/course_activities/activity_summaries_provider.dart) is a mixin that determines which topic is "active" for a user. Topics are sequential — a user must complete activities in topic N before topic N+1 unlocks.
### `currentTopicId()` logic
Iterates topics in order. For each topic:
1. Checks `activityListComplete` — requires ALL activities for the topic to be loaded
2. Calls `_hasCompletedTopic()` — checks if the user has archived enough activity sessions
3. Returns the first incomplete topic
### `_hasCompletedTopic()` dependency on activity data
The unlock heuristic counts "two-person activities" via `topic.loadedActivities.values.where((a) => a.req.numberOfParticipants <= 2).length`. This is the only reason all activities need to be loaded for unlock computation. If `numberOfParticipants` were available without loading full activity objects, `loadAllActivities()` could be eliminated from the page load path.
---
## 8. UI Display Pattern — `CourseSettings`
[`CourseSettings`](../../lib/pangea/course_settings/course_settings.dart) renders the course details tab:
- **All topics are displayed as a vertical list** — title, location pin, participant avatars
- **Locked topics** are dimmed with a lock icon — activities are NOT shown for locked topics
- **Unlocked topics** show activities in a **horizontal scrollable `TopicActivitiesList`** per topic
- **Activities** are rendered as `ActivitySuggestionCard` widgets
This means even for unlocked topics, only a subset of activity cards are visible at any given time (horizontal scroll). Loading all activities across all topics upfront is unnecessary for display purposes.
---
## Future Work
_(No linked issues yet.)_

View file

@ -0,0 +1,148 @@
# Client Layout System
Applies to: `lib/config/themes.dart`, `lib/widgets/layouts/**`, `lib/pangea/spaces/space_navigation_column.dart`, `lib/widgets/navigation_rail.dart`, `lib/config/routes.dart`
## Overview
The app uses a responsive two-column layout inherited from FluffyChat. On wide screens (desktop/tablet landscape), both columns are visible simultaneously. On narrow screens (mobile), only one column shows at a time and GoRouter handles full-screen navigation between them.
---
## Breakpoints
All breakpoint logic lives in [`FluffyThemes`](../../lib/config/themes.dart):
| Mode | Condition | Result |
|------|-----------|--------|
| **Column mode** (two-column) | `width > columnWidth * 2 + navRailWidth` = **~833px** | Nav rail + left column + right column all visible |
| **Single-column** (mobile) | `width ≤ 833px` | One screen at a time; nav rail may or may not show depending on route |
| **Three-column** | `width > columnWidth * 3.5` = **~1330px** | Used sparingly (chat search panel, toolbar positioning) |
Constants:
- `columnWidth` = **380px** — width of the left column (chat list, analytics, settings)
- `navRailWidth` = **72px** (Pangea override; upstream FluffyChat uses 80px)
---
## Layout Structure
### Wide Screen (column mode)
```
┌─────────┬──────────────────┬──────────────────────────────┐
│ Nav Rail │ Left Column │ Right Column │
│ (72px) │ (380px) │ (remaining width) │
│ │ │ │
│ [Avatar] │ Chat list │ Chat / Settings detail / │
│ [Chats] │ or Analytics │ Room / Empty page │
│ [Space1] │ or Settings │ │
│ [Space2] │ or Course bear │ │
│ [Course] │ │ │
│ [Gear] │ │ │
└─────────┴──────────────────┴──────────────────────────────┘
```
### Narrow Screen (mobile)
```
┌──────────────────────────────┐
│ [Nav Rail] (smaller, 64px) │ ← shown on some screens
├──────────────────────────────┤
│ │
│ Full-screen view │
│ (chat list OR chat OR │
│ settings OR analytics) │
│ │
└──────────────────────────────┘
```
On mobile, the nav rail visibility is conditional — it shows on "root" screens (chat list, analytics, settings, course finder) but hides when you're inside a chat room, a specific space, or certain creation flows (`newcourse`, `:construct`). This logic is in [`TwoColumnLayout.build()`](../../lib/widgets/layouts/two_column_layout.dart).
---
## Key Components
### [`TwoColumnLayout`](../../lib/widgets/layouts/two_column_layout.dart)
The GoRouter `ShellRoute` builder wraps **all** authenticated routes in `TwoColumnLayout`. This widget is always rendered — it's not conditionally swapped out. It uses a `Stack` with `Positioned.fill`:
- **`SpaceNavigationColumn`** is positioned on the left (nav rail + optional left column)
- **`sideView`** (the GoRouter child) fills the remaining space to the right
The `columnWidth` calculation determines the left inset:
- Column mode: `navRailWidth + 1 + columnWidth + 1` ≈ **454px**
- Mobile with rail: `navRailWidth + 1` ≈ **73px**
- Mobile without rail: **0px** (sideView fills the entire screen)
### [`SpaceNavigationColumn`](../../lib/pangea/spaces/space_navigation_column.dart)
A `StatefulWidget` that composes:
1. **Left column content** (`_MainView`) — only rendered in column mode. Shows different content based on the current route path:
- Default / no special path → `ChatList` (with `activeChat` and `activeSpace` params)
- Path contains `analytics``ConstructAnalyticsView` or `LevelAnalyticsDetailsContent` or `ActivityArchive`
- Path contains `settings``Settings`
- Path contains `course` → decorative bear image (placeholder)
2. **`SpacesNavigationRail`** — the narrow icon column. Shows when `showNavRail` is true.
The column has hover-expand behavior (desktop): hovering for 200ms expands the rail to show labels next to icons (~250px wide), collapsing when the mouse leaves.
### [`SpacesNavigationRail`](../../lib/widgets/navigation_rail.dart)
The vertical icon strip. Items top-to-bottom:
1. **User avatar** → navigates to analytics
2. **Chat icon** → navigates to `/rooms` (all chats)
3. **Space icons** — one per joined space, rendered with `MapClipper` shape. Tapping navigates to the space's chat list view
4. **Course finder icon** → navigates to `/rooms/course`
5. **Settings gear** → navigates to `/rooms/settings`
All navigation uses `context.go()` (GoRouter declarative navigation).
### [`MaxWidthBody`](../../lib/widgets/layouts/max_width_body.dart)
A utility wrapper that constrains content to a max width (default 600px) and centers it on wide screens. Used by settings pages, forms, and other non-chat content to prevent stretching. On narrow screens, the child fills the available width with no extra padding.
---
## Routing & Column Interaction
All authenticated routes live under a single `ShellRoute` that renders `TwoColumnLayout`. The GoRouter child (the page being navigated to) always appears in the **right column** on wide screens, or as the **full screen** on mobile.
Key routing patterns:
- **`/rooms`** — In column mode, left column shows `ChatList`, right shows `EmptyPage` (bear image). On mobile, shows `ChatList` full-screen.
- **`/rooms/:roomid`** — In column mode, left column shows `ChatList` with `activeChat` highlighted, right shows `ChatPage`. On mobile, shows `ChatPage` full-screen (back button returns to chat list).
- **`/rooms/spaces/:spaceid`** — Similar pattern with the space's details in the right column.
- **`/rooms/analytics`** — In column mode, left column shows analytics view, right shows `EmptyAnalyticsPage`. On mobile, shows analytics full-screen.
- **`/rooms/settings`** — In column mode, left column shows `Settings`, right shows settings sub-page or empty.
The left column content is determined by `_MainView` reading `GoRouterState.fullPath`, not by the route tree itself. This means the left column "reacts" to route changes but isn't directly part of the GoRouter page stack.
---
## Mobile-Specific Behavior
On mobile (single-column):
- The nav rail shows a **smaller** icon size (64px width vs 72px)
- The nav rail **hides** when inside a room or during certain flows (determined by `TwoColumnLayout`'s `showNavRail` logic)
- Navigation between "list" and "detail" views uses GoRouter's standard push/pop, appearing as full-screen transitions
- `PopScope` in `ChatListView` handles Android back button: if inside a space, goes back to all chats; if in search mode, cancels search
- App bar height is **56px** on mobile vs **72px** in column mode
- Snackbars are standard floating (not width-constrained) on mobile
---
## Theme Adaptations
`FluffyThemes.buildTheme()` adapts several theme properties based on `isColumnMode`:
- **App bar**: taller (72px vs 56px), with shadow on desktop
- **Snackbar**: width-constrained to `columnWidth * 1.5` (570px) on desktop, unconstrained on mobile
- **Actions padding**: extra horizontal padding on desktop app bars
---
## Future Work
_(No linked issues yet.)_

View file

@ -21,7 +21,8 @@ class PApiUrls {
static String appVersion = "${PApiUrls._choreoEndpoint}/version";
/// ---------------------- Languages --------------------------------------
static String getLanguages = "${PApiUrls._choreoEndpoint}/languages_v2";
/// CMS REST API endpoint for languages (public, no auth required)
static String cmsLanguages = "${Environment.cmsApi}/cms/api/languages";
/// ---------------------- Users --------------------------------------
static String paymentLink = "${PApiUrls._subscriptionEndpoint}/payment_link";

View file

@ -4,13 +4,11 @@ import 'dart:developer';
import 'package:flutter/foundation.dart';
import 'package:async/async.dart';
import 'package:http/http.dart';
import 'package:http/http.dart' as http;
import 'package:fluffychat/pangea/common/config/environment.dart';
import 'package:fluffychat/pangea/common/network/urls.dart';
import 'package:fluffychat/pangea/common/utils/error_handler.dart';
import 'package:fluffychat/pangea/languages/language_model.dart';
import '../common/network/requests.dart';
class LanguageRepo {
static Future<Result<List<LanguageModel>>> get() async {
@ -22,21 +20,27 @@ class LanguageRepo {
}
}
/// Fetches languages directly from CMS REST API (public, no auth required).
static Future<List<LanguageModel>> _fetch() async {
final Requests req = Requests(choreoApiKey: Environment.choreoApiKey);
final response = await http.get(
Uri.parse('${PApiUrls.cmsLanguages}?limit=500&sort=language_name'),
headers: {'Accept': 'application/json'},
);
final Response res = await req.get(url: PApiUrls.getLanguages);
if (res.statusCode != 200) {
if (response.statusCode != 200) {
throw Exception(
'Failed to fetch languages: ${res.statusCode} ${res.reasonPhrase}',
'Failed to fetch languages from CMS: ${response.statusCode} ${response.reasonPhrase}',
);
}
return (jsonDecode(utf8.decode(res.bodyBytes)) as List)
final json =
jsonDecode(utf8.decode(response.bodyBytes)) as Map<String, dynamic>;
final docs = json['docs'] as List;
return docs
.map((e) {
try {
return LanguageModel.fromJson(e);
return LanguageModel.fromJson(e as Map<String, dynamic>);
} catch (err, stack) {
debugger(when: kDebugMode);
ErrorHandler.logError(e: err, s: stack, data: e);

View file

@ -0,0 +1,216 @@
import 'package:flutter_test/flutter_test.dart';
import 'package:fluffychat/pangea/languages/l2_support_enum.dart';
import 'package:fluffychat/pangea/languages/language_constants.dart';
import 'package:fluffychat/pangea/languages/language_model.dart';
void main() {
group('LanguageModel.fromJson', () {
test('parses full CMS response with all fields', () {
final json = {
'id': 42,
'language_code': 'es',
'language_name': 'Spanish',
'locale_emoji': '🇪🇸',
'l2_support': 'full',
'script': 'Latn',
'voices': ['es-ES-Standard-A', 'es-ES-Wavenet-B'],
'createdAt': '2026-01-01T00:00:00.000Z',
'updatedAt': '2026-01-01T00:00:00.000Z',
};
final model = LanguageModel.fromJson(json);
expect(model.langCode, 'es');
expect(model.displayName, 'Spanish');
expect(model.localeEmoji, '🇪🇸');
expect(model.l2Support, L2SupportEnum.full);
expect(model.script, 'Latn');
expect(model.voices, ['es-ES-Standard-A', 'es-ES-Wavenet-B']);
});
test('parses minimal fields (only required)', () {
final json = {'language_code': 'en', 'language_name': 'English'};
final model = LanguageModel.fromJson(json);
expect(model.langCode, 'en');
expect(model.displayName, 'English');
expect(model.l2Support, L2SupportEnum.na);
expect(model.script, LanguageKeys.unknownLanguage);
expect(model.localeEmoji, isNull);
expect(model.voices, isEmpty);
});
test('parses each l2_support level', () {
for (final entry in {
'full': L2SupportEnum.full,
'beta': L2SupportEnum.beta,
'alpha': L2SupportEnum.alpha,
'na': L2SupportEnum.na,
}.entries) {
final json = {
'language_code': 'xx',
'language_name': 'Test',
'l2_support': entry.key,
};
final model = LanguageModel.fromJson(json);
expect(model.l2Support, entry.value, reason: 'l2_support=${entry.key}');
}
});
test('defaults l2_support to na when null', () {
final json = {
'language_code': 'xx',
'language_name': 'Test',
'l2_support': null,
};
final model = LanguageModel.fromJson(json);
expect(model.l2Support, L2SupportEnum.na);
});
test('handles missing locale_emoji', () {
final json = {
'language_code': 'yue-HK',
'language_name': 'Cantonese',
'l2_support': 'beta',
'script': 'Hant',
};
final model = LanguageModel.fromJson(json);
expect(model.localeEmoji, isNull);
});
test('handles missing voices', () {
final json = {'language_code': 'fr', 'language_name': 'French'};
final model = LanguageModel.fromJson(json);
expect(model.voices, isEmpty);
});
test('handles null voices', () {
final json = {
'language_code': 'fr',
'language_name': 'French',
'voices': null,
};
final model = LanguageModel.fromJson(json);
expect(model.voices, isEmpty);
});
test('handles empty voices list', () {
final json = {
'language_code': 'fr',
'language_name': 'French',
'voices': <dynamic>[],
};
final model = LanguageModel.fromJson(json);
expect(model.voices, isEmpty);
});
test('parses text_direction when present', () {
final json = {
'language_code': 'ar',
'language_name': 'Arabic',
'text_direction': 'rtl',
'script': 'Arab',
};
// RTL should be parsed TextDirection.rtl
final model = LanguageModel.fromJson(json);
expect(model.langCode, 'ar');
expect(model.script, 'Arab');
});
test('ignores extra CMS metadata fields', () {
// CMS returns id, createdAt, updatedAt, createdBy, updatedBy
// LanguageModel.fromJson should not break on them
final json = {
'id': 99,
'language_code': 'de',
'language_name': 'German',
'l2_support': 'full',
'script': 'Latn',
'locale_emoji': '🇩🇪',
'c_p_w': 5.2,
'createdAt': '2026-01-01T00:00:00.000Z',
'updatedAt': '2026-01-15T00:00:00.000Z',
'createdBy': {'id': 1, 'email': 'admin@example.com'},
'updatedBy': {'id': 1, 'email': 'admin@example.com'},
};
expect(() => LanguageModel.fromJson(json), returnsNormally);
final model = LanguageModel.fromJson(json);
expect(model.langCode, 'de');
expect(model.displayName, 'German');
});
});
group('LanguageModel.toJson', () {
test('round-trips through fromJson → toJson', () {
final original = {
'language_code': 'ja',
'language_name': 'Japanese',
'l2_support': 'beta',
'script': 'Jpan',
'locale_emoji': '🇯🇵',
'voices': ['ja-JP-Standard-A'],
};
final model = LanguageModel.fromJson(original);
final serialized = model.toJson();
expect(serialized['language_code'], 'ja');
expect(serialized['language_name'], 'Japanese');
expect(serialized['l2_support'], 'beta');
expect(serialized['script'], 'Jpan');
expect(serialized['locale_emoji'], '🇯🇵');
expect(serialized['voices'], ['ja-JP-Standard-A']);
});
test('serializes default values correctly', () {
final model = LanguageModel(langCode: 'sw', displayName: 'Swahili');
final json = model.toJson();
expect(json['language_code'], 'sw');
expect(json['language_name'], 'Swahili');
expect(json['l2_support'], 'na');
expect(json['script'], LanguageKeys.unknownLanguage);
expect(json['locale_emoji'], isNull);
expect(json['voices'], isEmpty);
});
});
group('LanguageModel properties', () {
test('l2 returns true for non-na support levels', () {
for (final level in [
L2SupportEnum.full,
L2SupportEnum.beta,
L2SupportEnum.alpha,
]) {
final model = LanguageModel(
langCode: 'xx',
displayName: 'Test',
l2Support: level,
);
expect(model.l2, isTrue, reason: 'l2Support=$level should be l2=true');
}
});
test('l2 returns false for na', () {
final model = LanguageModel(
langCode: 'xx',
displayName: 'Test',
l2Support: L2SupportEnum.na,
);
expect(model.l2, isFalse);
});
});
group('LanguageModel.unknown', () {
test('has correct defaults', () {
final unknown = LanguageModel.unknown;
expect(unknown.langCode, LanguageKeys.unknownLanguage);
expect(unknown.displayName, 'Unknown');
expect(unknown.l2Support, L2SupportEnum.na);
expect(unknown.l2, isFalse);
});
});
}