Monitor Learning and AI Diagnostics
Interpret course freshness, practice-item health, cohort outcomes, and AI or channel usage without acting on thin data.
In this guide4 sections and article resources
Article resources
Use diagnostics to decide what needs review, not to label a learner or make an automatic content change. Start from the owning report and keep the displayed time window and data volume in view.
Check course freshness
Open Creator console → Courses. Review the content-indexed time, status, and available progress signal. If an assistant cites old course content, confirm that the intended course revision is published and indexed before regenerating or changing the persona.
Interpret Item Health
Open Settings → AI → Item Health.
The nightly report can flag:
- Too easy — more than 90% of responses are correct.
- Too hard — fewer than 50% are correct.
- Low discrimination — the absolute discrimination value is below 0.15.
- Insufficient data — fewer than 10 responses.
Thin items are not also classified as too easy or too hard. A flag can indicate an unclear prompt, wrong answer key, content mismatch, or an intentionally introductory/advanced item. Review the source, objective, wording, answer, feedback, learner sample, and locale before revising it.
Preserve a new assessment revision and decide how prior attempts and certificates should be handled. Do not silently rewrite a live answer key.
Reconcile cohort and AI usage
Open Cohort outcomes for completion, status, and available AI-cost context. Compare aggregate outcomes with schedule, enrollment, access, and revision changes; avoid public member rankings.
Use Settings → Monetization → Billing and the available AI/channel reports for credits, model or SKU usage, and channel activity. Reconcile a spike by time window, feature, run/session identifier, provider attribution, and actual completed work. A reserved, retried, cancelled, or failed job can appear differently from a completed generation.
Troubleshoot empty or surprising data
- Confirm the community, role, locale, tab, filter, and date window.
- Distinguish loading, disabled, access-denied, and not-enough-real-data states.
- Refresh once and check the owning course, cohort, generation, or channel record.
- Record redacted evidence and identifiers before escalating.
Do not create fake learner responses, send real outreach, or run unnecessary AI generation just to populate a report.
Was this helpful?
Your response helps us improve this guide.