USC's PAL Watches Lectures With You and Adapts Every Quiz in Real Time
A new AI tutoring system turns lecture videos into interactive sessions, using reinforcement learning to adjust question difficulty in real time.
Takeaways
- PAL turns lecture videos into interactive sessions with adaptive multiple-choice questions timed to the content. Paper
- A four-stage pipeline combines transcript scanning, OCR, LLaVA-mini vision, and heuristic difficulty rating.
- Difficulty is chosen by blending an IRT statistical prior with epsilon-greedy Q-learning, weighted by session confidence.
- Learner state is six dimensions: skill, accuracy, response time, streak, velocity, confidence.
- Post-lesson summaries use SentenceTransformer retrieval plus Llama 3.2 1B to stay grounded in lecture text.
- Code and demo video are available; no learner-study results are reported yet. Code
Most AI education platforms claim to be personalized, then hand every learner the same quiz, the same pacing, and the same generic feedback. A team from the University of South Carolina's AI Institute proposes a different approach with PAL (Personal Adaptive Learner), a system that watches a lecture video alongside you, interrupts with questions calibrated to your current understanding, and closes the session with a summary tailored to what you struggled with.
The paper, accepted as a demo at AAAI, argues that existing adaptive learning approaches such as pretests or rigid rule-based adjustments capture only coarse measures of ability and struggle to keep learners in the sweet spot where material is neither trivial nor overwhelming. PAL's fix combines multimodal video analysis with a hybrid reinforcement learning controller that decides, question by question, how hard to push.
From lecture video to timestamped quiz
PAL's ingestion pipeline runs in four stages before any learner sees a question. The transcript analyzer scans for candidate insertion points, either through linguistic cues like the phrase "is defined as" or by sampling every N sentences. At each candidate point, a context validator grabs the corresponding video frame, runs OCR on slide text, and uses LLaVA-mini for visual description, bundling all three signals into a single context block.

A question generator produces multiple-choice items from that context, and a difficulty rater tags each one. The rating logic is refreshingly simple: "what is" maps to Easy, "why/how" to Medium, and "apply/predict" to Hard, with an LLM fallback for edge cases. Factual questions land as Easy, conceptual as Medium, applied as Hard.