
Brain2Frame (FYP 2)
Real-Time EEG-to-Video System Using Adaptive Lightweight AI
“Brain2Frame proves that you can convert thought into video using cheap hardware and clever algorithms, in real-time.”
The Vision
Brain2Frame (FYP 2) is a real-time brain-computer interface (BCI) that converts EEG brain signals directly into cinematic videos. Using a consumer Muse 2 headset, the system reads live brain activity, decodes it into meaningful sentences, and generates short AI-produced videos based on what the user is thinking. The core innovation is avoiding heavy AI models entirely. Instead of GPT or large transformers, the system uses a lightweight, adaptive, geometry-based approach that works in real time without expensive hardware or large-scale training.
End-to-End Pipeline
The pipeline runs in eight stages: (1) EEG Acquisition: the Muse 2 headset streams 4 channels at 256 Hz via BrainFlow into Python. (2) Real-Time Feature Streaming: raw EEG is converted into 30 spectral features (band powers, alpha/gamma ratios, left-right asymmetry, phase differences) at 10 Hz. (3) Deep Feature Extraction: a 5-second recording window generates a 294-dimensional feature vector including temporal dynamics (HFD, Hjorth mobility), statistical measures, and cross-channel correlations. (4) Dual-Phase Classification: Phase 1 uses a fixed Energy Backbone for immediate predictions (~46% accuracy); Phase 2 activates the Triangle Stability algorithm after ~3 calibration repetitions, searching 43,000 feature pairs to build a personalised model that improves accuracy to ~60%+. (5) Sentence Prediction: EEG features are matched against a predefined sentence set (3 to 10 sentences). (6) Prompt Engineering: the matched sentence is expanded into a cinematic prompt. (7) Video Generation: the prompt is sent to Google Veo3 via the fal.ai API, producing a ~4-second high-quality clip in seconds. (8) Dashboard Display: a Next.js UI shows live EEG signals, channel quality, prediction confidence, and video playback in real time.
Technologies, Frameworks & Tools
Hardware: Muse 2 EEG headset (TP9, AF7, AF8, TP10 electrodes, 256 Hz). Signal processing: Python, NumPy, SciPy (FFT, bandpass filtering, statistical features). EEG streaming: BrainFlow SDK, which handles live acquisition from the headset into Python with no mobile dependency. Classification: custom dual-phase engine with no heavy training, no LLMs, and no neural networks. The Triangle Stability Algorithm searches stable feature pairs geometrically and self-improves without retraining or human feedback. Video generation: Google Veo3 accessed via the fal.ai cloud API. Frontend: Next.js (React) real-time dashboard with live signal visualisation, confidence display, and video playback. Earlier experiments used PyTorch with DNN and CLIP models, but the final system minimised heavy dependencies in favour of the adaptive geometry-based approach.
Key Results & Contributions
Brain2Frame (FYP 2) is the first end-to-end EEG-to-video system built on consumer hardware. It achieves ~60% real-world accuracy and ~80% best-case (oracle) accuracy, filters out 98.5% of incorrect results, and retrieves the correct sentence from 2,500 options with 80% accuracy across sessions without retraining. Processing latency sits at ~85ms, genuinely real-time. The system solves three hard problems: the semantic gap between weak brain signals and meaningful content, session drift (brain signals shift over time, addressed through stable feature design), and accessibility (works on a consumer headset instead of medical-grade hardware). Every decision, including confidence display, signal quality monitoring, and visible reasoning, is built for interpretability alongside performance.
Project Gallery
17 items




Explore More Projects
Swipe or click arrows to see other work
















