Loading…
SUPABASE_URL and SUPABASE_ANON_KEY set. On Render: Dashboard → your service → Environment → add these variables, then redeploy.
Instructor-Informed Speech Assessment
or
Don't have an account? Create one
Enter your email address and we'll send you a link to reset your password.
Without an invitation link, you must use a .edu email and your request will need approval by an administrator after you confirm your email.
Already have an account? Login
Your account is pending approval. You'll receive an email within 24 hours when your account has been approved by an administrator.
Please confirm your email address (check your inbox) if you haven't already. Once your account is approved, you'll be able to access SpeechGradebook.
Loading…
Your evaluation content may be used to improve the speech assessment model. This supports research and product improvement while keeping data anonymized where used for training.
Click or drag & drop your audio/video file here
Supported Formats
Video: MP4, WebM, MOV, AVI, MKV
Audio: MP3, WAV, M4A
Files over 50 MB will be automatically compressed. Very large files may take longer to process.
Upload multiple student videos at once. You'll match each video to a student before processing.
Select the course for all videos in this batch.
All videos will be evaluated using this rubric.
The assignment type for all videos in this batch. You can process different assignments in separate batches.
💡 Tip: If you have videos for multiple assignments, process them in separate batches. Upload all videos for one assignment type (e.g., all Informative Speeches), match them to students, then process. Repeat for each assignment type.
Click or drag & drop multiple video files here
You can select multiple files at once
Or upload organized folders:
If you have folders organized by student (each folder contains multiple videos), you can upload the entire folder structure
Select the student for each video. All videos will be processed with the same rubric and assignment type.
| Video File | Size | Student * | Status | Actions |
|---|
| Video | Student | Status |
|---|
| Video File | Student | Status | Actions |
|---|
Analyzing content and delivery
Upload a CSV or Excel file. Required columns: First Name and Last Name (or a single Name column). Optional: Email, Student ID.
Drop file here or browse
CSV or Excel (.xlsx, .xls) — max 5MB
Move this evaluation to a different class and/or student. This is useful if an evaluation was accidentally saved to the wrong student.
This log shows all changes made to the AI's evaluation. This data helps improve future evaluations.
Upload a PDF, image, Excel spreadsheet, or Word document of your rubric (including blank rubrics—PNG, PDF, Google Sheets exported as XLSX/CSV), and AI will extract the categories, criteria, and point values to create a digital rubric. You can also create rubrics manually in Dashboard → Rubrics.
Loading analytics…
Comparison data will appear here...
Evaluation list will appear here...
Date |
Student |
Course |
Instructor |
Score % |
Rubric |
Assignment Type |
|---|---|---|---|---|---|---|
| Load data to see preview | ||||||
Exports only include evaluations from students who have granted data-use consent. Column filters (▾) apply to the export. Export CSV
Loading platform metrics…
| User | Account Type | Activity Status | Account Created | Last Activity | Total Evaluations | Recent (30 days) | Actions |
|---|
| Rubric Name | Uses (All) | Uses (30d) | Avg Score | Institutions |
|---|
Evaluation costs are automatically tracked when evaluations run. Infrastructure costs (Modal, Supabase, Render, etc.) can be tracked separately in a spreadsheet or billing tool.
| Tier | Evaluations | Total Cost | Cost/Eval | % of Total |
|---|
| Institution | Evals | Cost | Cost/Eval | Revenue |
|---|
Estimates based on published model benchmarks. Last updated .
Choose which tabs appear in the dashboard bar: Show / Hide to toggle, ↑ ↓ to reorder. To change which cards appear on a tab, close this and click Edit cards on that tab.
Choose which cards appear on this tab: Remove / Add to hide or show a card, ↑ ↓ to reorder. Changes are saved automatically.
| Course Name | Code | Instructor | Institution | Term/Year | Modality | Students | Evaluations | Actions |
|---|
| Course Name | Code | Instructor | Term/Year | Modality | Students | Evaluations | Actions |
|---|
Move students from this course to another course. This will transfer all selected students to the target course.
Loading students...
University or theme selection (light interface only)
Manage your account and password
Update your password to keep your account secure.
If you've forgotten your password, we can send you a reset link via email.
Encountered a problem? Report it to help us improve the application.
Generate unique consent links for students and manage data use consent
Generate unique consent links for students. Email each student their link; when they open it and submit, their response is saved in the database.
| Student | Status | Consent link | Actions |
|---|
View the status of errors you've reported and report new issues
Encountered a problem? Report it to help us improve the application.
Here you can see all the errors you've reported and track their status. When an administrator updates the status or resolves an issue, you'll see it here.
Loading your error reports...
View your evaluation usage and quota limits
Loading usage information...
| Date | Evaluations | Cost | Provider |
|---|
Error loading usage information.
Create and manage institutions
Add a new institution so you can assign users and create themes for it.
Approve users and manage roles
Review and approve or reject pending user account requests.
Loading pending users...
View and manage instructors in your department.
Click "Refresh Instructors" to load your department's instructors...
Manage user account tiers and research partner access.
Click "Refresh Users" to load user list...
Manage data exports, recovery, and deletion scope
Set your SpeechGradebook Text + Video Model (Qwen) service URL. Qwen runs on Modal (serverless GPU). Set QWEN_API_URL on Render to your Modal URL. See RECOMMENDED_SETUP.md for setup instructions.
Set this to your Modal URL (e.g. https://yourname--qwen-speechgradebook.modal.run). See llm_training/QWEN_MODAL_SETUP.md for deployment instructions.
Use case: Rubric extraction from PDF/images, video analysis (body movement, eye contact)
Setup: Deploy Qwen on Modal and configure the service URL. A new training method is being set up to replace the previous ISAAC-based workflow.
Note: The previous ISAAC-based training workflow is no longer in use. A new training method is being configured. For now, use Modal for Qwen service deployment. See llm_training/QWEN_MODAL_SETUP.md for current setup instructions.
Can't connect to the Modal URL? (1) Verify your Modal deployment is running and accessible. (2) Use Check Qwen status above to verify connectivity. (3) If the app runs on Render, set QWEN_API_URL in Render dashboard environment variables and redeploy so the app uses the correct URL.
Restore students, evaluations, or courses you removed with Delete/Remove (does not recover items permanently deleted by a super admin).
Export consented evaluations to train the SpeechGradebook Text Model (Mistral) and the SpeechGradebook Text + Video Model (Qwen).
Qwen (Video model)
Transcript + video notes. Save the downloaded file as exported.json in your repo root.
Terminal commands (from repo root):
node llm_training/export_to_jsonl.js exported.json --split 0.9
python llm_training/train_lora.py --train_file train.jsonl --validation_file validation.jsonl --output_dir llm_training/mistral7b-speech-loraHugging Face token: export HF_TOKEN=your_token. For lower VRAM add --load_in_8bit. Serve: python llm_training/serve_model.py --model_path llm_training/mistral7b-speech-lora
Evaluations saved with video (Supabase evaluation-media bucket). Save as train_qwen.jsonl.
Terminal commands (from repo root):
python llm_training/train_qwen_vl.py --manifest train_qwen.jsonl --output_dir llm_training/qwen2.5vl-speech-lora --validate_onlySee llm_training/DUAL_MODEL_TRAINING.md for full training instructions.
Optional Qwen exports
Comparison pairs: same student, two speeches. Correction pairs: AI vs instructor scores. See llm_training/COMPARISON_AND_CORRECTIONS_TRAINING.md.
Training Method Update
The previous ISAAC-based training workflow is no longer in use. A new training method is being configured. For now, export the data above and refer to TRAINING_REQUIREMENTS.md for training setup options.
Guides: TRAINING_REQUIREMENTS.md — Consent, export flow, scripts. LLM Training Quickstart — Prerequisites and setup.
When you delete data, choose where the deletion applies. Admins and Instructors always delete only from SpeechGradebook (local). Super Admins can choose per entity.
These settings apply only to your own delete actions as Super Admin. They are stored in this browser.
Track costs and system health
Monitor GPU costs for evaluations. Track spending by instructor, institution, and over time.
| Instructor | Evaluations | Total Cost | Avg Cost/Eval | GPU Seconds |
|---|---|---|---|---|
| Loading cost data... | ||||
| Institution | Evaluations | Total Cost | Avg Cost/Eval | GPU Seconds |
|---|---|---|---|---|
| Loading cost data... | ||||
Monthly cost alert will trigger when spending exceeds $100. Configure alerts in Sentry dashboard.
Track energy consumption and CO₂ emissions for your evaluations.
Loading energy data…
Verify RLS policies and audit logs
Verify RLS policies and audit logging for FERPA compliance. Run tests as different roles (instructor, admin, super admin) to confirm access is correct.
RLS test checks that you can see your own courses/evaluations and (as instructor) cannot see others'. Audit logs shows your recent access entries; if read is restricted, check the audit_logs table in Supabase Dashboard.
View and manage user-reported errors and system errors
Loading error reports...
No error reports found.
Loading…
SpeechGradebook v1.0
New to SpeechGradebook? Follow these steps to run your first evaluation in minutes.
Evaluations are processed by the SpeechGradebook Model, a fine-tuned version of Qwen running on Modal. No external API key is required.
Before storing evaluations in the cloud, students must give consent. Go to Settings → Consent to generate a consent link for your course. Share the link with students — they complete a short form, and their status updates automatically. Evaluations for students who haven't consented remain local only.
After running several evaluations, go to Dashboard → Course Insights to see score trends, category breakdowns, and comparisons across students. Use the Export tab to download data. Admin Super Admin users see department- and institution-wide views with instructor filters.
Admins can view and filter all courses in Settings → Admin. Super Admins can create and configure institutions and manage global settings.
A complete reference for every feature in SpeechGradebook, organized by task area. Use the Procedural library below for step-by-step instructions that stay inside the app.
Step-by-step topics you can scroll through here. At the end of the guided tour, Open in Support brings you to this section.
Applies to: Instructors
Next: set up Student consent before relying on cloud storage.
Wizard: Upload → Details → Evaluate → Results.
Single: Select Single Upload, add a file, Continue to Details, choose class, student, and rubric, then continue.
Bulk: Select Bulk Upload, pick course and rubric, choose assignment type for the batch, add files, Match videos to students, then Start Processing.
Supported formats appear on the upload screen; large files may be compressed automatically.
On Results, review scores and feedback. Use download or email actions when your deployment shows them. Reopen past runs from the student’s evaluation list.
Settings → General → sidebar Consent Management — pick a course, generate a link, share it with students. Status updates when they submit the form.
Until consent is recorded per your institution’s rules, some evaluations may stay local only.
Open Dashboard → Course Insights. Use sub-tabs for overview, comparisons, evaluation lists, and Export when you need CSV or reports.
The navigation bar provides access to all major areas of the app. What you see depends on your account role.
The evaluation flow is a step-by-step wizard:
Supported formats: MP3, MP4, MOV/QuickTime, WAV, M4A, WebM.
You can edit any saved evaluation to adjust scores, grades, or comments.
Courses organize your students and their evaluations in one place.
Rubrics define the criteria and point values the model uses to evaluate a speech. You can create rubrics manually or import them from an existing file.
Course Insights gives you analytics across all evaluations in a course.
Admin and Super Admin users see department- or institution-wide views with instructor filters.
Settings is divided into tabs based on your role.
Data management, security, model training, version history, and legal information.
SpeechGradebook uses Supabase for authentication and cloud storage. All data is stored per institution with row-level security (RLS) — each user can only access data their role permits.
Student consent — Evaluations for students who haven't given consent are not stored in the cloud. Consent status is tracked in the consent_forms table.
Audit logging — All access to student evaluation data is recorded in audit_logs for FERPA compliance. Each log entry includes the purpose, user, IP address, and user agent.
The SpeechGradebook Model is a fine-tuned version of Qwen trained on your institution's evaluation data. Training runs on Modal (serverless GPU infrastructure), which handles compute provisioning automatically — no server management required.
A trained model gives you scoring behavior calibrated to your rubrics and grading style, with all data remaining within your institution's environment.
To export training data: Go to Settings → Admin → Data Management.
Training guides
Recent changes to SpeechGradebook. See CHANGELOG.md in the project repository for the full history.
SpeechGradebook is provided for educational use. By using the app, you agree to your institution's terms and any applicable privacy policies. Student data is processed in accordance with FERPA and your institution's data agreements.
Model licensing — The SpeechGradebook Model is a fine-tuned version of Qwen (Apache 2.0 license). If you distribute the model, adapter weights, or any training or serving code that includes the model, you must include the Apache 2.0 license and required attribution. Internal use without redistribution doesn't require including the license in the app. See llm_training/LICENSE for the full license text and third-party notices.
For licensing or legal questions, contact your institution or the support address below.
For questions or technical support, contact: [email protected]