Project Aura
Last Updated: July 20, 2026
Purpose
This page defines how Project Aura should behave when the student interview does not proceed exactly as expected. It is a shared behavioral specification for reviewers, product owners, and engineers—not merely a software QA checklist.
The goal is to preserve student trust, protect reviewer interpretability, avoid unsupported conclusions, and ensure that recoverable operational problems do not become behavioral judgments about the student.
P0 Cases
16
Required for a credible shadow pilot
P1 Cases
12
Important calibration and product decisions
Implemented
3
Behavior currently represented in the prototype
Open or Planned
23
Requires policy, implementation, or validation
A disconnect, timeout, translation issue, duplicate submission, or interface error must not be interpreted as a student weakness.
Aura should retain the student's original language, submitted text, meaningful revisions, and relevant session events rather than silently rewriting history.
When possible, Aura should return the student to the last successfully committed turn instead of forcing the interview to begin again.
Aura may clarify approved scenario information but should not guide the student toward a preferred behavioral answer.
Short, unclear, translated, or nonresponsive answers may limit evidence, but limited evidence should not automatically become concern evidence.
Reviewers should see events that affect interpretation. Routine technical noise should remain in internal diagnostics.
Safety, conduct, and consequential policy decisions require an approved human process rather than an improvised model response.
Behavior is represented in the current prototype.
Some behavior exists, but important policy or recovery logic remains.
Desired behavior is defined but not yet implemented.
Reviewer or product policy is still unresolved.
The prototype currently lacks the necessary workflow.
Required for pilot continuity, safety, or credible reviewer use.
Important for trust, interpretability, or staff testing.
Useful later but not required for the first shadow deployment.
How Aura should respond when the student's submitted answer is empty, limited, unclear, revised, or otherwise outside the ideal interaction path.
The student submits without entering any text.
Current Behavior
Submission is blocked and the student is asked to enter a response.
Proposed Behavior
Continue blocking submission. Use neutral language that does not imply the student has answered incorrectly.
Reviewer Visibility
No reviewer flag unless repeated empty submissions become operationally relevant.
"Yes," "No," "I would leave," or another minimal answer.
Current Behavior
The interface encourages additional detail, but the exact sufficiency policy remains provisional.
Proposed Behavior
Ask once for additional detail. If the student still submits a short response, preserve it and continue rather than trapping the student.
Reviewer Visibility
Mark the turn as limited or insufficient evidence without treating brevity itself as negative behavioral evidence.
Open Question
Should sufficiency depend on word count, semantic content, or both?
The student discusses an unrelated topic or provides a response that does not address the scenario.
Current Behavior
The response is retained, but nonresponsive-answer policy is not yet fully defined.
Proposed Behavior
Offer one neutral clarification or restatement. If the second response remains nonresponsive, preserve both responses and continue with a reviewer-visible limitation.
Reviewer Visibility
Show that the answer was potentially nonresponsive and include the original response.
Open Question
Who determines nonresponsiveness during the pilot: a rule, the language model, or the reviewer?
"What do you mean?" or "Am I allowed to call someone for help?"
Current Behavior
No formal clarification policy is currently implemented.
Proposed Behavior
Clarify only information already present in the approved scenario. Do not coach the student toward a preferred answer.
Reviewer Visibility
Preserve the clarification request and the clarification provided in the transcript.
Open Question
Which clarifications may Aura provide without changing the assessment?
"I do not want to answer this question" or "I would rather skip."
Current Behavior
No explicit refusal workflow is currently implemented.
Proposed Behavior
Acknowledge the refusal neutrally, record it, and allow the interview to continue when policy permits.
Reviewer Visibility
Show the refusal as a transcript event, not automatically as concern evidence.
Open Question
Should students be allowed to skip one question, one scenario, or neither?
The student initially says they would leave, then later says they would stay and apologize.
Current Behavior
Later responses are retained, but contradiction handling is primarily factor-level.
Proposed Behavior
Preserve both answers. Treat the change as possible updating, clarification, or contradiction rather than automatically choosing one answer as true.
Reviewer Visibility
Display both responses and identify the possible contradiction or revision.
Open Question
When should a changed answer count as healthy reflection rather than inconsistency?
The student submits several pages of text or pastes unrelated content.
Current Behavior
The current interface is optimized for short responses but does not yet define an explicit maximum.
Proposed Behavior
Set a clear student-facing character or word limit. Preserve the full accepted response and avoid silently truncating submitted content.
Reviewer Visibility
Show the accepted response and any length-related system notice.
Open Question
What response limit supports meaningful evidence without burdening students or reviewers?
The student presses Continue multiple times during a slow network response.
Current Behavior
Duplicate-submission protection is not yet formally documented.
Proposed Behavior
Use an idempotent turn identifier and accept only one submission for the active turn.
Reviewer Visibility
Show one response only. Technical duplicates should not appear in the reviewer packet.
How Aura should preserve interview continuity when the browser, network, session, server, or model request behaves unexpectedly.
The student loses connectivity while writing or submitting an answer.
Current Behavior
Session recovery behavior is not yet guaranteed.
Proposed Behavior
Preserve locally entered text where practical, retry safely, and restore the student to the active turn without creating a duplicate response.
Reviewer Visibility
Do not show routine network retries. Show only meaningful interruption metadata if it affected completion.
Open Question
How long should Aura continue retrying before presenting a recovery message?
The student refreshes the page during the interview.
Current Behavior
The current prototype may restart or lose in-progress state depending on the point of refresh.
Proposed Behavior
Restore the last committed turn and return the student to the current unanswered question.
Reviewer Visibility
No reviewer flag unless refresh behavior caused a meaningful interruption.
The student accidentally closes the tab and returns later.
Current Behavior
Resume behavior is not yet fully implemented.
Proposed Behavior
Allow the student to resume the same interview from the last successfully saved turn within an approved expiration window.
Reviewer Visibility
Show resumed-session metadata only when operationally useful.
Open Question
How long should an unfinished interview remain resumable?
The student leaves the interview inactive for an extended period.
Current Behavior
No final inactivity policy has been selected.
Proposed Behavior
Warn before expiration, preserve committed responses, and allow a controlled resume where permitted.
Reviewer Visibility
Show that the interview timed out or resumed if it meaningfully affects interpretation.
Open Question
Should timeout be based on inactivity, total interview duration, or both?
Evidence extraction or next-probe selection returns an error.
Current Behavior
Prototype error handling may not always provide a graceful fallback.
Proposed Behavior
Preserve the submitted response, retry safely, and use an approved deterministic fallback probe when the adaptive service is unavailable.
Reviewer Visibility
Show that a fallback path was used if it affects the interpretation of the interview.
Open Question
Which fixed fallback sequence should be approved for each scenario?
The student begins on a phone and later resumes on a laptop.
Current Behavior
Cross-device resume is not currently guaranteed.
Proposed Behavior
Resume through a secure participant-session link or authenticated assignment rather than device-local state alone.
Reviewer Visibility
No special reviewer flag unless the session boundary is relevant.
How Aura should support multilingual, mixed-language, informal, and emoji-containing responses without confusing language form with behavioral quality.
The student answers in Russian, Spanish, Ukrainian, or another program-relevant language.
Current Behavior
The language model may understand the response, but multilingual product behavior is not yet formally validated.
Proposed Behavior
Accept the original-language response, extract evidence without requiring the student to switch languages, and preserve the original text.
Reviewer Visibility
Show the original response and a clearly labeled reviewer translation when needed.
Open Question
Which languages must be explicitly validated before the shadow pilot?
The student begins in English and completes part of the answer in another language.
Current Behavior
Likely processable by the model, but no defined user or reviewer policy exists.
Proposed Behavior
Accept mixed-language responses and preserve the exact original text.
Reviewer Visibility
Show original text plus translation where necessary.
A phrase has multiple possible meanings or depends heavily on cultural context.
Current Behavior
Translation confidence and ambiguity are not yet exposed.
Proposed Behavior
Preserve the original phrase, avoid overconfident interpretation, and flag meaningful ambiguity for reviewer attention.
Reviewer Visibility
Display the original text, translation, and an ambiguity note.
"😭", "🤷", "idk 😅", or a response whose meaning depends on emoji.
Current Behavior
Emoji may be interpreted by the language model but has no explicit policy.
Proposed Behavior
Treat emoji as part of the response context. Do not infer a behavioral factor from emoji alone without supporting language or scenario context.
Reviewer Visibility
Preserve emoji exactly in the transcript.
The response contains shorthand, misspellings, transliteration, or incomplete sentences.
Current Behavior
The language model generally tolerates informal language.
Proposed Behavior
Assess behavioral content rather than writing quality unless language proficiency is explicitly part of the approved evaluation policy.
Reviewer Visibility
Preserve the original response without correction.
How Aura should handle possible rehearsed answers, shared questions, copied content, hostile language, and safety-related disclosures.
The answer appears unusually polished, generic, or similar to a known generated response.
Current Behavior
No reliable AI-authorship determination is implemented.
Proposed Behavior
Do not claim that a response was AI-generated based on style alone. Use scenario randomization, contextual follow-ups, timing metadata, and reviewer judgment as supporting signals.
Reviewer Visibility
Show relevant interaction metadata without presenting an unsupported authorship verdict.
Open Question
What operational response should reviewers take when answer authenticity is uncertain?
A student sees scenarios or probes that another participant previously received.
Current Behavior
The prototype uses a small visible scenario bank.
Proposed Behavior
Randomize scenario assignment and probe paths within an approved bank. Treat exposure risk as a content-design and operations issue, not solely as misconduct detection.
Reviewer Visibility
No automatic concern flag solely because a response resembles an expected answer.
The answer closely mirrors ideal rubric language but lacks concrete reasoning.
Current Behavior
Evidence extraction may still identify apparently positive language.
Proposed Behavior
Use contextual follow-ups that require the student to adapt the stated plan. Ground evidence in specific behavior rather than rubric vocabulary alone.
Reviewer Visibility
Show the complete transcript and any evidence limitations.
The student uses swear words, insults, or expresses frustration.
Current Behavior
The response is retained, but safety and interpretation policy are not fully documented.
Proposed Behavior
Continue the interview unless the content presents a safety issue. Interpret the behavior in context rather than treating profanity alone as disqualifying.
Reviewer Visibility
Preserve the response and show any context-grounded behavioral flag.
The student describes immediate risk of harm, abuse, self-harm, or danger to another person.
Current Behavior
The prototype does not yet define a production escalation workflow.
Proposed Behavior
Pause ordinary assessment behavior and follow an American Councils-approved safety and escalation protocol.
Reviewer Visibility
Create a restricted urgent-review event with appropriate access controls.
Open Question
What disclosures require immediate escalation, to whom, and through which operational channel?
Which unusual interview events should remain visible to reviewers and which should remain implementation details.
A factor never surfaces or is supported only by vague language.
Current Behavior
Unobserved factors remain insufficient rather than negative.
Proposed Behavior
Continue distinguishing missing evidence from concern evidence. Show the reviewer what was and was not observed.
Reviewer Visibility
Factor status, evidence count, confidence, and limitations.
One response supports accountability while a later response suggests avoidance.
Current Behavior
Contradiction tracking exists at the factor-assessment layer.
Proposed Behavior
Preserve both evidence directions, avoid silently averaging them away, and expose the contradiction to the reviewer.
Reviewer Visibility
Positive evidence, concern evidence, contradiction status, and supporting quotes.
Aura cannot confidently select a contextual probe or a model request fails.
Current Behavior
No final reviewer-facing fallback indicator has been defined.
Proposed Behavior
Use a reviewer-approved catch-all probe and record that the fallback path was used.
Reviewer Visibility
Show fallback-probe metadata in the transcript or session diagnostics.
The reviewer does not read the student's original language.
Current Behavior
Reviewer translation display is not yet finalized.
Proposed Behavior
Display original text first, accompanied by a clearly labeled translation and any meaningful translation uncertainty.
Reviewer Visibility
Original response, translated response, language, and ambiguity note.
The student leaves after one scenario and resumes later.
Current Behavior
Session metadata may not yet be consistently surfaced.
Proposed Behavior
Preserve one coherent transcript while recording meaningful interruption and resume events.
Reviewer Visibility
Start time, completion time, resume events, and incomplete status where relevant.
The following behaviors represent the recommended minimum coverage before Aura is used in a 500-student shadow deployment.
Each P0 behavior should be tested through at least one successful path, one failure path, and one recovery path.
| Test Area | Normal Path | Failure Path | Recovery Evidence |
|---|---|---|---|
| Response submission | One answer creates one committed turn. | Student presses Continue repeatedly. | Only one response and one probe transition are stored. |
| Network interruption | Answer submits and next probe loads. | Connection drops during submission. | Submitted text is preserved and no duplicate turn is created. |
| Refresh and resume | Student proceeds through all six turns. | Page refreshes during an unanswered probe. | Student returns to the same unanswered turn. |
| Short answer | Student provides sufficient behavioral detail. | Student answers "I would apologize." | Aura requests detail once, then accepts the final response with an evidence limitation. |
| Multilingual response | Student answers in English. | Student answers in another approved language. | Original text is preserved and reviewer translation is available. |
| Probe-selection failure | Adaptive planner selects an approved probe. | Planner or model request returns an error. | Approved fallback probe is selected and the turn remains auditable. |
| Incomplete interview | Student completes both scenarios. | Student closes the browser after the first scenario. | Session resumes or is clearly marked incomplete according to policy. |
| Reviewer packet | Packet contains transcript, evidence, factors, and recommendation. | Interview includes interruption, limited answer, or translation. | Relevant limitations are visible without converting technical events into negative evidence. |
Joe and Elena review the expected student-facing behavior
↓
Reviewer team identifies missing real-world cases
↓
Richard and technical team mark current implementation status
↓
American Councils confirms timeout, resume, language, safety, and reviewer-visibility policies
↓
P0 behaviors become staff-testing acceptance criteria
↓
Staff test results update this page before the shadow deployment
Behavioral Principle
Aura should remain calm, recoverable, transparent, and evidence-grounded when a student gives an unexpected answer or the technology behaves unexpectedly. The system should make uncertainty visible without converting operational friction into an unsupported judgment about the student.