People, interfaces & ethics

Observe an action with a concrete endpoint. Existing enthusiasts are not automatically representative of first-time students.

Recruit participants from the intended first-time population, including relevant device, language, and accessibility needs. Give a realistic goal such as finding a course with a prerequisite and enrolling in an available session, using a scenario that does not name the controls to click. Define success before observing results: correct course and session, no unintended enrollment, and comprehension of next steps. Record completion, errors, assistance, time with context, and participants' explanations. Ask neutral follow-ups after the attempt and do not coach unless the protocol records the intervention.

A small formative study can reveal concrete obstacles and inform redesign; it does not reliably estimate a population-wide percentage improvement. For a causal comparison, randomly assign eligible users to versions, use comparable tasks and exposure, choose the primary outcome in advance, and report uncertainty and exclusions. Avoid treating satisfaction as a substitute for task success. A result may show that a visually preferred screen increases mistaken enrollments. Keep identifiable recordings only as long as needed and obtain appropriate participant consent.

Check your reasoning

• Recruit for the claimed user population.
• Define observable success before testing.
• Separate discovery of problems from estimation of causal effect.

Track focus, accessible names, error relationships, and the user's original task. Visual appearance alone does not expose state to assistive technology.

Open the dialog from a keyboard-operable control with a clear name. Move focus into an appropriate initial element, give the dialog an accessible name, and ensure background content is not accidentally interactive while it is modal. Keep keyboard navigation within the modal interaction, provide a named close control, and support the expected dismissal behavior without discarding work unexpectedly. On validation failure, identify each error in text, associate it with its field, and provide a useful summary or focus strategy when multiple fields fail. Do not communicate the error only through color.

On close, return focus to the invoking control if it still exists, or to a logical successor if the action changed the page. Announce successful enrollment through appropriate status feedback and update the visible state. Manually test keyboard order, visible focus, zoom and reflow, screen-reader names and announcements, dismissal, and error recovery. An automated scanner can detect some missing semantics but cannot establish whether the focus destination makes sense or whether a user can complete the task.

Check your reasoning

• Specify the complete focus lifecycle.
• Make errors textual and programmatically associated.
• Test task completion with keyboard and assistive technology.

Specify what the labels mean and whether they are trustworthy. Different error measures answer different normative questions.

A has accuracy 160/200=80%, TPR=80/100=80%, and FPR=20/100=20%. B has accuracy 130/200=65%, TPR=40%, and FPR=10%. Overall accuracy is 290/400=72.5%, TPR=120/200=60%, and FPR=30/200=15%. Approval rates are 100/200=50% for A and 50/200=25% for B. The aggregate hides the much higher missed-positive rate for B.

Equal approval rates alone would not establish fair treatment: labels may reflect historical exclusion, groups may contain unmeasured variation, error costs differ, and satisfying one statistical criterion can conflict with another. Investigate label construction, missing data, deployment consequences, and performance for relevant intersections with adequate uncertainty. Provide understandable reasons, a meaningful human review or appeal route, and a process for correcting data. Sensitive attributes may be needed to detect disparities, but collect only what is justified, restrict access and retention, separate audit use from operational decision use where appropriate, and assess whether the audit itself creates risk. A numerical metric informs a decision; it does not settle the ethical judgment.

Check your reasoning

• Compute group and aggregate rates with correct denominators.
• Question labels and consequences.
• Provide recourse and a justified data-minimization plan.

A proxy becomes a target when people are rewarded for it. Define an outcome closer to the educational purpose.

The intended benefit is sustained learning, while study time is only a noisy proxy. Public ranking can reward idle sessions, penalize efficient or offline learners, expose behavior, and misattribute shared-device activity. Alternatives include private personal goals, optional cohort challenges, and progress based on meaningful practice with user control over visibility. A reversible trial should be opt-in, limit retained data, avoid high-stakes consequences, and let users remove their participation.

Choose evidence closer to learning: delayed retrieval or transfer tasks, correction of prior errors, and voluntary sustained practice, with attention to gaming and unequal access. Predefine stop criteria such as increased unwanted disclosure, complaints about coercion, substantial subgroup harm, or engagement gains without learning gains. Record who owns the decision, which uncertainties remain, and when it will be reviewed. A reasonable conclusion may be to reject public ranking even if it increases session minutes. Ethical responsibility includes declining a technically feasible optimization when the measured target conflicts with the product's purpose.

Check your reasoning

• Separate the proxy from the intended benefit.
• Evaluate concrete alternatives and affected users.
• Specify reversible rollout and observable stop criteria.