My design responsibility
The customer experience
I shaped when learning happened, what evidence analysts could see, and how they could approve or stop a lesson. Model capabilities and API constraints informed those decisions.
Sublime Security · ASA Lesson Store
Helping an AI analyst learn from its team.
I designed how analyst corrections become reusable lessons, with clear controls for when ASA learns and how teams intervene.
- My role
- Lead Product Designer
- Timeline
- Nov 2025 to Aug 2026
- Team
- Product, engineering, ML
- Launch
- GA July 2026
00 · What is ASA?
Sublime’s AI security analyst.
ASA investigates suspicious emails and explains whether it thinks they're a threat. Security teams can review its reasoning and correct its verdict when it gets something wrong.
01 · The gap
Teaching ASA what each organization knows.
ASA could investigate an email and explain its verdict, but a correction only fixed that one message. The agent did not retain the organization-specific context behind it.
02 · Defining the system
Designing the interface and learning model together.
I worked across design, product, engineering, and ML to turn a model behavior into a learning loop customers could understand and govern.
01
Correction
An analyst changes ASA's verdict.
02
Evaluate
ML reads the message and its surrounding signals.
03
Learn
ASA creates an organization-specific lesson.
04
Govern
The lesson activates or waits for approval.
05
Apply
That context informs the next investigation.
The lesson feeds the next investigation
ML responsibility
The learning engine
The ML team owned lesson generation and evaluation. Together, we worked through the message-level signals and triggers the product could use.
03 · The pivotal decision
Choosing the right level of control.
Eight enterprise design partners did not agree on one answer. That disagreement revealed two legitimate governance models.
One learning model. Two trust models.
A correction creates a lesson.
The organization decides what happens next.
Low-friction path
Activate automatically
The analyst's correction is enough to put the lesson to work.
PendingActive
No additional review step
High-control path
Require human approval
A reviewer approves the lesson before it can influence future verdicts.
PendingReview
Active only after approval
Organization-level control
Customers could choose either path or disable lesson creation globally. The product adapted to their trust model instead of forcing one definition of autonomy.
AgentsASA
ASA Lesson Store
Capture context from analyst corrections.
Enable Lesson StoreSetup exploration
Choose how your organization starts capturing lessons.
Enable now
Start capturing lessons immediately.
Request approval
Submit for AI committee review before activating.
Capture lessons for
All correctionsWhich corrections are eligible to become lessons
Default lesson storage
Opt-inPre-check “Store this lesson” when submitting a correction
04 · Making the system legible
Making agent learning legible.
I reduced a complicated backend into three questions a customer could answer without understanding the model.
01
What is ASA doing?
Status, message volume, activity, and connected Automations created orientation at a glance.
02
Why does this lesson exist?
The correction, source message, creator, and activation state made the lesson traceable.
03
Where is it being used?
Application history showed whether the lesson was recurring context or a one-off to deactivate.
Trigger logic, creator, and lesson state in one view.
AgentsASALessons
● ActiveLSN-001Expires in 28 days
Shared drive notifications from corporate OneDrive bypass link analysis
Trigger logic
- When
- Sender domain matches onedrive.com or sharepoint.com
- And
- Subject contains ‘shared a file’ or ‘sent you’
- Then
- Skip link reputation checks, classify as Benign
History
- Created by
- AJ Williams
- Created on
- Mar 6, 2026
- Times applied
- 124
Pause lessonView all messages
Recent applications connect the lesson to the messages it influenced.
AgentsASALessons
● ActiveLSN-001Expires in 28 days
Shared drive notifications from corporate OneDrive bypass link analysis
124total applicationsLast applied Apr 6, 2026 · 1:58pm
Recent applications3 recent messages
05 · Learning through rollout
Learning through rollout.
Private beta, public beta, and GA each answered a different design question.
01 · First release
Private beta
A bare settings page showed ASA's connected Automations, their active or passive state, and a basic table of captured lessons.
What I learnedThe early surface let design partners react to the learning model before the team invested in a finished dashboard.
AgentsASA
ASA Lesson Store
Capture context from analyst corrections.
Enable Lesson StoreSetup exploration
Choose how your organization starts capturing lessons.
Enable now
Start capturing lessons immediately.
Request approval
Submit for AI committee review before activating.
Capture lessons for
All correctionsWhich corrections are eligible to become lessons
Default lesson storage
Opt-inPre-check “Store this lesson” when submitting a correction
02 · Second release
Public beta
This release added message-ingestion and lesson statistics, disagreement visibility, a lesson detail view, and the first complete governance controls.
What I learnedCustomers wanted different trust postures. Some preferred automatic activation. Others needed a second person to approve each lesson.
AgentsASA
ASA
Your AI analyst for ambiguous messages.
Receives142 messagesthis week
Analyzes94% agreementwith analysts
Learns6 active lessons2 expiring soon
Automations
Send User Reports to ASA
Send Suspicious messages to ASA
Lessons
LessonAppliedStatus
Shared drive notifications from corporate OneDrive bypass link analysis124Active
Treat DocuSign reminders from legal@client-firm.com as Benign47Active
Internal SPF failures from geo-redirected mailers are misconfiguration81Active
03 · July 2026
General availability
GA expanded the overview with activity trends and made each lesson traceable to its source, creator, state, and real application history.
What I learnedA lesson was only useful if a team could understand why it existed, where it was being applied, and how to stop it.
AgentsASALessons
● ActiveLSN-001Expires in 28 days
Shared drive notifications from corporate OneDrive bypass link analysis
Trigger logic
- When
- Sender domain matches onedrive.com or sharepoint.com
- And
- Subject contains ‘shared a file’ or ‘sent you’
- Then
- Skip link reputation checks, classify as Benign
History
- Created by
- AJ Williams
- Created on
- Mar 6, 2026
- Times applied
- 124
Pause lessonView all messages
04 · August 2026
Post-GA iteration
The lesson detail evolved to include a full message table, original-message drill-down, and clearer deactivate and reactivate actions.
What I learnedGovernance worked better when the evidence lived with the lesson instead of behind a link to another part of the product.
AgentsASALessons
● ActiveLSN-001Expires in 28 days
Shared drive notifications from corporate OneDrive bypass link analysis
124total applicationsLast applied Apr 6, 2026 · 1:58pm
Recent applications3 recent messages
MessageVerdictApplied
Anna shared 'Q2 roadmap.pptx' with youBenign2h ago
You've got a new file from SharePointBenign5h ago
Reminder: document expiring soonBenignYesterday
06 · Designing beyond ASA
Building a pattern beyond ASA.
I did not want customers to relearn the product every time they moved between agents. The layout became a shared design framework for the agent portfolio.
07 · Outcome
From roadmap concept to GA.
The work had just reached GA. I am grounding the outcome in what shipped and what changed through beta.
From roadmap concept to GA
I helped shape the learning model and product experience from the first roadmap conversations through private beta, public beta, and general availability.
Beta feedback changed governance
The shipped model supported automatic activation and secondary approval, so organizations could match learning to their team structure and trust model.
A pattern built beyond one agent
The agent page established a reusable anatomy for status, statistics, activity, Automations, learned behavior, and underlying data.
What I would carry forward
Let the learning disappear into the workflow. Keep the control within reach.