compai-an-ai-powered-secure-data-management-interface

Case study

CompAI: An AI Powered Secure Data Management Interface

This collaborative project, carried out in partnership with National Security Agency and Laboratory for Analytic Sciences, aimed to create an AI-powered interface to support Intelligence Analysts and Target Digital Network Analysts (TDNAs). The goal was to aid them in maintaining compliance during intelligence analysis tasks. The tool integrates visualization, informal education, and machine learning to assist analysts in maintaining compliance with complex legal and regulatory frameworks.

Project type

UX Design, UX Research, Prototype

Timeline

JAN-APR 2024

Role

For this project, I led the project’s planning and strategy, team coordination, and user research. Additionally, I spearheaded the ideation, interface design, and prototyping of the interface.

Approach

Our team took a structured approach by first breaking down project requirements and setting a clear timeline. We began with interviews to understand TDNA workflows, followed by benchmarking existing tools. Through user research, we developed personas, user stories, and journey maps, ensuring a strong foundation for design decisions. We then created rapid prototypes, tested solutions with TDNAs, and refined them based on feedback.

Context

Who is a TDNA and what do they do?

A Target Digital Network Analyst (TDNA) tracks and analyzes how targets communicate and operate in cyberspace. They use data from networks to uncover patterns, profile targets, and develop intelligence techniques. TDNAs are tech-savvy, adaptable, and skilled in analyzing digital communications to support cybersecurity and intelligence operations.

Challenge

The challenge is designing an interface that helps TDNAs analyze data efficiently while ensuring compliance with complex regulations. Analysts struggle with applying formal training, managing high cognitive load, and accessing up-to-date resources due to outdated tradecraft repositories. Fear of non-compliance leads to cautious queries, limiting innovation and slowing intelligence analysis.

Final Design

ComplAI

ComplAI is an AI-powered compliance assistant designed to help Target Digital Network Analysts (TDNAs) navigate complex regulatory frameworks efficiently. By leveraging data from multiple sources and advanced tradecraft capabilities, ComplAI provides real-time alerts, detects potential compliance risks, and offers actionable recommendations. This ensures that analysts can stay compliant while maintaining analytical agility, reducing cognitive load, and enhancing decision-making in an ever-evolving regulatory landscape.

Key features

Accessible Design

Movable widget to accommodate both left and right hand individuals

Switch between light & dark modes

Goal Setting

AI Chat

How does our AI model work?

This AI model combines a Large Language Model (LLM) with Retrieval Augmented Generation (RAG) to deliver accurate, context-specific responses. It retrieves data from Cognitive Search and AI Search systems, each sourcing from separate, structured datasets to maintain confidentiality and prevent data mixing. This ensures secure handling of sensitive information while providing precise, reliable results.

Real-time personalised alerts

ComplAI tailors personalised notifications based on the analyst’s workflow and goals, delivering real-time updates on relevant legal and business changes to ensure compliance and efficiency.

High priority Alert Delivery

ComplAI intelligently detects critical alerts and enhances notifications to ensure analysts immediately notice urgent compliance or security updates.

Recording Justification & Risk Assessment

Analysts must document justifications for each query, but manually tracking thousands of queries daily is overwhelming. ComplAI streamlines this process by providing a justification panel, allowing analysts to log explanations in real-time. Additionally, it assesses query risk before execution, preventing potential compliance issues.

Test Mode

To reduce uncertainty, ComplAI offers a sandbox environment where analysts can test their queries before execution. This ensures accuracy and compliance, giving analysts confidence in their inputs.

Compliance Break Freeze & Report

When a compliance breach occurs, ComplAI freezes the screen and immediately notifies the analyst, eliminating the need for manual backtracking. It also automatically logs all queries, allowing analysts to quickly submit an audit report with the necessary justifications to their auditor.

Audit Trail, Reporting & Sanitize

ComplAI maintains a detailed log of all queries and justifications, ensuring easily retrievable compliance records for audits and legal processing. It visualises query patterns, assesses query quality, and provides actionable recommendations to help analysts refine their approach and improve future queries.

Within the audit trail, analysts can file or review compliance break reports, access mandatory compliance training, and sanitize their workspace by removing any data retrieved from non-compliant queries.

Research

User Interview Insights

view detailed interview insights

User interviews provided valuable insights into the TDNA workflow, helping us understand their processes, challenges, and decision-making. We then synthesized our findings, refining the information to identify key pain points, opportunities, and user needs for optimization.

Fig. Interview finding: A typical workflow of a new TDNA

Research Questions

How might the design of an interface use the affordances of AI to assist Target Digital Network Analysts (TDNAs) in constructing compliant queries, streamlining data analysis, and navigating evolving regulations while minimizing cognitive load?

Benchmarking

Benchmarking became a key tool in understanding how to apply AI’s capabilities effectively. One major hurdle was confidentiality, as we couldn’t directly access the analysts’ interfaces. Despite this, benchmarking provided valuable insights, enabling us to navigate these limitations and refine our approach.

Design Process

"You Can’t Do That" – A Compliance Violation Scenario

Michele, a former Zendian language analyst, transitioned into intelligence analysis amid the Zendian-Macondian war, gradually gaining exposure to technical data. Now investigating Zendian military drone tactics, she uncovers SkyOne drone activity linked to a single IP address on a critical attack date.

Before running a follow-up query, she verifies compliance—but an hour later, her auditor informs her that her query violates compliance policy due to improper structuring. Michele must now sanitize her workspace and file a compliance violation report. Frustrated, despite her strict adherence to training and protocols, she receives guidance from her auditor on how to properly structure queries to prevent future violations.

Persona

Michele, a Zendian language graduate, transitioned to the network analyst team seven months ago after completing a diversity tour in intelligence analysis. While she lacks a background in global telecommunications, her extensive intelligence career has given her familiarity with high-level concepts, though she is not highly tech-savvy.

As is User Journey Map

We created a user flow and journey map to understand our persona’s workflow, pain points, and compliance challenges. It revealed that analysts value task complexity but need AI support for efficiency, with compliance interpretations varying by experience. Key challenges included aligning compliance perspectives, balancing automation, and navigating confidentiality constraints.

Pain points addressed

Struggles to keep up with the changing laws and policies

Cannot think outside the box, queries tend to be overly cautious

Time consuming & exhausting breach report process

Ideation

Our group held engaging brainstorming sessions, exploring endless possibilities with whiteboards, sticky notes, and an AI ideation card deck. These sessions were both fun and productive, helping us think outside the box and generate creative solutions.

To be User Flow: ComplAI to the rescue

We designed Michelle’s user flow with ComplAI, identifying key points where she struggles most. This allowed us to map feature integrations that enhance her workflow and ensure our solution effectively supports her needs

Initial sketches, Wireframes and Thoughts

So many ideas!

Explored the idea: What if compliance became a system setting?

Critique and usability testing

This project pushed me to grow as a designer in ways I never expected. I gained a deeper understanding of AI, how it integrates into UX, and what the future of AI-powered design could look like. My technical skills improved significantly, especially in Figma, but more importantly, I refined my design thinking, problem-solving, and collaboration skills. Navigating a complex workflow and working through team challenges taught me the importance of communication, adaptability, and leadership. More than anything, this experience reinforced that design is an ongoing learning process, and each challenge is an opportunity to grow.

Reflection

What I learnt

This project pushed me to grow as a designer in ways I never expected. I gained a deeper understanding of AI, how it integrates into UX, and what the future of AI-powered design could look like. My technical skills improved significantly, especially in Figma, but more importantly, I refined my design thinking, problem-solving, and collaboration skills. Navigating a complex workflow and working through team challenges taught me the importance of communication, adaptability, and leadership. More than anything, this experience reinforced that design is an ongoing learning process, and each challenge is an opportunity to grow.

What’s next for this case study

User Testing and Validation: Conduct testing with TDNAs to evaluate the effectiveness of compliance assistance features and identify areas for improvement.
Enhancing Workflow Integration: Optimize the query structuring process and refine real-time compliance alerts for smoother, more intuitive interactions within analysts’ workflows.
Audit Trail Improvements: Strengthen the audit trail and justification logging to streamline reporting, making compliance tracking more efficient and transparent.

Let’s create your next big idea!