PII-Constrained Scenario Generation System

AI-Driven Workflow Architecture & Systems Design

Context

Our team was contracted to build a dataset of call-center conversations that contained realistic, naturally occurring Personally Identifiable Information (PII). The client needed these conversations for evaluating an AI system’s ability to detect sensitive information across multiple domains such as healthcare, finance, telecom, retail, travel, and more.

This dataset came with unusually strict and conflicting requirements:

  • Every conversation needed to contain specific types and amounts of PII.

  • PII had to emerge naturally, not in a forced or scripted way.

  • The dataset had to cover 7 different business domains.

  • It needed representation of 42 PII categories, balanced across the corpus.

  • We needed ~240 final conversations across three length tiers (short, medium, long).

  • Conversations had to feel believable, motivated, and human — not robotic or repetitive.

  • And it all had to be created fast — far faster than human writers or vendors could deliver.

Manually writing these conversations was not feasible. Human improvisation produced inconsistent quality; vendor-generated content was too slow and often incorrect; and naive prompting of ChatGPT produced stiff, repetitive, or unrealistic outputs.

To meet the client’s constraints and the project timeline, I designed an AI-driven scenario generation system that automated the creation of hundreds of high-quality scenario seeds. These scenario seeds became the foundation for the final conversations, guiding human participants toward naturalistic, domain-appropriate PII disclosure.

This case study focuses on how I architected that system.

1. Mapping PII Likelihood Across Domains

I began by analyzing which PII types realistically occur in each business domain. This included:

  • Identifying high-, medium-, and low-likelihood PII types per domain

  • Using linguistic analysis (plus LLM exploration) to detect natural patterns

  • Building a Domain × PII matrix that defined valid combinations

This ensured scenarios remained plausible, not forced.

2. Scenario Blueprint Architecture

Next, I created a structured “scenario blueprint” system using Google Sheets.
Each row represented a scenario configuration; each column controlled a key dimension:

  • Domain

  • Relevant PII types

  • Length tier (short / medium / long)

  • Caller and agent roles

  • Conversational goal

  • Tone and emotional framing

  • Constraints around naturalness (“allow PII to arise contextually; do not force it”)

This gave me modular, mix-and-match control over scenario variety and constraint handling.

3. Prompt Engineering Through Google Sheets

I built an API-style prompt engine inside Google Sheets using CONCAT formulas.

Each column generated a prompt fragment, and a final column assembled them into full prompts such as:

“Generate 5 plausible call-center scenarios in the [DOMAIN] domain that could naturally involve [RELEVANT PII TYPES], written at [LENGTH TIER]. Focus on realistic motivations, conversational flow, and natural language.”

Because ChatGPT was integrated directly into Sheets, I could generate:

  • 300–500 scenarios programmatically,

  • across hundreds of prompt variations,

  • without manually rewriting anything.

One prompt produced multiple scenarios; we manually audited the best ones.

4. Quality Control & Iteration

Early outputs had issues typical of LLMs:

  • Unrealistic topics

  • Overly formal or stiff language

  • Inconsistent tone

  • Imbalanced PII distribution

To resolve this, I iterated on:

  • Tone instructions

  • Domain/PII pairings

  • Length constraints

  • Role and goal framing

  • Scenario templates

Each refinement meaningfully improved plausibility and variety.

Ultimately, we curated ~250 high-quality scenarios from the generated set. Human participants then used these as guidelines to produce more natural-sounding conversations.

5. Impact

This system:

  • Replaced an unscalable manual writing process

  • Eliminated dependence on vendor-generated content

  • Ensured realistic PII distribution across domains

  • Enabled rapid generation of hundreds of high-quality scenario seeds

  • Delivered a dataset that aligned with strict client constraints

  • Demonstrated how AI workflows can augment (not replace) human creativity

This project is a strong example of my approach to AI-driven workflow design: combining linguistic insight, prompt engineering, and systems thinking to build scalable solutions under complexity and constraint.