Call Analysis Pipeline

Build a workflow that processes audio recordings through transcription, translation, PII removal, summarization, and generates recommendations based on configurable criteria.

What You’ll Build

graph TD
    subgraph "Transcription"
        A[Audio Input] --> B[STT + Diarization]
    end

    subgraph "Processing"
        B --> C[LLM: Translate]
        C --> D[LLM: Remove PII]
        D --> E[LLM: Summarize]
    end

    subgraph "Analysis"
        E --> G[LLM: Generate Recommendations]
        F[Evaluation Criteria] -.->|context| G
    end

    subgraph "Output"
        G --> H[Results Dataset]
    end

Use cases:

Call center quality analysis
Compliance review of recorded conversations
Interview evaluation and scoring
Customer feedback analysis
Sales call coaching

Prerequisites

Before building this workflow, you’ll need:

Component	Description	Example
STT Model	Speech-to-text with diarization	Whisper, WhisperX
LLM Model	For translation, PII removal, summarization, and recommendations	Ollama (Llama, Mistral), or external (GPT-4, Claude)
Criteria Dataset	Evaluation criteria for recommendations	Dataset with scoring rubrics
Output Dataset	Where results will be stored	Empty dataset with appropriate columns

Step 1: Create the Workflow

from seeme import Client

client = Client()

## Create the workflow
workflow = client.create_workflow(
    name="Call Analysis Pipeline",
    description="Process calls: transcribe → translate → remove PII → summarize → recommend"
)

# Get the initial version
version = client.create_workflow_version(
    workflow_id=workflow.id,
    name="v1"
)

print(f"Workflow created: {workflow.id}")
print(f"Version: {version.id}")

Step 2: Add Speech-to-Text with Diarization

The first node transcribes audio and identifies different speakers.

# Speech-to-Text with speaker diarization
stt_node = client.create_workflow_node(
    version_id=version.id,
    name="Transcribe Audio",
    entity_type="model",
    entity_id=stt_model.id,  # Your STT model with diarization
    config={
        "input_template": "{{input}}",
        "timeout": 600,  # 10 minutes for long recordings
        "config": {
            "diarization": True,
            "language": "auto",  # Auto-detect language
            "timestamps": True
        }
    }
)

Expected output format:

{
  "segments": [
    {"speaker": "SPEAKER_00", "text": "Hello, thank you for calling support.", "start": 0.0, "end": 2.5},
    {"speaker": "SPEAKER_01", "text": "Hi, I'm having trouble with my account.", "start": 2.8, "end": 5.1},
    ...
  ],
  "speakers": ["SPEAKER_00", "SPEAKER_01"],
  "language": "en",
  "duration": 324.5
}

Step 3: Add Translation Node

If calls may be in different languages, translate to your target language.

# Translation node
translate_node = client.create_workflow_node(
    version_id=version.id,
    name="Translate to English",
    entity_type="model",
    entity_id=llm_model.id,
    config={
        "input_template": """
You are a professional translator. Translate the following conversation transcript to English.

**Important rules:**
- Preserve speaker labels (SPEAKER_00, SPEAKER_01, etc.)
- Maintain the original meaning and tone
- Keep timestamps if present
- If already in English, return the transcript unchanged

**Transcript:**
{{Transcribe Audio}}

**Translated transcript:**
""",
        "config": {
            "temperature": 0.1,  # Low temperature for consistent translation
            "max_tokens": 8000
        }
    }
)

# Connect STT → Translation
client.create_workflow_edge(
    version_id=version.id,
    begin_node_id=stt_node.id,
    end_node_id=translate_node.id,
    edge_type="data"
)

Step 4: Add PII Removal Node

Remove or redact personally identifiable information for privacy/compliance.

# PII removal node
pii_node = client.create_workflow_node(
    version_id=version.id,
    name="Remove PII",
    entity_type="model",
    entity_id=llm_model.id,
    config={
        "input_template": """
You are a data privacy specialist. Remove or redact all Personally Identifiable Information (PII) from this transcript.

**PII to redact:**
- Full names → [NAME]
- Phone numbers → [PHONE]
- Email addresses → [EMAIL]
- Physical addresses → [ADDRESS]
- Social security numbers → [SSN]
- Credit card numbers → [CREDIT_CARD]
- Account numbers → [ACCOUNT_NUMBER]
- Dates of birth → [DOB]
- Any other identifying information → [REDACTED]

**Important:**
- Keep speaker labels intact (SPEAKER_00, etc.)
- Preserve the conversation flow and context
- Do NOT redact company names, product names, or general locations (cities, countries)
- Do NOT change anything else - only replace PII

**Original transcript:**
{{Translate to English}}

**Redacted transcript:**
""",
        "config": {
            "temperature": 0.0,  # Zero temperature for consistent redaction
            "max_tokens": 8000
        }
    }
)

# Connect Translation → PII Removal
client.create_workflow_edge(
    version_id=version.id,
    begin_node_id=translate_node.id,
    end_node_id=pii_node.id,
    edge_type="data"
)

Step 5: Add Summarization Node

Create a structured summary of the conversation.

# Summarization node
summary_node = client.create_workflow_node(
    version_id=version.id,
    name="Summarize Conversation",
    entity_type="model",
    entity_id=llm_model.id,
    config={
        "input_template": """
Analyze this conversation and create a structured summary.

**Transcript:**
{{Remove PII}}

**Provide the summary in this exact JSON format:**
```json
{
  "call_type": "support|sales|complaint|inquiry|other",
  "duration_category": "short|medium|long",
  "participants": {
    "customer": "brief description of customer situation",
    "agent": "agent name if mentioned, otherwise 'Agent'"
  },
  "main_topic": "one sentence describing the main topic",
  "key_points": [
    "first key point",
    "second key point",
    "third key point"
  ],
  "customer_sentiment": "positive|neutral|negative|mixed",
  "resolution_status": "resolved|unresolved|escalated|pending",
  "action_items": [
    "action item 1",
    "action item 2"
  ],
  "summary": "2-3 sentence executive summary"
}

Return only valid JSON, no additional text. “”", “config”: { “temperature”: 0.2, “max_tokens”: 2000 } } )

Connect PII Removal → Summary

client.create_workflow_edge( version_id=version.id, begin_node_id=pii_node.id, end_node_id=summary_node.id, edge_type=“data” )


## Step 6: Set Up Evaluation Criteria Dataset

Create a dataset containing your evaluation criteria. This provides context for the recommendation node.

```python
# Create criteria dataset (one-time setup)
criteria_dataset = client.create_dataset(
    name="Call Evaluation Criteria",
    description="Scoring rubrics for call analysis"
)

criteria_version = client.create_dataset_version(
    dataset_id=criteria_dataset.id,
    name="v1"
)

# Add evaluation criteria
criteria_items = [
    {
        "category": "Communication",
        "criteria": "Clear and professional communication",
        "scoring": "5: Excellent - Clear, concise, professional throughout. 4: Good - Mostly clear with minor issues. 3: Average - Some unclear communication. 2: Below average - Frequently unclear. 1: Poor - Difficult to understand.",
        "weight": 0.25
    },
    {
        "category": "Problem Resolution",
        "criteria": "Effectively resolved customer issue",
        "scoring": "5: Fully resolved with customer satisfaction. 4: Resolved with minor follow-up needed. 3: Partially resolved. 2: Issue remains mostly unresolved. 1: No resolution achieved.",
        "weight": 0.30
    },
    {
        "category": "Empathy",
        "criteria": "Showed understanding and empathy",
        "scoring": "5: Exceptional empathy and rapport. 4: Good empathy demonstrated. 3: Adequate but could improve. 2: Limited empathy shown. 1: No empathy demonstrated.",
        "weight": 0.20
    },
    {
        "category": "Process Adherence",
        "criteria": "Followed required procedures",
        "scoring": "5: Perfect adherence. 4: Minor deviations. 3: Some procedures missed. 2: Multiple procedures missed. 1: Major compliance issues.",
        "weight": 0.15
    },
    {
        "category": "Efficiency",
        "criteria": "Handled call efficiently without unnecessary delays",
        "scoring": "5: Highly efficient. 4: Good pace. 3: Average efficiency. 2: Some unnecessary delays. 1: Very inefficient.",
        "weight": 0.10
    }
]

for item in criteria_items:
    client.create_dataset_item(
        version_id=criteria_version.id,
        data=item
    )

Step 7: Add Criteria Context Node

Add the evaluation criteria as context for the recommendation node.

# Criteria context node
criteria_node = client.create_workflow_node(
    version_id=version.id,
    name="Evaluation Criteria",
    entity_type="dataset",
    entity_id=criteria_dataset.id,
    config={
        "context_config": {
            "dataset_version_id": criteria_version.id,
            "field_mapping": {
                "category": "category",
                "criteria": "criteria",
                "scoring": "scoring",
                "weight": "weight"
            },
            "context_name": "evaluation_criteria"
        }
    }
)

Step 8: Add Recommendations Node

Generate scored recommendations based on the summary and criteria.

# Recommendations node
recommend_node = client.create_workflow_node(
    version_id=version.id,
    name="Generate Recommendations",
    entity_type="model",
    entity_id=llm_model.id,
    config={
        "input_template": """
You are a call center quality analyst. Evaluate this call and provide recommendations.

**Call Summary:**
{{Summarize Conversation}}

**Redacted Transcript (for reference):**
{{Remove PII}}

**Evaluation Criteria:**
{{#each evaluation_criteria}}
### {{category}} (Weight: {{weight}})
{{criteria}}
{{scoring}}

{{/each}}

**Your task:**
1. Score each category (1-5) based on the criteria
2. Calculate weighted total score
3. Provide specific, actionable recommendations

**Return your evaluation in this exact JSON format:**
```json
{
  "scores": {
    "Communication": {"score": X, "reasoning": "brief explanation"},
    "Problem Resolution": {"score": X, "reasoning": "brief explanation"},
    "Empathy": {"score": X, "reasoning": "brief explanation"},
    "Process Adherence": {"score": X, "reasoning": "brief explanation"},
    "Efficiency": {"score": X, "reasoning": "brief explanation"}
  },
  "weighted_total": X.XX,
  "grade": "A|B|C|D|F",
  "strengths": [
    "strength 1",
    "strength 2"
  ],
  "areas_for_improvement": [
    "improvement area 1",
    "improvement area 2"
  ],
  "recommendations": [
    {
      "priority": "high|medium|low",
      "recommendation": "specific actionable recommendation",
      "expected_impact": "what improvement this would bring"
    }
  ],
  "coaching_notes": "2-3 sentences of coaching feedback for the agent",
  "follow_up_required": true|false,
  "follow_up_reason": "reason if follow-up required, null otherwise"
}

Grade scale: A (4.5-5.0), B (3.5-4.49), C (2.5-3.49), D (1.5-2.49), F (below 1.5)

Return only valid JSON. “”", “config”: { “temperature”: 0.3, “max_tokens”: 3000 } } )

Connect Summary → Recommendations (data edge)

client.create_workflow_edge( version_id=version.id, begin_node_id=summary_node.id, end_node_id=recommend_node.id, edge_type=“data” )

Connect PII-removed transcript → Recommendations (data edge for reference)

client.create_workflow_edge( version_id=version.id, begin_node_id=pii_node.id, end_node_id=recommend_node.id, edge_type=“data” )

Connect Criteria → Recommendations (context edge)

client.create_workflow_edge( version_id=version.id, begin_node_id=criteria_node.id, end_node_id=recommend_node.id, edge_type=“context” )


## Step 9: Add Output Dataset Node

Store all results in an output dataset.

```python
# Create output dataset (one-time setup)
output_dataset = client.create_dataset(
    name="Call Analysis Results",
    description="Processed call analysis with recommendations"
)

output_version = client.create_dataset_version(
    dataset_id=output_dataset.id,
    name="v1"
)

# Output node
output_node = client.create_workflow_node(
    version_id=version.id,
    name="Store Results",
    entity_type="dataset",
    entity_id=output_dataset.id,
    config={
        "output_dataset_id": output_dataset.id,
        "output_version_id": output_version.id,
        "column_mapping": {
            "original_audio": "{{input}}",
            "transcript": "{{Transcribe Audio}}",
            "translated_transcript": "{{Translate to English}}",
            "redacted_transcript": "{{Remove PII}}",
            "summary": "{{Summarize Conversation}}",
            "recommendations": "{{Generate Recommendations}}",
            "processed_at": "{{timestamp}}"
        }
    }
)

# Connect Recommendations → Output
client.create_workflow_edge(
    version_id=version.id,
    begin_node_id=recommend_node.id,
    end_node_id=output_node.id,
    edge_type="data"
)

Complete Workflow Code

Here’s the full workflow creation in one script:

from seeme import Client

client = Client()

# --- Prerequisites: Get your models ---
stt_model = client.get_model("your-stt-model-id")  # WhisperX or similar
llm_model = client.get_model("your-llm-model-id")  # Ollama, GPT-4, etc.

# --- Create workflow ---
workflow = client.create_workflow(
    name="Call Analysis Pipeline",
    description="Process calls: transcribe → translate → remove PII → summarize → recommend"
)
version = client.create_workflow_version(workflow_id=workflow.id, name="v1")

# --- Node 1: STT with Diarization ---
stt_node = client.create_workflow_node(
    version_id=version.id,
    name="Transcribe Audio",
    entity_type="model",
    entity_id=stt_model.id,
    config={
        "input_template": "{{input}}",
        "timeout": 600,
        "config": {"diarization": True, "language": "auto", "timestamps": True}
    }
)

# --- Node 2: Translation ---
translate_node = client.create_workflow_node(
    version_id=version.id,
    name="Translate to English",
    entity_type="model",
    entity_id=llm_model.id,
    config={
        "input_template": """Translate this conversation to English. Preserve speaker labels. If already in English, return unchanged.

Transcript:
{{Transcribe Audio}}

Translated transcript:""",
        "config": {"temperature": 0.1, "max_tokens": 8000}
    }
)

# --- Node 3: PII Removal ---
pii_node = client.create_workflow_node(
    version_id=version.id,
    name="Remove PII",
    entity_type="model",
    entity_id=llm_model.id,
    config={
        "input_template": """Remove PII from this transcript. Replace names with [NAME], phones with [PHONE], emails with [EMAIL], addresses with [ADDRESS], account numbers with [ACCOUNT_NUMBER]. Keep speaker labels and conversation flow intact.

Transcript:
{{Translate to English}}

Redacted transcript:""",
        "config": {"temperature": 0.0, "max_tokens": 8000}
    }
)

# --- Node 4: Summarization ---
summary_node = client.create_workflow_node(
    version_id=version.id,
    name="Summarize Conversation",
    entity_type="model",
    entity_id=llm_model.id,
    config={
        "input_template": """Analyze this conversation and return a JSON summary with: call_type, main_topic, key_points (array), customer_sentiment, resolution_status, action_items (array), summary (2-3 sentences).

Transcript:
{{Remove PII}}

JSON summary:""",
        "config": {"temperature": 0.2, "max_tokens": 2000}
    }
)

# --- Node 5: Evaluation Criteria (context) ---
criteria_node = client.create_workflow_node(
    version_id=version.id,
    name="Evaluation Criteria",
    entity_type="dataset",
    entity_id=criteria_dataset.id,
    config={
        "context_config": {
            "dataset_version_id": criteria_version.id,
            "context_name": "evaluation_criteria"
        }
    }
)

# --- Node 6: Recommendations ---
recommend_node = client.create_workflow_node(
    version_id=version.id,
    name="Generate Recommendations",
    entity_type="model",
    entity_id=llm_model.id,
    config={
        "input_template": """Evaluate this call based on the criteria. Return JSON with: scores (per category with reasoning), weighted_total, grade (A-F), strengths, areas_for_improvement, recommendations (with priority and expected_impact), coaching_notes.

Summary: {{Summarize Conversation}}
Transcript: {{Remove PII}}

Criteria:
{{#each evaluation_criteria}}
{{category}} ({{weight}}): {{scoring}}
{{/each}}

Evaluation JSON:""",
        "config": {"temperature": 0.3, "max_tokens": 3000}
    }
)

# --- Node 7: Output Dataset ---
output_node = client.create_workflow_node(
    version_id=version.id,
    name="Store Results",
    entity_type="dataset",
    entity_id=output_dataset.id,
    config={
        "output_dataset_id": output_dataset.id,
        "column_mapping": {
            "transcript": "{{Transcribe Audio}}",
            "redacted_transcript": "{{Remove PII}}",
            "summary": "{{Summarize Conversation}}",
            "recommendations": "{{Generate Recommendations}}"
        }
    }
)

# --- Connect all nodes ---
edges = [
    (stt_node.id, translate_node.id, "data"),
    (translate_node.id, pii_node.id, "data"),
    (pii_node.id, summary_node.id, "data"),
    (summary_node.id, recommend_node.id, "data"),
    (pii_node.id, recommend_node.id, "data"),  # Transcript reference
    (criteria_node.id, recommend_node.id, "context"),  # Criteria context
    (recommend_node.id, output_node.id, "data"),
]

for begin_id, end_id, edge_type in edges:
    client.create_workflow_edge(
        version_id=version.id,
        begin_node_id=begin_id,
        end_node_id=end_id,
        edge_type=edge_type
    )

print(f"Workflow ready: {workflow.id}")

Step 10: Execute the Workflow

Single Call Analysis

# Process a single audio file
execution = client.execute_workflow(
    workflow_id=workflow.id,
    input_mode="single",
    item="./calls/customer_call_001.mp3"
)

# Monitor progress
import time
while execution.status in ["pending", "running"]:
    time.sleep(10)
    execution = client.get_workflow_execution(
        workflow_id=workflow.id,
        execution_id=execution.id
    )
    print(f"Status: {execution.status}, Progress: {execution.progress}%")

# Get results
print("\n=== RESULTS ===")
print(f"Summary: {execution.results['Summarize Conversation']}")
print(f"Recommendations: {execution.results['Generate Recommendations']}")

Batch Processing

Process multiple calls at once:

import glob

# Get all audio files
audio_files = glob.glob("./calls/*.mp3")

# Execute in batch mode
execution = client.execute_workflow(
    workflow_id=workflow.id,
    input_mode="batch",
    batch_config={
        "file_paths": audio_files,
        "parallelism": 5,  # Process 5 calls simultaneously
        "continue_on_error": True
    }
)

# Monitor batch progress
while execution.status in ["pending", "running"]:
    time.sleep(30)
    execution = client.get_workflow_execution(
        workflow_id=workflow.id,
        execution_id=execution.id
    )
    print(f"Progress: {execution.completed}/{execution.total} calls processed")

print(f"Batch complete: {execution.completed} succeeded, {execution.failed} failed")

Workflow Diagram

The final workflow looks like this:

graph TD
    A[Audio File] --> B[Transcribe Audio
STT + Diarization]
    B --> C[Translate to English
LLM]
    C --> D[Remove PII
LLM]
    D --> E[Summarize Conversation
LLM]
    D --> F[Generate Recommendations
LLM]
    E --> F
    G[Evaluation Criteria
Dataset] -.->|context| F
    F --> H[Store Results
Output Dataset]

    style A fill:#e0f2fe
    style B fill:#fef3c7
    style C fill:#fef3c7
    style D fill:#fce7f3
    style E fill:#d1fae5
    style F fill:#ede9fe
    style G fill:#f3f4f6
    style H fill:#e0f2fe

Customization Options

Skip Translation

If all calls are in the same language:

# Remove translate_node and connect STT directly to PII
client.create_workflow_edge(
    version_id=version.id,
    begin_node_id=stt_node.id,
    end_node_id=pii_node.id,
    edge_type="data"
)

Add Sentiment Analysis

Add a dedicated sentiment model before recommendations:

sentiment_node = client.create_workflow_node(
    version_id=version.id,
    name="Analyze Sentiment",
    entity_type="model",
    entity_id=sentiment_model.id,
    config={"input_template": "{{Remove PII}}"}
)

Custom Evaluation Criteria

Modify the criteria dataset for your specific use case:

Use Case	Criteria Categories
Sales calls	Rapport, Product Knowledge, Objection Handling, Closing
Support calls	Communication, Problem Resolution, Empathy, Efficiency
Compliance	Script Adherence, Disclosure Accuracy, Documentation
Healthcare	HIPAA Compliance, Clinical Accuracy, Patient Empathy

Best Practices

Use low temperatures for PII removal and translation (0.0-0.1) for consistency
Set appropriate timeouts - long calls may take minutes to transcribe
Review PII removal periodically to ensure nothing is missed
Version your criteria - update weights and scoring as you learn
Sample and validate - check a percentage of results manually
Use batch processing with parallelism for large volumes

Troubleshooting

Issue	Solution
Transcription fails on long files	Increase timeout, or split audio into chunks
PII still appearing	Add specific PII patterns to the prompt, or add a second PII pass
Low-quality recommendations	Review criteria dataset, ensure weights sum to 1.0
JSON parsing errors	Add “Return only valid JSON” to prompts, lower temperature
Slow processing	Increase batch parallelism, use faster models for simpler tasks

Document Processing Pipeline