Automate TikTok & Loom Transcripts to CRM: Webhooks & Pipelines

Unlocking Unstructured Video Data at Scale

Video has become the primary medium for asynchronous business communication and consumer engagement. Loom hosts millions of sales walkthroughs, bug reports, and customer onboarding sessions, while TikTok serves as a real-time repository of consumer trends, competitor messaging, and brand sentiment.

However, video files represent an opaque, unstructured data silo. Without automated transcription pipelines, enterprise intelligence engines cannot parse keyword triggers, summarize call notes into CRMs, or aggregate market feedback.

The TikTok & Loom Video AI Transcript Extractor - Audio (madeingermany/tiktok-loom-transcript-extractor) converts raw audio streams from TikTok and Loom videos into rich, timestamped textual records. By pairing this cloud actor with automated webhooks and orchestration workflows, you can build a resilient event-driven data pipeline that deposits extracted transcripts directly into data warehouses like PostgreSQL, knowledge bases like Airtable, or CRM suites like HubSpot.

Architecture: Event-Driven Transcript Ingestion Pipeline

A production-grade transcription pipeline requires automated triggers, scalable extraction, and reliable downstream delivery. Rather than continuously polling API endpoints, an event-driven webhook architecture minimizes compute overhead and latency.

+--------------------------------------------------------+
| 1. Ingestion Triggers                                  |
|    - Scheduled CRON Jobs (Daily Competitor Audit)      |
|    - CRM Trigger (HubSpot Deal Stage: "Demo Recorded") |
+---------------------------+----------------------------+
                            |
                            v
+--------------------------------------------------------+
| 2. Cloud Extraction (Apify Cloud)                     |
|    - Actor: madeingermany/tiktok-loom-transcript-extractor
|    - Automated Browser Emulation & Media Demuxing     |
|    - AI-Powered Speech-to-Text Transcription           |
+---------------------------+----------------------------+
                            | (HTTP POST Webhook on SUCCEEDED)
                            v
+--------------------------------------------------------+
| 3. Ingestion Microservice / Webhook Gateway           |
|    - FastAPI / Express Endpoint                        |
|    - Payload Validation & Normalization               |
+---------------------------+----------------------------+
                            |
        +-------------------+-------------------+
        v                                       v
+-----------------------+           +-----------------------+
| 4A. Analytical Storage|           | 4B. Operational Tools |
|     - PostgreSQL      |           |     - HubSpot Notes   |
|     - Snowflake       |           |     - Airtable Base   |
+-----------------------+           +-----------------------+

1. Trigger: An operational trigger (such as a sales representative attaching a Loom URL to a deal, or a scheduled social monitoring script) dispatches an execution request to Apify via the REST API or SDK.
2. Extraction: The actor spins up in the Apify Cloud, demuxes audio streams, runs speech-to-text inference, and structures the transcript.
3. Webhook Dispatch: Upon run completion (ACTOR.RUN.SUCCEEDED), Apify sends an HTTP POST payload to your webhook listener.
4. Data Sinks: Your ingestion service reads the extracted records and appends them to PostgreSQL for long-term analytics or updates the matching contact or lead profile in your CRM.

Extracted Data Schema

The actor formats extracted video records into normalized JSON. Below is the standard schema available in the resulting dataset:

Programmatic Execution with Python

The following script uses apify-client to trigger the extractor programmatically, configuring a webhook directly within the run options so that Apify notifies your backend the moment the transcription finishes.

import os
from apify_client import ApifyClient

# Initialize the ApifyClient with your API token
client = ApifyClient(os.getenv("APIFY_TOKEN", "YOUR_APIFY_TOKEN"))

# Define input parameters containing target video URLs
run_input = {
    "videoUrls": [
        "https://www.loom.com/share/example-demo-recording-id",
        "https://www.tiktok.com/@examplecreator/video/1234567890123456789"
    ],
    "language": "auto",
    "includeTimestamps": True
}

# Configure a webhook to be invoked immediately when the actor run succeeds
webhooks = [
    {
        "eventTypes": ["ACTOR.RUN.SUCCEEDED"],
        "requestUrl": "https://api.yourdomain.com/v1/webhooks/transcripts",
        "payloadTemplate": '{"actorRunId": {{userId}}, "datasetId": {{defaultDatasetId}}}'
    }
]

print("Dispatching transcript extraction job to Apify Cloud...")

# Start the actor run asynchronously
actor_run = client.actor("madeingermany/tiktok-loom-transcript-extractor").call(
    run_input=run_input,
    webhooks=webhooks
)

print(f"Run initiated. Run ID: {actor_run['id']}")
print(f"Default Dataset ID: {actor_run['defaultDatasetId']}")

Programmatic Execution with Node.js

For TypeScript and Node.js environments, the official apify-client provides equivalent functionality. This pattern is ideal for integrating into NestJS, Next.js server actions, or Express services.

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
    token: process.env.APIFY_TOKEN || 'YOUR_APIFY_TOKEN',
});

async function triggerTranscriptPipeline() {
    const runInput = {
        videoUrls: [
            'https://www.loom.com/share/abc123def456-walkthrough',
            'https://www.tiktok.com/@brand/video/9876543210987654321'
        ],
        language: 'auto',
        includeTimestamps: true
    };

    console.log('Initiating Apify Actor execution...');

    const run = await client.actor('madeingermany/tiktok-loom-transcript-extractor').call(runInput, {
        webhooks: [
            {
                eventTypes: ['ACTOR.RUN.SUCCEEDED'],
                requestUrl: 'https://api.yourdomain.com/v1/webhooks/transcripts',
                payloadTemplate: JSON.stringify({
                    datasetId: '{{defaultDatasetId}}',
                    runId: '{{id}}'
                })
            }
        ]
    });

    console.log(`Execution completed. Dataset ID: ${run.defaultDatasetId}`);
}

triggerTranscriptPipeline().catch(console.error);

Webhook Consumer: Piping Records into PostgreSQL and CRMs

Below is an end-to-end FastAPI webhook consumer designed to receive Apify’s completion event, fetch the structured records from the dataset, and stream them into a PostgreSQL database and an external CRM (such as HubSpot or Airtable).

import os
import requests
from fastapi import FastAPI, HTTPException, Request
from apify_client import ApifyClient
import psycopg2
from psycopg2.extras import execute_batch

app = FastAPI(title="Apify Transcript Ingestion Gateway")

APIFY_TOKEN = os.getenv("APIFY_TOKEN")
DATABASE_URL = os.getenv("DATABASE_URL")
HUBSPOT_ACCESS_TOKEN = os.getenv("HUBSPOT_ACCESS_TOKEN")

apify_client = ApifyClient(APIFY_TOKEN)

def get_db_connection():
    return psycopg2.connect(DATABASE_URL)

@app.post("/v1/webhooks/transcripts")
async def handle_apify_webhook(request: Request):
    payload = await request.json()
    dataset_id = payload.get("datasetId")
    
    if not dataset_id:
        raise HTTPException(status_code=400, detail="Missing datasetId in webhook payload")

    # Fetch all items from the Apify default dataset
    dataset_client = apify_client.dataset(dataset_id)
    items = dataset_client.list_items().items

    if not items:
        return {"status": "no_data_extracted", "datasetId": dataset_id}

    # 1. Pipe records into PostgreSQL
    conn = get_db_connection()
    cursor = conn.cursor()
    
    insert_query = """
        INSERT INTO video_transcripts (
            source_url, platform, video_title, duration, 
            full_transcript, detected_language, scraped_at
        ) VALUES (
            %(url)s, %(platform)s, %(videoTitle)s, %(duration)s, 
            %(fullTranscript)s, %(detectedLanguage)s, %(scrapedAt)s
        )
        ON CONFLICT (source_url) DO UPDATE SET
            full_transcript = EXCLUDED.full_transcript,
            scraped_at = EXCLUDED.scraped_at;
    """
    
    execute_batch(cursor, insert_query, items)
    conn.commit()
    cursor.close()
    conn.close()

    # 2. Pipe to CRM (Example: HubSpot Deal/Contact Note Attachment)
    if HUBSPOT_ACCESS_TOKEN:
        for item in items:
            if item.get("platform") == "loom":
                sync_loom_to_hubspot(item)

    return {"status": "success", "processed_records": len(items)}

def sync_loom_to_hubspot(item: dict):
    """Creates a CRM note entry containing the transcript summary."""
    headers = {
        "Authorization": f"Bearer {HUBSPOT_ACCESS_TOKEN}",
        "Content-Type": "application/json"
    }
    note_payload = {
        "properties": {
            "hs_timestamp": item.get("scrapedAt"),
            "hs_note_body": f"Loom Walkthrough Transcript ({item.get('videoTitle')}):\n\n{item.get('fullTranscript')