{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Netflix Content Analytics — Coding Challenge\n",
    "\n",
    "## Scenario\n",
    "\n",
    "You're a data scientist on Netflix's Content Analytics team. The product team ran an initiative in Q1 2026 aimed at improving how often users complete the content they start watching.\n",
    "\n",
    "You have a dataset of ~215,000 viewing sessions across Q4 2025 and Q1 2026. Explore the data and evaluate whether the initiative improved completion rates.\n",
    "\n",
    "---\n",
    "**Dataset:** generated by the setup cell below — nothing to download.  \n",
    "**Key columns:** `quarter`, `watch_time_minutes`, `completed`, `genre`, `device_type`, `content_type`, `user_age_group`, `subscription_tier`"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# ---------------------------------------------------------------------------\n",
    "# Setup. Running this cell generates the dataset — nothing to download.\n",
    "# Synthetic data, seeded, so everyone gets identical numbers.\n",
    "# ---------------------------------------------------------------------------\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set_theme(style='whitegrid')\n",
    "%matplotlib inline\n",
    "\n",
    "import pandas as pd\n",
    "np.random.seed(42)\n",
    "# ---------------------------------------------------------------------------\n",
    "# Data parameters\n",
    "# ---------------------------------------------------------------------------\n",
    "SESSIONS = {\n",
    "    \"Q4_2025\": 100_000,\n",
    "    \"Q1_2026\": 115_000,\n",
    "}\n",
    "# Genre mix: Documentary jumps from 5% → 35% in Q1 (Netflix pushed non-fiction content).\n",
    "# This mix shift is the root cause of the aggregate paradox.\n",
    "GENRE_DIST = {\n",
    "    \"Q4_2025\": {\"Drama\": 0.30, \"Comedy\": 0.25, \"Action\": 0.25, \"Romance\": 0.15, \"Documentary\": 0.05},\n",
    "    \"Q1_2026\": {\"Drama\": 0.20, \"Comedy\": 0.18, \"Action\": 0.17, \"Romance\": 0.10, \"Documentary\": 0.35},\n",
    "}\n",
    "# Every genre improves — the product initiative genuinely worked at the genre level.\n",
    "# But Documentary's baseline rate (43–46%) drags down the aggregate when it dominates Q1.\n",
    "COMPLETION_RATES = {\n",
    "    \"Drama\":       {\"Q4_2025\": 0.72, \"Q1_2026\": 0.74},\n",
    "    \"Comedy\":      {\"Q4_2025\": 0.71, \"Q1_2026\": 0.73},\n",
    "    \"Action\":      {\"Q4_2025\": 0.75, \"Q1_2026\": 0.76},\n",
    "    \"Romance\":     {\"Q4_2025\": 0.70, \"Q1_2026\": 0.72},\n",
    "    \"Documentary\": {\"Q4_2025\": 0.43, \"Q1_2026\": 0.46},\n",
    "}\n",
    "# Documentaries are much longer on average (~95 min vs ~50 min for others),\n",
    "# which also drags down aggregate average watch time despite per-genre improvement.\n",
    "WATCH_TIME_PARAMS = {\n",
    "    \"Drama\":       {\"mean\": 48, \"std\": 14},\n",
    "    \"Comedy\":      {\"mean\": 52, \"std\": 16},\n",
    "    \"Action\":      {\"mean\": 55, \"std\": 17},\n",
    "    \"Romance\":     {\"mean\": 50, \"std\": 15},\n",
    "    \"Documentary\": {\"mean\": 95, \"std\": 29},\n",
    "}\n",
    "DEVICE_TYPES  = [\"TV\", \"Mobile\", \"Tablet\", \"Desktop\"]\n",
    "DEVICE_PROBS  = [0.50, 0.30, 0.10, 0.10]\n",
    "CONTENT_TYPES = [\"Movie\", \"Series\"]\n",
    "CONTENT_PROBS = [0.40, 0.60]\n",
    "AGE_GROUPS    = [\"18-24\", \"25-34\", \"35-44\", \"45-54\", \"55+\"]\n",
    "SUB_TIERS     = [\"Basic\", \"Standard\", \"Premium\"]\n",
    "SUB_PROBS     = [0.30, 0.50, 0.20]\n",
    "# ---------------------------------------------------------------------------\n",
    "# Generate rows\n",
    "# ---------------------------------------------------------------------------\n",
    "rows = []\n",
    "session_counter = 1\n",
    "for quarter, n_sessions in SESSIONS.items():\n",
    "    genres      = list(GENRE_DIST[quarter].keys())\n",
    "    genre_probs = list(GENRE_DIST[quarter].values())\n",
    "    genre_arr   = np.random.choice(genres, size=n_sessions, p=genre_probs)\n",
    "    device_arr  = np.random.choice(DEVICE_TYPES, size=n_sessions, p=DEVICE_PROBS)\n",
    "    content_arr = np.random.choice(CONTENT_TYPES, size=n_sessions, p=CONTENT_PROBS)\n",
    "    age_arr     = np.random.choice(AGE_GROUPS, size=n_sessions)\n",
    "    tier_arr    = np.random.choice(SUB_TIERS, size=n_sessions, p=SUB_PROBS)\n",
    "    # ~10 sessions per user on average\n",
    "    n_users        = n_sessions // 10\n",
    "    user_arr       = np.random.randint(1, n_users + 1, size=n_sessions)\n",
    "    content_id_arr = np.random.randint(1, 5001, size=n_sessions)  # pool of 5,000 titles\n",
    "    for i in range(n_sessions):\n",
    "        genre           = genre_arr[i]\n",
    "        completion_rate = COMPLETION_RATES[genre][quarter]\n",
    "        completed       = int(np.random.rand() < completion_rate)\n",
    "        params     = WATCH_TIME_PARAMS[genre]\n",
    "        watch_time = round(max(1.0, np.random.normal(params[\"mean\"], params[\"std\"])), 1)\n",
    "        rows.append({\n",
    "            \"session_id\":         session_counter,\n",
    "            \"user_id\":            f\"u_{user_arr[i]:06d}\",\n",
    "            \"content_id\":         f\"c_{content_id_arr[i]:05d}\",\n",
    "            \"quarter\":            quarter,\n",
    "            \"watch_time_minutes\": watch_time,\n",
    "            \"completed\":          completed,\n",
    "            \"device_type\":        device_arr[i],\n",
    "            \"genre\":              genre,\n",
    "            \"content_type\":       content_arr[i],\n",
    "            \"user_age_group\":     age_arr[i],\n",
    "            \"subscription_tier\":  tier_arr[i],\n",
    "        })\n",
    "        session_counter += 1\n",
    "# ---------------------------------------------------------------------------\n",
    "# Save and summarize\n",
    "# ---------------------------------------------------------------------------\n",
    "df = pd.DataFrame(rows)\n",
    "\n",
    "df = pd.DataFrame(rows)\n",
    "print(df.shape)\n",
    "df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24a08o28bgu",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Question 1: Headline metrics\n",
    "\n",
    "Calculate the following by quarter:\n",
    "- Overall **completion rate**\n",
    "- Average and total **watch time**\n",
    "\n",
    "What story do these top-line numbers tell? Does anything surprise you?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ib40ee29ssr",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Completion rate by quarter\n",
    "df.groupby('quarter')['completed'].mean().round(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "pony7xncikr",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Watch time by quarter\n",
    "df.groupby('quarter')['watch_time_minutes'].agg(\n",
    "    avg_minutes='mean',\n",
    "    total_hours=lambda x: x.sum() / 60\n",
    ").round(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ulr1xw1dzda",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Question 2: Break it down by genre\n",
    "\n",
    "The headline metric may be hiding something. Explore completion rate and genre mix at the genre level:\n",
    "\n",
    "- What is the completion rate **per genre** for each quarter? Did individual genres improve or decline?\n",
    "- How did the **genre mix** (share of sessions per genre) shift between quarters?\n",
    "- What does this tell you about why the aggregate number moved the way it did?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e05vrz18sh5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Completion rate by genre and quarter\n",
    "completion_by_genre = (\n",
    "    df.groupby(['genre', 'quarter'])['completed']\n",
    "    .mean().round(4)\n",
    "    .unstack()\n",
    ")\n",
    "\n",
    "completion_by_genre.sort_values('Q4_2025')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "u2hla72lwik",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Genre mix (share of sessions) by quarter\n",
    "genre_mix = (\n",
    "    df.groupby('quarter')['genre']\n",
    "    .value_counts(normalize=True)\n",
    "    .mul(100).round(1)\n",
    "    .rename('pct')\n",
    "    .reset_index()\n",
    "    .pivot(index='genre', columns='quarter', values='pct')\n",
    ")\n",
    "genre_mix['change'] = (genre_mix['Q1_2026'] - genre_mix['Q4_2025']).round(1)\n",
    "genre_mix.sort_values('change')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "jhqqr1sbeda",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Question 3: Visualize the story\n",
    "\n",
    "Build charts that together tell the Simpson's Paradox story clearly enough for a non-technical stakeholder (e.g., a PM or VP):\n",
    "\n",
    "1. **Overall completion rate by quarter** — the misleading headline\n",
    "2. **Line chart: completion rate by genre** — every genre trended up\n",
    "3. **Grouped bar: completion rate by genre and quarter** — the per-genre improvement\n",
    "4. **Genre mix shift** — what actually changed between quarters\n",
    "\n",
    "What would you say in a slide or readout to explain why the headline metric went down even though the product initiative worked?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "y35xse9h3ir",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Chart 1: Overall completion rate by quarter\n",
    "quarter_order = ['Q4_2025', 'Q1_2026']\n",
    "overall = df.groupby('quarter')['completed'].mean().reindex(quarter_order).reset_index()\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(6, 4))\n",
    "sns.barplot(data=overall, x='quarter', y='completed', order=quarter_order,\n",
    "            hue='quarter', palette='Blues_d', legend=False, ax=ax)\n",
    "ax.set_ylim(0.60, 0.75)\n",
    "ax.set_title('Overall Completion Rate by Quarter', fontsize=14)\n",
    "ax.set_xlabel('')\n",
    "ax.set_ylabel('Completion Rate')\n",
    "for bar, val in zip(ax.patches, overall['completed']):\n",
    "    ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height() + 0.002,\n",
    "            f'{val:.1%}', ha='center', va='bottom', fontsize=11)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "invkd3mvk8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Chart 2: Completion rate by genre over time (line chart)\n",
    "quarter_order = ['Q4_2025', 'Q1_2026']\n",
    "genre_comp = (\n",
    "    df.groupby(['quarter', 'genre'])['completed']\n",
    "    .mean().round(4)\n",
    "    .reset_index()\n",
    ")\n",
    "genre_comp['quarter'] = pd.Categorical(genre_comp['quarter'], categories=quarter_order, ordered=True)\n",
    "genre_comp = genre_comp.sort_values('quarter')\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(8, 5))\n",
    "for genre, grp in genre_comp.groupby('genre'):\n",
    "    ax.plot(grp['quarter'], grp['completed'], marker='o', linewidth=2, label=genre)\n",
    "    ax.annotate(f\"{grp['completed'].iloc[-1]:.0%}\",\n",
    "                xy=(grp['quarter'].iloc[-1], grp['completed'].iloc[-1]),\n",
    "                xytext=(5, 0), textcoords='offset points', va='center', fontsize=9)\n",
    "\n",
    "ax.set_ylim(0.35, 0.85)\n",
    "ax.set_title('Completion Rate by Genre (Q4 2025 → Q1 2026)', fontsize=14)\n",
    "ax.set_xlabel('')\n",
    "ax.set_ylabel('Completion Rate')\n",
    "ax.legend(title='Genre', bbox_to_anchor=(1.01, 1), loc='upper left')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "app0mpor1jk",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Chart 3: Genre mix shift between quarters\n",
    "quarter_order = ['Q4_2025', 'Q1_2026']\n",
    "mix = (\n",
    "    df.groupby('quarter')['genre']\n",
    "    .value_counts(normalize=True)\n",
    "    .mul(100).round(1)\n",
    "    .rename('pct')\n",
    "    .reset_index()\n",
    ")\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(9, 5))\n",
    "sns.barplot(data=mix, x='genre', y='pct', hue='quarter',\n",
    "            hue_order=quarter_order, palette=['#90CAF9', '#1565C0'], ax=ax)\n",
    "ax.set_title('Genre Mix: Share of Sessions by Quarter', fontsize=14)\n",
    "ax.set_xlabel('')\n",
    "ax.set_ylabel('% of Sessions')\n",
    "ax.legend(title='Quarter')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ntl0vcslc1f",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Question 4: Predicting Completion\n",
    "\n",
    "Using the features available in this dataset, build a model to predict whether a viewing session will result in a completed watch.\n",
    "\n",
    "- What features would you use? Are there any you'd exclude and why?\n",
    "- Choose an appropriate model and train it on Q4 2025 data\n",
    "- Evaluate it on Q1 2026 data — what metrics matter here, and why?\n",
    "- What does the model tell you about the most important drivers of completion?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}