{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "open-in-colab-kaggle",
   "metadata": {
    "nbsphinx": "hidden"
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/CVC-DAG/cvcdocdb/blob/main/docs/tutorials/notebooks/demos/memgraph_demo.ipynb)\n",
    "[![Open in Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://www.kaggle.com/kernels/welcome?src=https://raw.githubusercontent.com/CVC-DAG/cvcdocdb/main/docs/tutorials/notebooks/demos/memgraph_demo.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fbdc935",
   "metadata": {},
   "source": [
    "# Propagation Properties — Full Memgraph Demo\n",
    "\n",
    "The workflow of the Neo4j demo (`propagation_demo.ipynb`) on **Memgraph**.\n",
    "`MemgraphGraph` has the same API and propagation behaviour as `Neo4jGraph` and\n",
    "speaks the same Cypher, so the code only changes where the graph is created.\n",
    "\n",
    "**Steps:**\n",
    "1. Connect to Memgraph and load the Game of Thrones dataset\n",
    "2. Generate the YAML schema and the Python entity classes\n",
    "3. Initialize propagation properties\n",
    "4. Create a Document → Section → Page group with `create_group()`\n",
    "5. Run Cypher queries, get subdocuments, and explore the graph in the viewer\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "A running Memgraph server, for example in Docker (Memgraph Community is\n",
    "distributed under the Business Source License, see the README):\n",
    "\n",
    "```bash\n",
    "docker run -p 7687:7687 memgraph/memgraph\n",
    "```\n",
    "\n",
    "The notebook reads `MEMGRAPH_URL` (default `bolt://localhost:7687`),\n",
    "`MEMGRAPH_USER` and `MEMGRAPH_PASSWORD` (default: no authentication).\n",
    "\n",
    "**Warning:** Step 1.5 deletes everything in the database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "332201e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib.util\n",
    "\n",
    "# cvcdocdb with the [notebook] extra (anywidget) for the graph viewer at the end.\n",
    "if (importlib.util.find_spec(\"cvcdocdb\") is None\n",
    "        or importlib.util.find_spec(\"cvcdocdb.viewer\") is None\n",
    "        or importlib.util.find_spec(\"anywidget\") is None):\n",
    "    print(\"⚠️ Installing cvcdocdb[notebook]...\")\n",
    "    %pip install -q --upgrade \"cvcdocdb[notebook]\"\n",
    "    print(\"✅ Installation complete. Restart the kernel if an import below fails.\")\n",
    "else:\n",
    "    print(\"✅ cvcdocdb and anywidget are already installed.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4db2ae3a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import tempfile\n",
    "from pathlib import Path\n",
    "\n",
    "from cvcdocdb import MemgraphGraph\n",
    "from cvcdocdb.exemples import load_got_characters\n",
    "from cvcdocdb.schema_gen import generate_classes\n",
    "\n",
    "def section(title):\n",
    "    \"\"\"Print a section header.\"\"\"\n",
    "    print(f\"\\n{'=' * 60}\")\n",
    "    print(f\"  {title}\")\n",
    "    print(f\"{'=' * 60}\")\n",
    "\n",
    "\n",
    "def sub_section(title):\n",
    "    \"\"\"Print a subsection header.\"\"\"\n",
    "    print(f\"\\n--- {title} ---\")\n",
    "\n",
    "\n",
    "# Generated files (schema and entity classes) go to a fresh temporary directory.\n",
    "output_dir = Path(tempfile.mkdtemp(prefix=\"cvcdocdb_got_\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9fa0a19",
   "metadata": {},
   "source": [
    "## Step 1: Connect to Memgraph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b79c4c33",
   "metadata": {},
   "outputs": [],
   "source": [
    "url = os.environ.get(\"MEMGRAPH_URL\", \"bolt://localhost:7687\")\n",
    "graph = MemgraphGraph(\n",
    "    url,\n",
    "    os.environ.get(\"MEMGRAPH_USER\", \"\"),\n",
    "    os.environ.get(\"MEMGRAPH_PASSWORD\", \"\"),\n",
    ")\n",
    "print(f\"Connected to {url} ({graph.server_edition} edition)\")\n",
    "\n",
    "total = graph.query(\"MATCH (n) RETURN count(n) AS total\")[0][\"total\"]\n",
    "print(f\"Nodes in database: {total}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2357f021",
   "metadata": {},
   "source": [
    "## Step 1.5: Clear existing data\n",
    "\n",
    "Delete all nodes and relationships before loading fresh data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7cc9aaba",
   "metadata": {},
   "outputs": [],
   "source": [
    "graph.query(\"MATCH (n) DETACH DELETE n\")\n",
    "total = graph.query(\"MATCH (n) RETURN count(n) AS total\")[0][\"total\"]\n",
    "print(f\"✅ Database cleared. Nodes remaining: {total}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "832d8069",
   "metadata": {},
   "source": [
    "## Step 2: Load the Game of Thrones dataset\n",
    "\n",
    "Load character and house data from [thronesapi.com](https://thronesapi.com).\n",
    "Falls back to bundled data if the API is unavailable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72dcb4b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "stats = load_got_characters(graph, limit=25)\n",
    "print(\"Loaded:\", json.dumps(stats, indent=2))\n",
    "\n",
    "sub_section(\"Nodes by label\")\n",
    "for row in graph.query(\"MATCH (n) RETURN labels(n)[0] AS label, count(n) AS cnt ORDER BY cnt DESC\"):\n",
    "    print(f\"  {row['label']}: {row['cnt']}\")\n",
    "\n",
    "sub_section(\"Relations by type\")\n",
    "for row in graph.query(\"MATCH ()-[r]->() RETURN type(r) AS rel_type, count(r) AS cnt ORDER BY cnt DESC\"):\n",
    "    print(f\"  {row['rel_type']}: {row['cnt']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6b59bc9",
   "metadata": {},
   "source": [
    "## Step 3: Generate the YAML schema\n",
    "\n",
    "Introspect the graph and generate a YAML schema description: labels,\n",
    "properties, relationships and counts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c041a0a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "yaml_content = graph.schema_yaml(db_name=\"got\")\n",
    "yaml_path = output_dir / \"got_schema.yaml\"\n",
    "yaml_path.write_text(yaml_content)\n",
    "\n",
    "print(f\"Written: {yaml_path} ({len(yaml_content)} bytes)\")\n",
    "print(\"\\nSchema preview:\")\n",
    "for line in yaml_content.split(\"\\n\")[:20]:\n",
    "    print(f\"  {line}\")\n",
    "print(\"  ...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a615c6a1",
   "metadata": {},
   "source": [
    "## Step 4: Generate Python entity classes\n",
    "\n",
    "Generate `Node`, `WeakNode`, `Relation` and `WeakRelation` subclasses for each\n",
    "label and relationship type in the schema."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "611da42f",
   "metadata": {},
   "outputs": [],
   "source": [
    "code = generate_classes(yaml_path.read_text())\n",
    "py_path = output_dir / \"got_entities.py\"\n",
    "py_path.write_text(code)\n",
    "\n",
    "print(f\"Written: {py_path} ({len(code)} bytes)\")\n",
    "print(\"\\nGenerated code preview:\")\n",
    "for line in code.split(\"\\n\")[:25]:\n",
    "    print(f\"  {line}\")\n",
    "print(\"  ...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f34d0b67",
   "metadata": {},
   "source": [
    "## Step 5: Initialize propagation properties\n",
    "\n",
    "`init_propagation()` scans the graph and initialises the propagation properties:\n",
    "\n",
    "- `_propagate` on edges with cascade delete enabled;\n",
    "- `is_weak` on the nodes at the child end of those edges;\n",
    "- `parent_relation`: the edge type linking parent → child;\n",
    "- `_weak_init_done` on the nodes already initialised.\n",
    "\n",
    "The Game of Thrones data has no WeakNodes, so the counts only change after the\n",
    "group of Step 6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f0eabcbc",
   "metadata": {},
   "outputs": [],
   "source": [
    "COUNTS = (\"MATCH (n) RETURN count(n) AS total, count(n.is_weak) AS weak_count, \"\n",
    "          \"count(n._weak_init_done) AS done_count\")\n",
    "\n",
    "def show_counts(title):\n",
    "    sub_section(title)\n",
    "    row = graph.query(COUNTS)[0]\n",
    "    print(f\"  Total nodes:                {row['total']}\")\n",
    "    print(f\"  Nodes with is_weak:         {row['weak_count']}\")\n",
    "    print(f\"  Nodes with _weak_init_done: {row['done_count']}\")\n",
    "    edges = graph.query(\"MATCH ()-[r]->() WHERE r._propagate = TRUE RETURN count(r) AS c\")[0][\"c\"]\n",
    "    print(f\"  Edges with _propagate:      {edges}\")\n",
    "\n",
    "show_counts(\"Before init_propagation()\")\n",
    "print(\"\\nRunning init_propagation()...\", graph.init_propagation())\n",
    "show_counts(\"After init_propagation()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ba98cfe",
   "metadata": {},
   "source": [
    "## Step 6: Transactional group creation\n",
    "\n",
    "`create_group()` creates a strong node with its WeakNodes and WeakRelations in a\n",
    "single transaction: if any step fails, nothing is written. We create a\n",
    "Document → Section → Page hierarchy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4f4c5cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "from cvcdocdb.base import Node, WeakNode\n",
    "\n",
    "doc = Node(\n",
    "    pk={\"doc_id\": \"GOT-DEMO-001\"},\n",
    "    main_label=\"Document\",\n",
    "    title=\"Game of Thrones — Episode Guide\",\n",
    "    author=\"Demo Script\",\n",
    ")\n",
    "section1 = WeakNode(parent=doc, parent_relation=\"HAS_SECTION\", pk={\"section\": \"intro\"},\n",
    "                    main_label=\"Section\", title=\"Introduction\", order=1)\n",
    "section2 = WeakNode(parent=doc, parent_relation=\"HAS_SECTION\", pk={\"section\": \"characters\"},\n",
    "                    main_label=\"Section\", title=\"Main Characters\", order=2)\n",
    "page1 = WeakNode(parent=section1, parent_relation=\"HAS_PAGE\", pk={\"page\": 1}, main_label=\"Page\",\n",
    "                 content=\"Welcome to the Game of Thrones episode guide.\", order=1)\n",
    "page2 = WeakNode(parent=section2, parent_relation=\"HAS_PAGE\", pk={\"page\": 1}, main_label=\"Page\",\n",
    "                 content=\"Jon Snow, Daenerys Targaryen, Tyrion Lannister...\", order=1)\n",
    "\n",
    "doc_id = graph.create_group(strong_node=doc, weak_nodes=[section1, section2, page1, page2])\n",
    "print(f\"✅ Group created (strong node id: {doc_id}): Document, 2 Sections, 2 Pages\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff042769",
   "metadata": {},
   "source": [
    "## Step 7: Verify the group"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5bc50648",
   "metadata": {},
   "outputs": [],
   "source": [
    "sub_section(\"Section and Page nodes\")\n",
    "rows = graph.query(\n",
    "    \"MATCH (d:Document {doc_id: 'GOT-DEMO-001'})-[*1..2]->(n) \"\n",
    "    \"RETURN labels(n)[0] AS label, coalesce(n.title, n.content) AS text, \"\n",
    "    \"n.is_weak AS is_weak ORDER BY label, text\"\n",
    ")\n",
    "for row in rows:\n",
    "    print(f\"  {row['label']}: {row['text'][:45]!r} is_weak={row['is_weak']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "972234c7",
   "metadata": {},
   "source": [
    "## Step 8: Sample queries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf965b93",
   "metadata": {},
   "outputs": [],
   "source": [
    "sub_section(\"Characters by House\")\n",
    "rows = graph.query(\n",
    "    \"MATCH (c:Character)-[:MEMBER_OF]->(h:House) \"\n",
    "    \"RETURN h.name AS house, collect(c.name) AS characters ORDER BY house\"\n",
    ")\n",
    "for row in rows:\n",
    "    names = row[\"characters\"]\n",
    "    print(f\"  {row['house']}: {', '.join(names[:5])}\" + (f\" ... and {len(names) - 5} more\" if len(names) > 5 else \"\"))\n",
    "\n",
    "sub_section(\"Edges with _propagate\")\n",
    "rows = graph.query(\n",
    "    \"MATCH (a)-[r]->(b) WHERE r._propagate = TRUE \"\n",
    "    \"RETURN labels(a)[0] AS from_label, type(r) AS rel_type, labels(b)[0] AS to_label, count(*) AS count \"\n",
    "    \"ORDER BY count DESC\"\n",
    ")\n",
    "for row in rows:\n",
    "    print(f\"  {row['from_label']} -[{row['rel_type']}]-> {row['to_label']}: {row['count']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a93bf99",
   "metadata": {},
   "source": [
    "## Step 9: Get subdocuments\n",
    "\n",
    "`get_subdocuments()` returns every node reachable from a strong node through\n",
    "`_propagate` edges: the document's sections and pages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01b2076f",
   "metadata": {},
   "outputs": [],
   "source": [
    "strong_doc = Node(pk={\"doc_id\": \"GOT-DEMO-001\"}, main_label=\"Document\")\n",
    "subdocuments = graph.get_subdocuments(strong_doc)\n",
    "\n",
    "print(f\"Found {len(subdocuments)} subdocuments:\")\n",
    "for sd in subdocuments:\n",
    "    print(f\"  - {sd['label']}: {sd['pk']}\")  # a WeakNode's key includes its parents' keys"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73d25fec",
   "metadata": {},
   "source": [
    "## Step 10: Explore the graph\n",
    "\n",
    "The graph viewer works on every backend. Click a node to see its properties,\n",
    "double-click it to expand its neighbours (try the Document), and search or filter\n",
    "from the sidebar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8dd5905",
   "metadata": {},
   "outputs": [],
   "source": [
    "from cvcdocdb.viewer import show\n",
    "\n",
    "show(graph, load_all=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32ff5445",
   "metadata": {},
   "source": [
    "## Cleanup\n",
    "\n",
    "Closing the graph disconnects the viewer: run this once you are done exploring.\n",
    "The data stays in Memgraph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a09ec752",
   "metadata": {},
   "outputs": [],
   "source": [
    "# graph.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (cvcdocdb)",
   "language": "python",
   "name": "cvcdocdb"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}