{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# RiC-O — Graph Model Demo\n", "\n", "This notebook demonstrates how to model a **RiC-O** (Records in Contexts — Ontology)\n", "graph using the DRM tools. RiC-O is the ICA ontology for archival description.\n", "\n", "**Key concepts:**\n", "- `Thing` is the single strong (root) node in RiC-O\n", "- All other entities (`Agent`, `Person`, `CorporateBody`, `Date`, `Place`, `Event`, `RecordResource`, etc.) are `WeakNode` subclasses\n", "- `create_group()` atomically creates a `Thing` with all its weak children\n", "- `get_subdocuments()` retrieves all weak descendants of a strong node\n", "- `load_ric_o_naf()` loads real RiC-O data from the National Archives of France example\n", "\n", "**Steps:**\n", "1. Connect to Neo4j DEV\n", "2. Load real RiC-O data from the National Archives of France (NAF) example\n", "3. Initialize propagation properties\n", "4. Query subdocuments\n", "5. Inspect the hierarchy\n", "6. Summary statistics\n", "\n", "## Prerequisites\n", "\n", "- A running Neo4j instance (DEV target)\n", "- Environment variables: `NEO4J_DEV_URL`, `NEO4J_DEV_USER`, `NEO4J_DEV_PASSWORD`\n", "- Default connection: `bolt://localhost:7687` / `neo4j` / `neo4j2026`\n", "\n", "## Connection configuration\n", "\n", "The notebook reads connection parameters from environment variables\n", "with sensible defaults. Set these before running:\n", "\n", "```bash\n", "export NEO4J_DEV_URL=bolt://your-host:7687\n", "export NEO4J_DEV_USER=neo4j\n", "export NEO4J_DEV_PASSWORD=your-password\n", "```" ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2026-07-12T15:36:19.132606Z", "start_time": "2026-07-12T15:36:18.914139Z" } }, "source": [ "import importlib.util\n", "\n", "# Check if drm is installed\n", "package_to_check = 'drm'\n", "spec = importlib.util.find_spec(package_to_check)\n", "\n", "if spec is None:\n", " print(f'⚠️ {package_to_check} not installed. Installing...')\n", " %pip install -q --upgrade drm-tools\n", " print(\"✅ Installation complete.\")\n", "else:\n", " print(f'✅ {package_to_check} already installed. Skipping.')" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ drm already installed. Skipping.\n" ] } ], "execution_count": 1 }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2026-07-12T15:36:20.267766Z", "start_time": "2026-07-12T15:36:19.159962Z" } }, "source": [ "import os\n", "import sys\n", "from pathlib import Path\n", "\n", "# Read .env file manually (no python-dotenv dependency)\n", "def read_env(env_path):\n", " \"\"\"Read a .env file and return a dict of key=value pairs.\"\"\"\n", " env_vars = {}\n", " with open(env_path, 'r') as f:\n", " for line in f:\n", " line = line.strip()\n", " if not line or line.startswith('#'):\n", " continue\n", " if '=' in line:\n", " key, value = line.split('=', 1)\n", " env_vars[key.strip()] = value.strip()\n", " return env_vars\n", "\n", "# Find .env in repo root\n", "_env_path = Path.cwd() / \".env\"\n", "if not _env_path.exists():\n", " for parent in Path.cwd().parents:\n", " candidate = parent / \".env\"\n", " if candidate.exists():\n", " _env_path = candidate\n", " break\n", "\n", "if _env_path.exists():\n", " _env_vars = read_env(_env_path)\n", " for key, value in _env_vars.items():\n", " os.environ.setdefault(key, value)\n", " print(f\"✅ Loaded .env from {_env_path}\")\n", " print(f\" Found {len(_env_vars)} variables\")\n", "else:\n", " print(\"⚠️ .env file not found. Set NEO4J_DEV_PASSWORD environment variable manually.\")\n", "\n", "# Add drm package to path for development\n", "_repo_root = Path.cwd()\n", "if not (_repo_root / \"drm\").exists():\n", " for parent in Path.cwd().parents:\n", " if (parent / \"drm\").exists():\n", " _repo_root = parent\n", " break\n", "# Add repo root to sys.path so local drm loads before PyPI version\n", "if str(_repo_root) not in sys.path:\n", " sys.path.insert(0, str(_repo_root))\n", " print(f\"✅ Added {_repo_root} to sys.path\")\n", "\n", "from drm import Neo4jGraph\n", "from drm.rico_entities import (\n", " Thing,\n", " Agent, Person, CorporateBody, AgentName, Appellation,\n", " Date, Place, PlaceName, Event, Activity,\n", " RecordResource, Instantiation, Title,\n", " Group, Name,\n", ")\n", "\n", "def neo4j_config(default_target=\"DEV\"):\n", " \"\"\"Build Neo4j connection config from environment variables.\"\"\"\n", " target = os.environ.get(\"NEO4J_TARGET\", default_target).upper()\n", " prefix = f\"NEO4J_{target}_\"\n", " return {\n", " \"target\": target,\n", " \"url\": os.environ.get(f\"{prefix}URL\", os.environ.get(\"NEO4J_URL\", \"bolt://localhost:7687\")),\n", " \"user\": os.environ.get(f\"{prefix}USER\", os.environ.get(\"NEO4J_USER\", \"neo4j\")),\n", " \"password\": os.environ.get(f\"{prefix}PASSWORD\", os.environ.get(\"NEO4J_PASSWORD\", \"secret\")),\n", " \"database\": os.environ.get(f\"{prefix}DATABASE\", os.environ.get(\"NEO4J_DATABASE\", \"neo4j\")),\n", " }\n", "\n", "def section(title):\n", " \"\"\"Print a section header.\"\"\"\n", " print(f\"\\n{'=' * 60}\")\n", " print(f\" {title}\")\n", " print(f\"{'=' * 60}\")\n", "\n", "def sub_section(title):\n", " \"\"\"Print a subsection header.\"\"\"\n", " print(f\"\\n--- {title} ---\")\n", "\n", "print(\"✅ All imports successful.\")" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ Loaded .env from /Users/oriol/Desenvolupament/drm-tools/.env\n", " Found 5 variables\n", "✅ All imports successful.\n" ] } ], "execution_count": 2 }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1: Connect to Neo4j\n", "\n", "Connect to the DEV Neo4j instance and verify the connection." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2026-07-12T15:36:20.961041Z", "start_time": "2026-07-12T15:36:20.389990Z" } }, "source": [ "cfg = neo4j_config()\n", "print(f\"Target: {cfg['target']}\")\n", "print(f\"URL: {cfg['url']}\")\n", "print(f\"User: {cfg['user']}\")\n", "print(f\"Database: {cfg['database']}\")\n", "\n", "graph = Neo4jGraph(\n", " url=cfg[\"url\"],\n", " user=cfg[\"user\"],\n", " password=cfg[\"password\"],\n", " database=cfg[\"database\"],\n", ")\n", "\n", "# Verify connection\n", "result = graph.query(\"RETURN 1 AS test\")\n", "if result and result[0].get(\"test\") == 1:\n", " print(\"\\n✅ Connection verified.\")\n", "else:\n", " print(\"\\n⚠️ Connection returned unexpected result.\")\n", "\n", "# Check current state\n", "count_result = graph.query(\"MATCH (n) RETURN count(n) AS total\")\n", "total = count_result[0][\"total\"] if count_result else 0\n", "print(f\"Nodes in database: {total}\")" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Target: DEV\n", "URL: bolt://localhost:7687\n", "User: neo4j\n", "Database: neo4j\n", "\n", "✅ Connection verified.\n", "Nodes in database: 99\n" ] } ], "execution_count": 3 }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1.5: Clear existing data\n", "\n", "Delete all nodes and relationships before loading fresh RiC-O data." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2026-07-12T15:36:21.501933Z", "start_time": "2026-07-12T15:36:21.092875Z" } }, "source": [ "# Clear all existing data from the database\n", "graph.query(\"MATCH (n) DETACH DELETE n\")\n", "print(\"✅ All existing nodes and relationships deleted.\")\n", "\n", "# Verify database is empty\n", "count_result = graph.query(\"MATCH (n) RETURN count(n) AS total\")\n", "total = count_result[0][\"total\"] if count_result else 0\n", "print(f\"Nodes remaining: {total}\")" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ All existing nodes and relationships deleted.\n", "Nodes remaining: 0\n" ] } ], "execution_count": 4 }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 2: Load real RiC-O data from the National Archives of France\n", "\n", "The DRM tools include a loader that downloads and parses RiC-O RDF/XML data\n", "from the [ICA-EGAD/RiC-O repository](https://github.com/ICA-EGAD/RiC-O/tree/master/examples/examples_v1-1/NationalArchivesOfFrance).\n", "\n", "**What gets loaded:**\n", "\n", "| RDF Type | DRM Entity | Parent |\n", "|----------|-----------|--------|\n", "| `rico:Record` | `Thing` | — (strong node) |\n", "| `rico:Agent` | `Agent` | `Thing` |\n", "| `rico:CorporateBody` | `CorporateBody` | `Agent` |\n", "| `rico:Individual` | `Person` | `Agent` |\n", "| `rico:AgentName` | `AgentName` | `Agent`/`Person` |\n", "| `rico:RecordResource` | `RecordResource` | `Thing` |\n", "| `rico:Instantiation` | `Instantiation` | `Thing`/`RecordResource` |\n", "| `rico:Activity` | `Activity` | `Thing` |\n", "\n", "**Hierarchy example:**\n", "```\n", "Thing (strong)\n", "├── Agent (WeakNode)\n", "│ ├── AgentName (WeakNode of Agent)\n", "│ └── Instantiation (WeakNode of Agent)\n", "└── RecordResource (WeakNode)\n", " └── Instantiation (WeakNode of RecordResource)\n", "```\n", "\n", "The loader downloads files from GitHub, parses the RDF/XML, and inserts\n", "entities into the graph with proper parent-child relationships." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2026-07-12T15:36:34.744216Z", "start_time": "2026-07-12T15:36:21.514709Z" } }, "source": [ "from drm.exemples import load_ric_o_naf\n", "\n", "section(\"Loading RiC-O data from National Archives of France\")\n", "\n", "# Load real RiC-O data\n", "# limit: total entities to load (soft limit)\n", "# max_agents: number of agent files to load\n", "# max_records: number of record resource files to load\n", "stats = load_ric_o_naf(\n", " graph,\n", " limit=50,\n", " max_agents=10,\n", " max_records=3,\n", ")\n", "\n", "print(f\"\\n📊 Load statistics:\")\n", "print(f\" Agents loaded: {stats['agents']}\")\n", "print(f\" Records loaded: {stats['records']}\")\n", "print(f\" Things created: {stats['things']}\")\n", "print(f\" Child entities: {stats.get('children', '?')}\")\n", "print(f\" Total entities: {stats['total']}\")" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "============================================================\n", " Loading RiC-O data from National Archives of France\n", "============================================================\n" ] }, { "ename": "RuntimeError", "evalue": "FK violation: dst node with pk={'identifier': 'thing-rr-recordResource-recordResource-top-003500', 'recordResourceId': 'recordResource-recordResource-top-003500', 'instantiationId': 'instantiation-4598756304'} and main_label=Instantiation does not exist.", "output_type": "error", "traceback": [ "\u001B[31m---------------------------------------------------------------------------\u001B[39m", "\u001B[31mRuntimeError\u001B[39m Traceback (most recent call last)", "\u001B[36mCell\u001B[39m\u001B[36m \u001B[39m\u001B[32mIn[5]\u001B[39m\u001B[32m, line 9\u001B[39m\n\u001B[32m 5\u001B[39m \u001B[38;5;66;03m# Load real RiC-O data\u001B[39;00m\n\u001B[32m 6\u001B[39m \u001B[38;5;66;03m# limit: total entities to load (soft limit)\u001B[39;00m\n\u001B[32m 7\u001B[39m \u001B[38;5;66;03m# max_agents: number of agent files to load\u001B[39;00m\n\u001B[32m 8\u001B[39m \u001B[38;5;66;03m# max_records: number of record resource files to load\u001B[39;00m\n\u001B[32m----> \u001B[39m\u001B[32m9\u001B[39m stats = load_ric_o_naf(\n\u001B[32m 10\u001B[39m graph,\n\u001B[32m 11\u001B[39m limit=\u001B[32m50\u001B[39m,\n\u001B[32m 12\u001B[39m max_agents=\u001B[32m10\u001B[39m,\n", "\u001B[36mFile \u001B[39m\u001B[32m~/Desenvolupament/drm-tools/drm/exemples/load_ric_o_naf.py:784\u001B[39m, in \u001B[36mload_ric_o_naf\u001B[39m\u001B[34m(graph, limit, max_agents, max_records)\u001B[39m\n\u001B[32m 782\u001B[39m \u001B[38;5;28;01mfor\u001B[39;00m node \u001B[38;5;129;01min\u001B[39;00m weak_nodes:\n\u001B[32m 783\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28mhasattr\u001B[39m(node, \u001B[33m\"\u001B[39m\u001B[33m_parent\u001B[39m\u001B[33m\"\u001B[39m) \u001B[38;5;129;01mand\u001B[39;00m node._parent \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[32m--> \u001B[39m\u001B[32m784\u001B[39m \u001B[30;43mgraph\u001B[39;49m\u001B[30;43m.\u001B[39;49m\u001B[30;43minsertNode\u001B[39;49m\u001B[30;43m(\u001B[39;49m\u001B[30;43mnode\u001B[39;49m\u001B[30;43m,\u001B[39;49m\u001B[30;43m \u001B[39;49m\u001B[30;43minsert_parent\u001B[39;49m\u001B[30;43m=\u001B[39;49m\u001B[30;43;01mTrue\u001B[39;49;00m\u001B[30;43m)\u001B[39;49m\n\u001B[32m 786\u001B[39m \u001B[38;5;66;03m# ── 4. Compute final stats from entity_map ────────────────────\u001B[39;00m\n\u001B[32m 787\u001B[39m agent_labels = {\u001B[33m\"\u001B[39m\u001B[33mAgent\u001B[39m\u001B[33m\"\u001B[39m, \u001B[33m\"\u001B[39m\u001B[33mCorporateBody\u001B[39m\u001B[33m\"\u001B[39m, \u001B[33m\"\u001B[39m\u001B[33mPerson\u001B[39m\u001B[33m\"\u001B[39m, \u001B[33m\"\u001B[39m\u001B[33mFamily\u001B[39m\u001B[33m\"\u001B[39m}\n", "\u001B[36mFile \u001B[39m\u001B[32m~/Desenvolupament/drm-tools/drm/neo4j_graph.py:142\u001B[39m, in \u001B[36mNeo4jGraph.insertNode\u001B[39m\u001B[34m(self, node, insert_parent, update, replace, **kwargs)\u001B[39m\n\u001B[32m 134\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m has_parent:\n\u001B[32m 135\u001B[39m \u001B[38;5;28mself\u001B[39m.insertNode(\n\u001B[32m 136\u001B[39m node[\u001B[33m\"\u001B[39m\u001B[33mparent\u001B[39m\u001B[33m\"\u001B[39m],\n\u001B[32m 137\u001B[39m insert_parent=insert_parent,\n\u001B[32m 138\u001B[39m update=\u001B[38;5;28;01mTrue\u001B[39;00m,\n\u001B[32m 139\u001B[39m replace=\u001B[38;5;28;01mFalse\u001B[39;00m,\n\u001B[32m 140\u001B[39m )\n\u001B[32m--> \u001B[39m\u001B[32m142\u001B[39m \u001B[38;5;28mid\u001B[39m = \u001B[30;43mself\u001B[39;49m\u001B[30;43m.\u001B[39;49m\u001B[30;43m_insertNode\u001B[39;49m\u001B[30;43m(\u001B[39;49m\u001B[30;43mnode\u001B[39;49m\u001B[30;43m,\u001B[39;49m\u001B[30;43m \u001B[39;49m\u001B[30;43mupdate\u001B[39;49m\u001B[30;43m=\u001B[39;49m\u001B[30;43mupdate\u001B[39;49m\u001B[30;43m,\u001B[39;49m\u001B[30;43m \u001B[39;49m\u001B[30;43mreplace\u001B[39;49m\u001B[30;43m=\u001B[39;49m\u001B[30;43mreplace\u001B[39;49m\u001B[30;43m)\u001B[39;49m\n\u001B[32m 144\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m has_dependencies:\n\u001B[32m 145\u001B[39m deps = node[\u001B[33m\"\u001B[39m\u001B[33mdependencies\u001B[39m\u001B[33m\"\u001B[39m]\n", "\u001B[36mFile \u001B[39m\u001B[32m~/Desenvolupament/drm-tools/drm/neo4j_graph.py:1161\u001B[39m, in \u001B[36mNeo4jGraph._insertNode\u001B[39m\u001B[34m(self, node, update, replace)\u001B[39m\n\u001B[32m 1156\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m node[\u001B[33m\"\u001B[39m\u001B[33mparent\u001B[39m\u001B[33m\"\u001B[39m][\u001B[33m\"\u001B[39m\u001B[33mpk\u001B[39m\u001B[33m\"\u001B[39m][\u001B[33m\"\u001B[39m\u001B[33mpk\u001B[39m\u001B[33m\"\u001B[39m][ppk] == node[\u001B[33m\"\u001B[39m\u001B[33mpk\u001B[39m\u001B[33m\"\u001B[39m][\u001B[33m\"\u001B[39m\u001B[33mpk\u001B[39m\u001B[33m\"\u001B[39m][ppk]:\n\u001B[32m 1157\u001B[39m \u001B[38;5;28;01mraise\u001B[39;00m \u001B[38;5;167;01mRuntimeError\u001B[39;00m(\n\u001B[32m 1158\u001B[39m \u001B[33m\"\u001B[39m\u001B[33mADGT Exception: Integrity Constraint Violated. Child node keys does not reference proper parent keys\u001B[39m\u001B[33m\"\u001B[39m\n\u001B[32m 1159\u001B[39m )\n\u001B[32m-> \u001B[39m\u001B[32m1161\u001B[39m \u001B[30;43mself\u001B[39;49m\u001B[30;43m.\u001B[39;49m\u001B[30;43minsertRelation\u001B[39;49m\u001B[30;43m(\u001B[39;49m\n\u001B[32m 1162\u001B[39m \u001B[30;43m \u001B[39;49m\u001B[30;43mWeakRelation\u001B[39;49m\u001B[30;43m(\u001B[39;49m\n\u001B[32m 1163\u001B[39m \u001B[30;43m \u001B[39;49m\u001B[30;43mnode\u001B[39;49m\u001B[30;43m[\u001B[39;49m\u001B[30;43m\"\u001B[39;49m\u001B[30;43mparent\u001B[39;49m\u001B[30;43m\"\u001B[39;49m\u001B[30;43m]\u001B[39;49m\u001B[30;43m,\u001B[39;49m\n\u001B[32m 1164\u001B[39m \u001B[30;43m \u001B[39;49m\u001B[30;43mnode\u001B[39;49m\u001B[30;43m,\u001B[39;49m\n\u001B[32m 1165\u001B[39m \u001B[30;43m \u001B[39;49m\u001B[30;43mnode\u001B[39;49m\u001B[30;43m[\u001B[39;49m\u001B[30;43m\"\u001B[39;49m\u001B[30;43mparent_relation\u001B[39;49m\u001B[30;43m\"\u001B[39;49m\u001B[30;43m]\u001B[39;49m\u001B[30;43m,\u001B[39;49m\n\u001B[32m 1166\u001B[39m \u001B[30;43m \u001B[39;49m\u001B[30;43mpropagate\u001B[39;49m\u001B[30;43m=\u001B[39;49m\u001B[30;43;01mTrue\u001B[39;49;00m\u001B[30;43m,\u001B[39;49m\n\u001B[32m 1167\u001B[39m \u001B[30;43m \u001B[39;49m\u001B[30;43m)\u001B[39;49m\u001B[30;43m,\u001B[39;49m\n\u001B[32m 1168\u001B[39m \u001B[30;43m \u001B[39;49m\u001B[30;43mupdate\u001B[39;49m\u001B[30;43m=\u001B[39;49m\u001B[30;43;01mTrue\u001B[39;49;00m\u001B[30;43m,\u001B[39;49m\n\u001B[32m 1169\u001B[39m \u001B[30;43m \u001B[39;49m\u001B[30;43mreplace\u001B[39;49m\u001B[30;43m=\u001B[39;49m\u001B[30;43;01mFalse\u001B[39;49;00m\u001B[30;43m,\u001B[39;49m\n\u001B[32m 1170\u001B[39m \u001B[30;43m \u001B[39;49m\u001B[30;43m)\u001B[39;49m\n\u001B[32m 1172\u001B[39m \u001B[38;5;28;01mexcept\u001B[39;00m ConstraintError \u001B[38;5;28;01mas\u001B[39;00m err:\n\u001B[32m 1173\u001B[39m warnings.warn(\u001B[33mf\u001B[39m\u001B[33m\"\u001B[39m\u001B[33m[XPP Message]: \u001B[39m\u001B[38;5;132;01m{\u001B[39;00merr.message\u001B[38;5;132;01m}\u001B[39;00m\u001B[33m\"\u001B[39m)\n", "\u001B[36mFile \u001B[39m\u001B[32m~/Desenvolupament/drm-tools/drm/neo4j_graph.py:419\u001B[39m, in \u001B[36mNeo4jGraph.insertRelation\u001B[39m\u001B[34m(self, rel, update, replace, **kwargs)\u001B[39m\n\u001B[32m 414\u001B[39m \u001B[38;5;28;01mraise\u001B[39;00m \u001B[38;5;167;01mRuntimeError\u001B[39;00m(\n\u001B[32m 415\u001B[39m \u001B[33mf\u001B[39m\u001B[33m\"\u001B[39m\u001B[33mFK violation: src node with pk=\u001B[39m\u001B[38;5;132;01m{\u001B[39;00mrel[\u001B[33m'\u001B[39m\u001B[33msrc\u001B[39m\u001B[33m'\u001B[39m].get(\u001B[33m'\u001B[39m\u001B[33mpk\u001B[39m\u001B[33m'\u001B[39m)\u001B[38;5;132;01m}\u001B[39;00m\u001B[33m \u001B[39m\u001B[33m\"\u001B[39m\n\u001B[32m 416\u001B[39m \u001B[33mf\u001B[39m\u001B[33m\"\u001B[39m\u001B[33mand main_label=\u001B[39m\u001B[38;5;132;01m{\u001B[39;00mrel[\u001B[33m'\u001B[39m\u001B[33msrc\u001B[39m\u001B[33m'\u001B[39m].get(\u001B[33m'\u001B[39m\u001B[33mmain_label\u001B[39m\u001B[33m'\u001B[39m)\u001B[38;5;132;01m}\u001B[39;00m\u001B[33m does not exist.\u001B[39m\u001B[33m\"\u001B[39m\n\u001B[32m 417\u001B[39m )\n\u001B[32m 418\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m dst_exists:\n\u001B[32m--> \u001B[39m\u001B[32m419\u001B[39m \u001B[38;5;28;01mraise\u001B[39;00m \u001B[38;5;167;01mRuntimeError\u001B[39;00m(\n\u001B[32m 420\u001B[39m \u001B[33mf\u001B[39m\u001B[33m\"\u001B[39m\u001B[33mFK violation: dst node with pk=\u001B[39m\u001B[38;5;132;01m{\u001B[39;00mrel[\u001B[33m'\u001B[39m\u001B[33mdst\u001B[39m\u001B[33m'\u001B[39m].get(\u001B[33m'\u001B[39m\u001B[33mpk\u001B[39m\u001B[33m'\u001B[39m)\u001B[38;5;132;01m}\u001B[39;00m\u001B[33m \u001B[39m\u001B[33m\"\u001B[39m\n\u001B[32m 421\u001B[39m \u001B[33mf\u001B[39m\u001B[33m\"\u001B[39m\u001B[33mand main_label=\u001B[39m\u001B[38;5;132;01m{\u001B[39;00mrel[\u001B[33m'\u001B[39m\u001B[33mdst\u001B[39m\u001B[33m'\u001B[39m].get(\u001B[33m'\u001B[39m\u001B[33mmain_label\u001B[39m\u001B[33m'\u001B[39m)\u001B[38;5;132;01m}\u001B[39;00m\u001B[33m does not exist.\u001B[39m\u001B[33m\"\u001B[39m\n\u001B[32m 422\u001B[39m )\n\u001B[32m 423\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m update:\n\u001B[32m 424\u001B[39m \u001B[38;5;66;03m# return self._session.write_transaction(self._update_relation,rel )\u001B[39;00m\n\u001B[32m 425\u001B[39m \u001B[38;5;28mid\u001B[39m = \u001B[38;5;28mself\u001B[39m._update_relation(\u001B[38;5;28mself\u001B[39m._tx, rel)\n", "\u001B[31mRuntimeError\u001B[39m: FK violation: dst node with pk={'identifier': 'thing-rr-recordResource-recordResource-top-003500', 'recordResourceId': 'recordResource-recordResource-top-003500', 'instantiationId': 'instantiation-4598756304'} and main_label=Instantiation does not exist." ] } ], "execution_count": 5 }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 3: Initialize propagation properties\n", "\n", "Scan the graph and initialize `_propagate` flags on edges between\n", "parent and child nodes. This enables cascade delete behavior." ] }, { "cell_type": "code", "metadata": {}, "source": [ "# Check state before init\n", "sub_section(\"Before init_propagation()\")\n", "result = graph.query(\n", " \"MATCH (n) RETURN count(n) AS total, \"\n", " \"count(n.is_weak) AS weak_count, \"\n", " \"count(n._weak_init_done) AS done_count\"\n", ")\n", "row = result[0] if result else {}\n", "print(f\" Total nodes: {row.get('total', 0)}\")\n", "print(f\" Nodes with is_weak: {row.get('weak_count', 0)}\")\n", "print(f\" Nodes with _weak_init_done: {row.get('done_count', 0)}\")\n", "\n", "# Run initialization\n", "print(\"\\nRunning init_propagation()...\")\n", "init_result = graph.init_propagation()\n", "print(f\"Result: {init_result}\")\n", "\n", "# Check state after init\n", "sub_section(\"After init_propagation()\")\n", "result = graph.query(\n", " \"MATCH (n) RETURN count(n) AS total, \"\n", " \"count(n.is_weak) AS weak_count, \"\n", " \"count(n._weak_init_done) AS done_count\"\n", ")\n", "row = result[0] if result else {}\n", "print(f\" Total nodes: {row.get('total', 0)}\")\n", "print(f\" Nodes with is_weak: {row.get('weak_count', 0)}\")\n", "print(f\" Nodes with _weak_init_done: {row.get('done_count', 0)}\")\n", "\n", "# Show propagation edges\n", "result = graph.query(\n", " \"MATCH ()-[r]->() WHERE r._propagate = TRUE RETURN count(r) AS c\"\n", ")\n", "prop_count = result[0][\"c\"] if result else 0\n", "print(f\" Edges with _propagate: {prop_count}\")" ], "outputs": [], "execution_count": null }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 4: Inspect the loaded graph structure\n", "\n", "Query the database to see what entities were loaded and how they are organized." ] }, { "cell_type": "code", "metadata": {}, "source": [ "# Show nodes by label\n", "sub_section(\"Nodes by label\")\n", "result = graph.query(\n", " \"MATCH (n) RETURN labels(n)[0] AS label, count(n) AS cnt \"\n", " \"ORDER BY cnt DESC\"\n", ")\n", "for row in result:\n", " print(f\" {row['label']}: {row['cnt']}\")\n", "\n", "# Show relations by type\n", "sub_section(\"Relations by type\")\n", "result = graph.query(\n", " \"MATCH ()-[r]->() RETURN type(r) AS rel_type, count(r) AS cnt \"\n", " \"ORDER BY cnt DESC\"\n", ")\n", "for row in result:\n", " print(f\" {row['rel_type']}: {row['cnt']}\")" ], "outputs": [], "execution_count": null }, { "cell_type": "code", "metadata": {}, "source": [ "# Show all Things\n", "sub_section(\"All Things (strong nodes)\")\n", "result = graph.query(\n", " \"MATCH (t:Thing) RETURN t.identifier AS id, t.name AS name \"\n", " \"ORDER BY id\"\n", ")\n", "for row in result:\n", " print(f\" {row['id']}: {row['name']}\")" ], "outputs": [], "execution_count": null }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 5: Get subdocuments\n", "\n", "Use `get_subdocuments()` to retrieve all weak descendants of each `Thing`.\n", "This traverses only edges with `_propagate=True`." ] }, { "cell_type": "code", "metadata": {}, "source": [ "# Show subdocuments for each Thing\n", "result = graph.query(\n", " \"MATCH (t:Thing) RETURN t.identifier AS id ORDER BY id\"\n", ")\n", "\n", "for row in result:\n", " thing_id = row['id']\n", " thing_node = Thing(pk={\"identifier\": thing_id})\n", " subdocs = graph.get_subdocuments(thing_node)\n", " \n", " sub_section(f\"Subdocuments of Thing({thing_id})\")\n", " print(f\" Found {len(subdocs)} subdocuments:\")\n", " \n", " # Group by label\n", " by_label = {}\n", " for sd in subdocs:\n", " label = sd['label']\n", " if label not in by_label:\n", " by_label[label] = []\n", " by_label[label].append(sd)\n", " \n", " for label in sorted(by_label.keys()):\n", " items = by_label[label]\n", " for item in items[:5]: # Show max 5 per label\n", " name = item['pk'].get('fullName', item['pk'].get('name', item['pk'].get('agentId', '?')))\n", " print(f\" - {label}: {name}\")\n", " if len(items) > 5:\n", " print(f\" ... and {len(items) - 5} more\")" ], "outputs": [], "execution_count": null }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 6: Query the hierarchy paths\n", "\n", "Visualize the full hierarchy for a sample `Thing` using Cypher." ] }, { "cell_type": "code", "metadata": {}, "source": [ "# Get first Thing\n", "result = graph.query(\n", " \"MATCH (t:Thing) RETURN t.identifier AS id ORDER BY id LIMIT 1\"\n", ")\n", "if result:\n", " thing_id = result[0]['id']\n", " \n", " sub_section(f\"Hierarchy: Thing({thing_id})\")\n", " \n", " # Query full hierarchy\n", " result = graph.query(\n", " f\"MATCH path = (t:Thing {{identifier: '{thing_id}'}})-[*1..6]->(n) \"\n", " \"RETURN path\"\n", " )\n", " if result:\n", " for row in result:\n", " path = row.get(\"path\")\n", " if path:\n", " if isinstance(path, dict):\n", " nodes = path.get(\"nodes\", [])\n", " rels = path.get(\"relationships\", [])\n", " else:\n", " nodes = path.nodes\n", " rels = path.relationships\n", " parts = []\n", " for node in nodes:\n", " if isinstance(node, dict):\n", " label = list(node.get(\"labels\", [\"Node\"]))[0]\n", " props = node.get(\"properties\", {})\n", " else:\n", " label = list(node.labels)[0] if hasattr(node, 'labels') else 'Node'\n", " props = dict(node)\n", " name = props.get(\"fullName\", props.get(\"name\", props.get(\"agentId\", props.get(\"recordResourceId\", str(getattr(node, 'element_id', '?'))))))\n", " parts.append(f\"{label}({name})\")\n", " for rel in rels:\n", " if isinstance(rel, dict):\n", " parts.append(f\"-[{rel.get('type', '')}]->\")\n", " else:\n", " parts.append(f\"-[{rel.type}]->\")\n", " print(f\" {' → '.join(parts)}\")" ], "outputs": [], "execution_count": null }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 7: Inspect specific entities\n", "\n", "Look at the properties of loaded entities to understand the data model." ] }, { "cell_type": "code", "metadata": {}, "source": [ "# Show sample Agent entities\n", "sub_section(\"Sample Agent entities\")\n", "result = graph.query(\n", " \"MATCH (a:Agent) RETURN a.agentId AS id, a.label AS label, \"\n", " \"a.beginningDate AS birth, a.endDate AS death \"\n", " \"ORDER BY id LIMIT 5\"\n", ")\n", "for row in result:\n", " print(f\" {row['id']}: {row['label']}\")\n", " print(f\" Born: {row['birth'] or 'N/A'}, Died: {row['death'] or 'N/A'}\")\n", "\n", "# Show sample RecordResource entities\n", "sub_section(\"Sample RecordResource entities\")\n", "result = graph.query(\n", " \"MATCH (r:RecordResource) RETURN r.recordResourceId AS id, \"\n", " \"r.title AS title, r.beginningDate AS from_date, r.endDate AS to_date \"\n", " \"ORDER BY id LIMIT 5\"\n", ")\n", "for row in result:\n", " print(f\" {row['id']}: {row['title'] or 'No title'}\")\n", " print(f\" Date range: {row['from_date'] or 'N/A'} — {row['to_date'] or 'N/A'}\")" ], "outputs": [], "execution_count": null }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 8: Summary statistics\n", "\n", "Show the final state of the RiC-O graph." ] }, { "cell_type": "code", "metadata": {}, "source": [ "section(\"Final summary\")\n", "\n", "# Nodes by label\n", "result = graph.query(\n", " \"MATCH (n) RETURN labels(n)[0] AS label, count(n) AS cnt \"\n", " \"ORDER BY cnt DESC\"\n", ")\n", "print(\"Nodes by label:\")\n", "for row in result:\n", " print(f\" {row['label']}: {row['cnt']}\")\n", "\n", "# Propagation edges\n", "result = graph.query(\n", " \"MATCH ()-[r]->() WHERE r._propagate = TRUE \"\n", " \"RETURN type(r) AS rel_type, count(r) AS cnt \"\n", " \"ORDER BY cnt DESC\"\n", ")\n", "print(\"\\nPropagation edges by type:\")\n", "for row in result:\n", " print(f\" {row['rel_type']}: {row['cnt']}\")\n", "\n", "# All Things\n", "result = graph.query(\n", " \"MATCH (t:Thing) RETURN t.identifier AS id, t.name AS name \"\n", " \"ORDER BY id\"\n", ")\n", "print(\"\\nAll Things:\")\n", "for row in result:\n", " print(f\" {row['id']}: {row['name']}\")\n", "\n", "# Subdocument counts\n", "print(\"\\nSubdocuments per Thing:\")\n", "for row in graph.query(\"MATCH (t:Thing) RETURN t.identifier AS id ORDER BY id\"):\n", " thing_node = Thing(pk={\"identifier\": row['id']})\n", " subdocs = graph.get_subdocuments(thing_node)\n", " print(f\" {row['id']}: {len(subdocs)} subdocuments\")\n", "\n", "print(\"\\n\" + \"=\" * 60)\n", "print(\" ✅ RiC-O demo completed successfully!\")\n", "print(\"=\" * 60)" ], "outputs": [], "execution_count": null }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cleanup\n", "\n", "Close the Neo4j connection when done." ] }, { "cell_type": "code", "metadata": {}, "source": [ "graph.close()\n", "print(\"✅ Neo4j connection closed.\")" ], "outputs": [], "execution_count": null } ], "metadata": { "kernelspec": { "display_name": "Python (drm-tools)", "language": "python", "name": "drm-env" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.14" } }, "nbformat": 4, "nbformat_minor": 4 }