Unlocking the PANTHEON Data Engine: A deep dive into the secure data repository -D7.3

At the heart of the PANTHEON Community-Based Smart City Digital Twin lies a complex, living system: the Secure Data Repository. Developed under Deliverable D7.3, this infrastructure is responsible for ingesting, securing, and processing terabytes of data to support disaster resilience in Athens and Vienna.
Because this deliverable encompasses everything from high-level architecture to specific JSON schemas and security encryption standards, a single summary cannot do it justice. To provide a clear, technical understanding of how this engine works, we have broken down the deliverable into a 7-part deep dive series.
Below is your roadmap to understanding the PANTHEON Secure Data Repository.
1. The Architecture
To understand PANTHEON, you must first understand the foundation. This post explains the “Federated Architecture” strategy—how we manage over 1,000 datasets across two diverse pilot cities by decoupling storage from processing. Read: Blog Post 1: The Digital Backbone – Architecting the PANTHEON Secure Data Repository
2. The Data Sources
A Digital Twin is only as reliable as its inputs. Here, we explore the ecosystem of authoritative data—from UN demographics and Copernicus satellite imagery to local seismic sensors—and how we validate provenance to ensure “garbage in, garbage out” never happens. Read: Blog Post 2: Provenance & Precision – The Data Ecosystem
3. The Scenarios
Data serves a purpose. This post details exactly how we operationalize data for our four core disaster scenarios: Earthquake Planning (Attica), Heatwave Planning (Vienna), Wildfire Training (Attica), and Cyberattack Training (Vienna). Read: Blog Post 3: Scenario Deep Dive – Modeling Earthquakes, Fire, Heat, and Cyber Threats
4. The Workflow & Real-Time Streaming
How does a static map become a live simulation? We explain the “Nervous System” of PANTHEON—specifically how Apache Kafka streams facilitate event-driven architectures that react to fire spreads and blocked roads in seconds. Read: Blog Post 4: The Nervous System – Workflows and Real-Time Kafka Streaming
5. The Tech Stack
For the engineers and architects: A look under the hood at the “Big Four” technologies driving the platform (PostgreSQL, MinIO, Neo4j, Kafka) and the performance metrics that justify their selection. Read: Blog Post 5: The Engine Room – Technical Infrastructure & Performance
6. Security & Governance
Disaster data reveals critical vulnerabilities. This post outlines our “Defense in Depth” strategy, covering Identity Management (Keycloak), encryption standards (TLS/AES-256), and how we handle privacy in compliance with GDPR. Read: Blog Post 6: The Ironclad Gateway – Security, Identity, and Encryption
7. Operations & Resilience
How do we ensure the system is available 24/7? We conclude the series by examining the “Scenario as a Workspace” model, our No Data Loss (NDL) backup policies, and the auditing trails that ensure full accountability. Read: Blog Post 7: Operational Resilience – Maintenance, Auditing, and Recovery
The PANTHEON project has received funding from the European Union’s Horizon Europe programme under Grant Agreement N°101074008.
