Compiling SHACL instead of interpreting it

The gap

SHACL is a good language for data rules. It is declarative, standardized by the W3C, and understood by tools outside your team. But every SHACL engine assumes RDF: a triple store, IRIs, literals. Graphs in Neo4j or LadybugDB are built by applications. They have labels, typed properties and relationships that carry their own properties.

The obvious workaround is to export the graph to RDF and validate the copy. It is expensive for large graphs, stale by the time it finishes, and it answers questions about a projection rather than the data your application reads. It also loses relationship properties, which RDF cannot represent without reification.

shacl2cypher takes the other route. The shapes travel to the data: each constraint becomes Cypher that runs where the data is, uses the database's indexes, and never writes.

One query per constraint

A single giant validation query would be simpler to generate and impossible to live with. You couldn't time it, couldn't tell which rule was slow, and couldn't skip the expensive one. So the unit of compilation is one constraint component instance: sh:minCount on ex:name in ex:PersonShape is one rule.

Each rule gets two queries that share one violation pattern:

• A summary query returns exactly one row, violationCount plus a sample of focus nodes, even when the count is zero. A passing rule is therefore distinguishable from a rule that never ran.
• A detail query returns one row per violation: rule, focus node, offending value, message, and for nested shapes the inner rules that failed.

The runner executes every summary and runs details only where something failed. Because every detail query returns the same columns, a runner may also UNION ALL them.

Names that survive edits

Rule ids are structural, not positional: ex:PersonShape/ex:address/sh:or[0]/ex:street/sh:minCount. Moving a shape within a file, reordering sh:property values or renaming blank nodes changes nothing. A determinism test shuffles every triple of a shapes graph and requires byte-identical output. That test caught an early version ordering rules by source position.

Each rule also carries a fingerprint: a hash of its canonical meaning and dialect. When a rule keeps its name but changes what it checks, the fingerprint changes, and code review shows it.

Cypher's null logic is the enemy

The subtlest failure a validator can have is a violation that silently isn't reported. Cypher's three-valued logic makes that easy. Take a naive range check:

MATCH (p:Person)
WHERE NOT (p.age >= 0)
RETURN p

If age is the string "forty", the comparison is null, NOT null is null, and the row is filtered out. The broken value passes. In SHACL it is a violation.

So every value test is null-safe by construction. Predicates are wrapped as coalesce(test, false) before negation, so a type-incompatible comparison counts as a failure. Functions that raise on the wrong type, such as Neo4j's size() on a number, sit behind explicit type guards. Scalars and lists are normalized into value sets so that absent, null and empty lists all mean "no values". A lint test scans every rendered query for a bare comparison directly under NOT and fails the build if it finds one.

Regexes are parsed, never passed through

sh:pattern uses XSD regular expressions with substring matching. Neo4j's =~ is Java regex and fully anchored. LadybugDB uses RE2, which silently returns false for back-references instead of failing. Passing a pattern through verbatim would be wrong on every engine in a different way.

Patterns are therefore parsed into an AST, validated at compile time with errors at the source line, and rendered per engine. XSD features neither engine supports, such as character-class subtraction ([a-z-[aeiou]]) and \i/\c name classes, are expanded into explicit ranges. Constructs one engine can't evaluate, such as back-references on LadybugDB, are compile errors rather than quietly wrong queries.

Convention, evidence, annotation

Mapping RDF terms to a property graph needs defaults, or nobody would bother. It also needs a way to be exact, or teams wouldn't trust it. The resolution order reflects that: naming conventions first, schema-snapshot evidence over conventions, and explicit s2c: annotations over everything.

Annotation parsing is deliberately strict. A misspelt s2c:relatonship is a compile error, not a silent fall-back to convention. --strict goes further and rejects any name that only a convention chose.

Let the schema answer what it can

LadybugDB enforces types. If email is a STRING column, checking sh:datatype xsd:string by scanning every node wastes time, so the rule is marked guaranteed-by-schema and gets no query. If the column is INT64, no data can ever satisfy the constraint; that is a design error, reported as a static diagnostic at compile time. Neo4j snapshots describe observed data rather than enforced types, so on Neo4j those rules always run.

How we know it is right

A compiler that emits validation queries needs its own validation. Four layers provide it:

• Differential conformance fixtures. Each fixture is a neutral YAML graph with a hand-written list of expected violations. It is projected to RDF for pySHACL, and to Neo4j and LadybugDB for shacl2cypher. All must agree with the list; documented engine differences are recorded per engine, never as silent passes.
• The W3C SHACL Core suite runs on Neo4j, projected RDF→LPG. Of 98 tests, 27 pass. 46 are skipped because they use sh:targetNode, 19 because they validate literal focus nodes, 2 for language tags and 2 because a predicate has both literal and IRI values, which has no property-graph form. One is a compile skip, and one fails because it needs property shapes nested inside property shapes, which are not supported yet. Every outcome is committed in tests/w3c/status.yaml.
• Literal round-trip fuzzing renders random strings, identifiers and typed constants, runs RETURN <literal> on the database, and compares. It found two real defects: Neo4j decodes \uXXXX inside backticked identifiers, and the neo4rs 0.8 driver decoded the integers −16…−1 as 240…255.
• Renderer smoke runs execute every rendered query on both engines and compare counts. They found a LadybugDB bug where UNWIND list_filter(coalesce(…)) leaks list elements into rows whose list is null. The renderer now avoids that construct.

One engine, three front doors

The compiler is a pure Rust crate with no database I/O. The runner adds Neo4j (Bolt) and LadybugDB (embedded) behind cargo features. The CLI, the Python package and the Node.js package are thin layers over the same two crates. Their manifests, queries and reports are byte-identical, and the binding test suites check that parity against the CLI.

What's next

• FalkorDB, as a third dialect with its own renderer. See Bringing SHACL validation to FalkorDB.
• Property shapes nested inside property shapes, the one W3C Core failure.
• Partitioned execution with $skip/$batch for very large graphs. Today, cost classes and recommended indexes are the tools for ordering and speeding up expensive rules.