SHACL validation from Python

pip install shacl2cypher

Why a Python package

Most of the code that fills a property graph is Python: ETL jobs, notebooks, Airflow tasks, GraphRAG pipelines, and the test suites around them. Until now, validating that graph from Python meant installing a separate binary, calling it through subprocess, and parsing its JSON. The package removes all three steps. You compile shapes, open a database, validate it and inspect the results in the same process, as ordinary Python values.

Validate a graph in five lines

Take the shapes file from Get started. Every :Person needs a name, a name should start with a capital letter (a warning), and a person works for at most one company:

@prefix sh:  <http://www.w3.org/ns/shacl#> .
@prefix ex:  <http://example.org/> .

ex:PersonShape sh:targetClass ex:Person ;
    sh:property [ sh:path ex:name ; sh:minCount 1 ; sh:message "Person needs a name" ] ,
                [ sh:path ex:name ; sh:pattern "^[A-Z]" ; sh:severity sh:Warning ] ,
                [ sh:path ex:worksFor ; sh:maxCount 1 ] .

Open the database, validate it, and print the report:

import shacl2cypher

with shacl2cypher.Ladybug("graph.lbug") as db:
    report = db.validate(["shapes.ttl"], node_key="id")

print(report.render("table"))

Against a small graph with three people, this prints:

STATUS      SEVERITY   VIOLATIONS  TIME(ms)  RULE
failed      Violation           1        27  PersonShape.name.minCount
            - Person id=p3: Person needs a name
failed      Warning             1         3  PersonShape.name.pattern
            - Person id=p1: ex:PersonShape violates sh:pattern on ex:name (value: "ann")
failed      Violation           1         7  PersonShape.worksFor.maxCount
            - Person id=p1: ex:PersonShape violates sh:maxCount on ex:worksFor

3 rules: 0 passed, 3 failed, 0 timed out, 0 errors, 0 skipped, 0 guaranteed by schema; 3 violations in 37 ms

For Neo4j, only the handle changes:

with shacl2cypher.Neo4j("bolt://localhost:7687", user="neo4j", password="...") as db:
    report = db.validate(["shapes.ttl"], node_key="id", timeout=30)

The connection is checked before Neo4j(...) returns, so a wrong password fails there, not halfway through a validation. As with the CLI, validation is read-only: Neo4j queries run in a transaction that is always rolled back, and LadybugDB files are opened read-only.

Results are plain Python data

A Report wraps the same data the CLI writes with --format json. report.conforms and report.summary give the headline, and report.rules lists one dict per rule, with its status, severity, violation count, timing, source location and violation rows. A row looks like this:

{
  "ruleId": "PersonShape.name.minCount",
  "shape": "ex:PersonShape",
  "path": "ex:name",
  "constraint": "sh:minCount",
  "severity": "Violation",
  "focus": {"label": "Person", "key": "id", "keyValue": "p3", "elementId": "1:2"},
  "value": null,
  "message": "Person needs a name",
  "details": []
}

Keys stay camelCase, exactly as in manifest.json and the JSON report, so one data model is documented once and works everywhere. You can load a CLI report into Python, or hand a Python report to a tool that reads the CLI's JSON:

• report.render(fmt) returns "table", "json", "junit" or "sarif" text, identical to the CLI's output files.
• report.exit_code("violation") returns the CLI's exit code. Pass "warning" or "info" to fail on lower severities.
• report.to_json() and Report.from_json(text) round-trip a report, so a report saved by the CLI can be rendered or scored in Python later.

By default each rule returns up to 100 rows. Pass limit=0 (or None) to get every violation, and timeout to cap each query, in seconds. A query that times out or fails appears as a rule status in the report, as it does in the CLI. It never raises an exception, so one slow rule doesn't hide the results of the others.

Use it as a data test

Since results are just dicts, a pytest suite can treat the graph like any other test fixture. This test fails the build when any Violation-severity rule fails, and lists each broken node in the assertion message:

import pytest
import shacl2cypher


@pytest.fixture(scope="module")
def report():
    with shacl2cypher.Ladybug("graph.lbug") as db:
        yield db.validate(["shapes.ttl"], node_key="id", limit=0)


def test_no_violations(report):
    failures = [
        f"{rule['name']}: {v['focus']['keyValue']} {v['message']}"
        for rule in report.rules
        if rule["severity"] == "Violation"
        for v in rule["violations"]
    ]
    assert not failures, "\n".join(failures)

E       AssertionError: PersonShape.name.minCount: p3 Person needs a name
E         PersonShape.worksFor.maxCount: p1 ex:PersonShape violates sh:maxCount on ex:worksFor

The warning on p1's lowercase name doesn't fail the test. If your CI already collects JUnit XML, write report.render("junit") to a file instead, and each rule shows up as a test case.

Compile without a database

compile needs no connection. It returns a Compilation with the manifest as a dict, the .cypher text, and any static diagnostics. write saves the same two files shacl2cypher compile writes:

compilation = shacl2cypher.compile(["shapes.ttl"], dialect="neo4j", node_key="id")
compilation.write("out")        # out/manifest.json and out/queries.cypher

for rule in compilation.manifest["rules"]:
    print(rule["ruleId"], rule["fingerprint"], rule["source"])

This lets you compile once in a build step, review the Cypher, and ship only the manifest to where validation runs:

with shacl2cypher.Neo4j("bolt://prod:7687", password="...") as db:
    report = db.validate(manifest=open("out/manifest.json").read())

A manifest compiled for one dialect is rejected on a database of another. compile supports all three dialects, neo4j, ladybug and falkordb, and every compile option the CLI has: schema, ontologies, neo4j_labels, strict, max_path_depth, allow_remote_imports and the rest, spelled in snake_case.

Schema snapshots

A database handle can dump its schema, and the snapshot can be passed straight back to compile. On LadybugDB, whose schema is strictly typed, a compile against the snapshot reports typos and missing labels statically, and skips constraints the schema already guarantees:

with shacl2cypher.Ladybug("graph.lbug") as db:
    snapshot = db.schema()              # {"nodeTypes": [...], "relTypes": [...]}
    compilation = shacl2cypher.compile(
        ["shapes.ttl"], dialect="ladybug", schema=snapshot, node_key="id"
    )
    report = db.validate(manifest=compilation)

db.schema_json() returns the exact text shacl2cypher schema dump writes, and a snapshot passed as a dict is re-serialized to that text. The manifest's schema hash is therefore the same whether the snapshot came from a file or from Python.

Shapes that aren't files

Shapes generated by code, fetched from a registry or kept in a database can be passed as in-memory documents. A Source is loaded as if it were the named file in base_dir, which sets its base IRI, how its relative owl:imports resolve, and its path in the manifest:

from shacl2cypher import Source, CompileError

shapes = Source("company.ttl", generated_turtle)
try:
    compilation = shacl2cypher.compile([shapes, "shapes/person.ttl"], dialect="neo4j")
except CompileError as error:
    for message in error.errors:        # every problem, not just the first
        print(message)

Every error derives from Shacl2CypherError: CompileError, ManifestError, DatabaseConnectionError, BackendUnavailableError and DatabaseClosedError. Invalid option values raise ValueError.

How it's built

The package is a thin layer over the Rust crates the CLI uses, built with PyO3 and maturin. It holds no compile or validate logic of its own. It converts values, errors and threads, and nothing else. That is why its output matches the CLI's byte for byte: the manifest JSON, the Cypher text and the rendered reports all come from the same functions.

• It doesn't block other threads. Every native call releases the GIL. Each database handle owns one worker thread that runs its calls one at a time, so a long validation on one handle doesn't stall other Python threads or other handles.
• It's typed. The package ships py.typed, and manifests, reports, rule results and schema snapshots are described by TypedDicts, so mypy --strict and editors know every key. The project's own example is checked with mypy --strict in CI.
• One wheel per platform. Wheels target the stable ABI (abi3), so one wheel covers CPython 3.9 and every later version. Each wheel bundles the Neo4j and LadybugDB backends and links OpenSSL statically, so the system needs no extra libraries.
• Every wheel is smoke-tested. Before any upload, each wheel is installed into a clean virtual environment on its platform. There it compiles shapes and validates a real LadybugDB database. The sdist is installed and tested as a compile-only source build. Packages are published only if every build and test passed, through PyPI trusted publishing, so no upload token is stored anywhere.

Platforms

A compile-only build, without database backends, needs only Rust. This is useful where you only generate manifests:

MATURIN_PEP517_ARGS="--no-default-features --features remote-imports" \
    pip install --no-binary shacl2cypher shacl2cypher

shacl2cypher.available_backends() tells you which backends a given install has.

Not yet

• FalkorDB handles. compile(dialect="falkordb") works, but the package can't open a FalkorDB connection yet. Validate FalkorDB graphs with the CLI's --falkordb flag for now.
• Async API. Calls block the calling thread but release the GIL. From asyncio, wrap them in asyncio.to_thread.
• Prebuilt wheels for Windows, Intel macOS and musl Linux. These platforms build from the sdist. If you need one of them, please open an issue.

Try it

pip install shacl2cypher
python -c "import shacl2cypher; print(shacl2cypher.__version__, shacl2cypher.available_backends())"

Then write a shapes file with Get started, learn how shapes map onto labels and relationships in the SHACL guide, or look up every option and report field in the reference. The package is on PyPI, and its source is in crates/s2c-python.