Querying with Cypher¶

There are two ways to run Cypher, and most code wants the first:

The examples on this page share this setup:

import pyarrow as pa
from lance_csr import CypherEngine, GraphConfig

config = (
    GraphConfig.builder()
    .node_table("Person", "id")
    .node_table("City", "id")
    .rel_table("KNOWS", "Person", "Person", "src", "dst")
    .rel_table("LIVES_IN", "Person", "City", "src", "dst")
    .build()
)
datasets = {
    "Person": pa.table({
        "id": [1, 2, 3, 4],
        "name": ["Alice", "Bob", "Carol", "David"],
        "age": [28, 34, 29, 42],
    }),
    "City": pa.table({"id": [10, 20], "name": ["London", "Sydney"]}),
    "KNOWS": pa.table({"src": [1, 1, 2, 3], "dst": [2, 3, 4, 4]}),
    "LIVES_IN": pa.table({"src": [1, 2, 3, 4], "dst": [10, 10, 20, 20]}),
}
engine = CypherEngine(config, datasets, use_csr=True)

CypherEngine¶

CypherEngine(config, datasets, use_csr=False)

The constructor validates the datasets against the config, registers them with a DataFusion session, and, with use_csr=True, builds the CSR index. After that, every query reuses the same catalog and session.

execute(query, params=None)¶

Runs the query and returns the whole result as a pyarrow.Table:

table = engine.execute("""
    MATCH (p:Person)-[:LIVES_IN]->(c:City)
    RETURN c.name AS city, collect(p.name) AS people, avg(p.age) AS avg_age
    ORDER BY city
""")
print(table.to_pylist())
# [{'city': 'London', 'people': ['Alice', 'Bob'], 'avg_age': 31.0},
#  {'city': 'Sydney', 'people': ['Carol', 'David'], 'avg_age': 35.5}]

Output columns are named by their AS alias, or by the expression text (p.name) when there is none.

Parameters¶

Write $name in the query and pass a dict as the second argument. Parameter values can be numbers, strings, booleans, None, and datetime.datetime or datetime.date values:

import datetime

engine.execute(
    "MATCH (p:Person) WHERE p.age >= $min_age AND p.name <> $skip RETURN p.name AS name",
    {"min_age": 30, "skip": "Bob"},
).to_pylist()  # [{'name': 'David'}]

Datetimes compare against timestamp columns, and dates against date columns. Parameters are substituted before planning, so they are also allowed in inline property maps: (p:Person {id: $id}).

execute_batches(query, params=None)¶

Streams the result as a pyarrow.RecordBatchReader instead of building one table:

reader = engine.execute_batches("MATCH (p:Person) RETURN p.id AS id, p.name AS name")
rows = 0
for batch in reader:  # each batch is a pyarrow.RecordBatch
    rows += batch.num_rows
print(rows)  # 4

• Batches are computed as you read them. Scans, filters and projections stream with bounded memory; joins, sorts and aggregations may still buffer their inputs.
• Planning errors (a syntax error, an unknown label) raise immediately. Execution errors raise while reading.
• The reader keeps its snapshot alive, so a concurrent refresh() doesn't affect it.
• Dropping the reader cancels the remaining work.

explain(query, params=None)¶

Returns the plan as text: the graph logical plan, DataFusion's logical plan and the physical plan. It's the quickest way to check whether a hop used the CSR (CsrExpandExec) or a join (HashJoinExec), and whether filters reached the Lance scan:

plan = engine.explain("MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN b.name")
print("CsrExpandExec" in plan)  # True

Other methods¶

CypherQuery¶

CypherQuery parses a query up front and runs it against whatever datasets you hand it. Each builder method returns a new query:

from lance_csr import CypherQuery

query = (
    CypherQuery("MATCH (p:Person) WHERE p.age > $age RETURN p.name AS name ORDER BY name")
    .with_config(config)
    .with_parameter("age", 30)
)
print(query.execute(datasets).to_pydict())  # {'name': ['Bob', 'David']}

strategy takes an ExecutionStrategy. ExecutionStrategy.DataFusion is the default and the only one implemented; ExecutionStrategy.LanceNative is reserved and raises an error.

Supported Cypher¶

The Graph Data Council (GDC), formerly known as the Linked Data Benchmark Council (LDBC), maintains the GDC SNB Interactive complex queries. lance-csr implements a large read-only subset of openCypher, enough to run all 14 (IC14 without reduce()).

Clauses¶

Patterns¶

// Direction: outgoing, incoming or either
MATCH (a:Person)-[:KNOWS]->(b)     // outgoing
MATCH (a:Person)<-[:KNOWS]-(b)     // incoming
MATCH (a:Person)-[:KNOWS]-(b)      // either direction

// Variable length: *, *n, *n..m, *..m, *n..
MATCH (a:Person {id: 1})-[:KNOWS*1..3]->(b:Person)

// Alternation, also with variable length
MATCH (p:Person)-[:LIKES_POST|LIKES_COMMENT]->(m)

// Relationship variables and properties
MATCH (a:Person)-[k:KNOWS]->(b:Person) WHERE k.since > 2020

// Shortest paths (one relationship type, both ends bound)
MATCH p = shortestPath((a:Person {id: 1})-[:KNOWS*]-(b:Person {id: 4}))
RETURN length(p), nodes(p)

• Variable length defaults to a minimum of 1 and a maximum of 20 hops; a maximum above 20 is rejected, and zero-length paths (*0..) are not supported. Each length becomes one branch of a union.
• Undirected variable-length paths skip immediate backtracking (a-b-a).
• shortestPath and allShortestPaths need exactly one relationship type, no relationship variable, and named end nodes. Their length is unbounded unless you give a maximum. nodes(p) returns the node keys. With no path, MATCH drops the row and OPTIONAL MATCH returns null.
• Named paths (p = ...) are only supported with shortestPath and allShortestPaths.

Expressions and predicates¶

Functions¶

Any other function raises NotImplementedError: Cypher function 'name' is not implemented. Notably missing: math functions (abs, round, sqrt, ...), most string and list functions, reduce, the list predicates any / all / none / single, and relationships(p).

Example: a little of everything¶

result = engine.execute("""
    MATCH (p:Person)
    OPTIONAL MATCH (p)-[:KNOWS]->(f:Person)
    WITH p, count(f) AS friends
    WHERE EXISTS { MATCH (p)-[:LIVES_IN]->(:City {name: 'London'}) }
    RETURN p.name AS name,
           friends,
           CASE WHEN p.age >= 30 THEN 'senior' ELSE 'junior' END AS band
    ORDER BY name
""")
print(result.to_pylist())
# [{'name': 'Alice', 'friends': 2, 'band': 'junior'},
#  {'name': 'Bob', 'friends': 1, 'band': 'senior'}]

Case sensitivity¶

Keywords, function names, variables, labels, relationship types and property names are all case-insensitive. match (P:person) return p.NAME works, and the output column keeps the spelling you wrote.

Errors¶