Python API

Five functions, no classes, no custom exceptions — every signature, default and return shape.

lzcomplexity exports five callables and one version string. There is nothing else on the public surface: no classes, no submodules, no exception hierarchy.

>>> import lzcomplexity as lz
>>> lz.__version__
'1.0.2'
>>> lz.__all__
['lz76', 'factorization', 'h', 'emc', 'nid']

5public functions

0classes

0custom exceptions

3.9+requires-python

At a glance

Function	Returns	What it gives you
factorization(seq, …)	tuple[int, list[int]]	complexity and the factor boundary list
h(seq, …)	float	normalized entropy density (entropy-rate estimator)
emc(seq, …)	tuple[float, list[float]]	effective measure complexity and its per-scale summands
nid(seq1, seq2, …)	float	normalized information distance between two sequences
lz76(seq, …)	dict (9 keys)	everything above plus ε, error estimates and extras

All five are compiled builtin_function_or_method objects. Every parameter is positional-or-keyword — there are no keyword-only parameters and no var-args — so lz.h(seq="abab") and lz.nid(seq1="ab", seq2="ba") both work.

Throughout this page, SEQ is the 47-symbol binary sequence "01001010101101010101110101010101010000100101011".

Shared parameters

Besides the sequence itself, four parameters appear on every function; max_block_size appears on emc and lz76 only.

Parameter	Type	Default	Effect
seq	str \| bytes \| list[int] \| list[str] \| sequence of ints	—	the input. See Inputs for the coercion rules
partitions	int	1	none — accepted and ignored
alphabet	int \| None	None	affects lz76(...)["epsilon"] and nothing else
log_base	int \| None	None	entropy units. None means "use the detected alphabet size"
max_block_size	int	-1	largest shuffle block size for EMC. Any value <= 0 selects automatically
jobs	int	0	none — accepted and ignored

alphabet affects only epsilon

The core reads the alphabet argument at exactly one place: the finite-size correction ε computed inside the factorizer. factorization, h, emc and nid never surface ε, so passing alphabet to those four is a no-op — verified bit-identical for alphabet=2 against alphabet=99 on a 500-symbol ternary sequence.

ε = 2 · (1 + log_b log_b(k·n)) ⁄ log_b n

k

max(alphabet, 2), or max(count of distinct bytes, 2) when alphabet=None

b

max(log_base, 2), or the detected alphabet size (minimum 2) when log_base=None — the detected size, not k, so passing alphabet does not move b

n

sequence length in bytes

>>> SEQ = "01001010101101010101110101010101010000100101011"
>>> lz.lz76(SEQ)["epsilon"]
1.336734504313188
>>> lz.lz76(SEQ, alphabet=4)["epsilon"]
1.4104926018454618
>>> lz.h(SEQ) == lz.h(SEQ, alphabet=4)      # h ignores alphabet entirely
True

The h docstring is wrong about alphabet

help(lz.h) calls alphabet the "effective alphabet size used in the entropy formula". It is not used in the entropy formula — only log_base enters h. The docstring is a known defect; the behaviour documented on this page is the measured one.

log_base sets the entropy units

log_base=None means "use the detected alphabet size", so h is normalized against the alphabet by default rather than reported in bits. Pass log_base=2 for bits.

>>> lz.h("ABRACADABRA")                 # b = 5 distinct symbols
0.6772255010931718
>>> lz.h("ABRACADABRA", log_base=2)     # bits
1.572468917562408

log_base is inert for factorization (a raw count) and for nid (a ratio of raw counts). Values 0 and 1 are silently clamped to 2. Only integers are accepted: log_base=2.718 raises TypeError, so there is no way to request natural log.

4294967295 is silently read as None

Passing alphabet=4294967295 or log_base=4294967295 returns the auto-detected result, not a base-4294967295 result. That value is the internal "no alphabet" sentinel (u32::MAX) and the API cannot distinguish it from None. 4294967294 is honoured normally.

>>> lz.h(SEQ, log_base=4294967295)
1.063644673725505
>>> lz.h(SEQ)                        # identical — the sentinel won
1.063644673725505

The two parameters that do nothing

partitions: int = 1 ignored

Accepted, stored, and never read by any algorithm.

what happens

The value is written to the internal LzArgs.chunks field. The only read of that field anywhere in the Rust workspace is inside a PartialEq implementation. No factorization, entropy, shuffle or distance code path consults it.

not a speed knob

It cannot be one. There is no suffix-array partitioning in the core — one suffix array is built per sequence. Any timing difference between partitions values is measurement noise.

range

i32. Negative values are accepted without complaint; partitions=2**31 raises OverflowError.

jobs: int = 0 ignored

Accepted and discarded on the first line of every function.

what happens

Each of the five bindings begins with let _ = jobs;. The value never reaches the argument struct and never reaches the core.

the real knob

The RAYON_NUM_THREADS environment variable. Results are bit-identical across thread counts, so it is a pure performance dial.

range

u32. jobs=-1 raises OverflowError. The parameter manages to be both inert and capable of failing.

Verified: lz76(SEQ, partitions=1) == lz76(SEQ, partitions=99) and lz76(SEQ, jobs=0) == lz76(SEQ, jobs=32), both True.

Both parameters exist because the C++ predecessor had them, and they are kept so that old call sites keep working. They are documented here as inert rather than quietly removed.

factorization

lz.factorization(seq, partitions=1, alphabet=None, log_base=None, jobs=0) → tuple[int, list[int]] stable

The LZ76 complexity — a count of complete components — together with the factor boundary indices.

seq

str | bytes | list[int] | list[str], or any object supporting the sequence protocol whose elements are Python ints. Iterators, generators and sets raise TypeError.

partitions

int — ignored. See above.

alphabet

int | None — no effect on this function.

log_base

int | None — no effect on this function; the return is a raw count.

jobs

int — ignored. See above.

returns

(complexity, factors). complexity is a Python int; factors is a list[int] of boundary indices, always beginning [0, 1, …]. Factor i spans seq[factors[i]:factors[i+1]].

>>> lz.factorization("banana")
(3, [0, 1, 2, 3, 7])
>>> lz.factorization("ABRACADABRA")
(5, [0, 1, 2, 3, 5, 7, 12])
>>> lz.factorization("01010101")
(2, [0, 1, 2, 9])
>>> lz.factorization(SEQ)
(9, [0, 1, 2, 4, 7, 12, 22, 31, 36, 41, 48])

len(factors) - 1 is not the complexity

Taking len(factors) - 1 overcounts by one on most inputs, because the final boundary usually runs past the end of the sequence. "banana" has n = 6 but a final boundary of 7; "01010101" has n = 8 and a final boundary of 9. On 4 990 random non-constant strings (lengths 2–300, alphabets of size ⅔/4/26) the boundary list overshot in 3 674 of them — 74 %. Use the returned integer, or:

complexity = len(factors) - 1 if factors[-1] <= len(seq) else len(factors) - 2

complexity counts complete components only. The trailing component that runs past the end of the sequence is not counted, so this library's number is one less than the textbook exhaustive-history count whenever the sequence ends mid-component. The conversion is exact:

c_textbook = c + 1  if  factors[−1] > n,  otherwise  c

c

the integer returned by factorization

factors[−1]

the last element of the returned boundary list

n

len(seq) in bytes

Checked against an independent exhaustive-history implementation on 497 random non-constant strings — zero disagreements, on both the boundary list and the converted count.

Background: LZ76 factorization.

Empty and constant sequences take a shortcut path that returns a synthetic boundary list and bypasses the rule above:

input	result	note
""	(1, [0, 1, 0])	non-monotonic; an empty sequence reports complexity 1, not 0
"a"	(1, [0, 1, 1])
"aaaa"	(1, [0, 1, 4])
[], (), b""	(1, [0, 1, 0])	same as ""

h

lz.h(seq, partitions=1, alphabet=None, log_base=None, jobs=0) → float stable

Normalized entropy density — the LZ76 entropy-rate estimator.

seq

as for factorization.

partitions

int — ignored.

alphabet

int | None — no effect, despite what the docstring says.

log_base

int | None — the logarithm base. None uses the detected alphabet size (minimum 2); pass 2 for bits. 0 and 1 clamp to 2.

jobs

int — ignored.

returns

a Python float. Sequences of length ≤ 1 short-circuit to 0.0.

h ≈ c(S) · log_b n ⁄ n

c(S)

the complexity returned by factorization

n

sequence length in bytes

b

max(log_base, 2), or the detected alphabet size when log_base=None

The core evaluates it as c / (n / log_b(n)). Recomputing the algebraically equal c * log_b(n) / n in Python reproduced the returned float on 1 452 of 2 000 random sequences and differed by 1–2 ulp on the rest — reproduce the evaluation order, not only the algebra.

>>> lz.h("01010101")            # c = 2, n = 8, b = 2  ->  2 * 3 / 8
0.75
>>> lz.h("banana")
0.8154648767857287
>>> lz.h("ABRACADABRA")
0.6772255010931718
>>> lz.h(SEQ)
1.063644673725505
>>> lz.h(""), lz.h("a"), lz.h("aa"), lz.h("ab")
(0.0, 0.0, 0.5, 1.0)

h is not bounded by 1

Values above 1 are normal on short random sequences and do not indicate a bug. h is a finite-size estimator that approaches its limit from above: averaged over random binary strings it reads about 1.18 at n = 10, 1.13 at n = 100 and 1.03 at n = 10 000 (20 000 / 5 000 / 20 draws). h(SEQ) above is 1.0636… for exactly this reason. See Finite-size convergence.

emc

lz.emc(seq, partitions=1, alphabet=None, log_base=None, max_block_size=-1, jobs=0) → tuple[float, list[float]] stable

Effective measure complexity via block shuffling, with the per-block-size terms that sum to it.

seq

as for factorization.

partitions

int — ignored.

alphabet

int | None — no effect.

log_base

int | None — sets the scaling factor g = log_b(N) ⁄ N.

max_block_size

int — the largest shuffle block size. Any value <= 0 (including -1, 0 and -100) selects automatically from the sequence length. It is never clamped to len(seq).

jobs

int — ignored.

returns

(emc_value, summands). summands always has exactly as many entries as the effective block size — never empty, never None, even for degenerate input. Every entry is >= 0.0 and they sum to emc_value, which is itself >= 0.0; summands[l-1] estimates the scale-l conditional-entropy excess h(l) − h. emc_value is read off the projected ladder rather than accumulated, so sum(summands) can land 1 ulp away. summands[0] usually equals multi_information, but not always — see the field note.

>>> lz.emc("ABRACADABRA")
(0.5417804008745377, [0.27089020043726886, 0.27089020043726886])
>>> lz.emc(SEQ)
(0.17727411228758438, [0.05909137076252813, 0.0,
                       0.11818274152505626, 0.0])
>>> lz.emc(""), lz.emc("a"), lz.emc("abab")
((0.0, [0.0]), (0.0, [0.0]), (0.0, [0.0]))
>>> lz.emc("a", max_block_size=5)
(0.0, [0.0, 0.0, 0.0, 0.0, 0.0])

A sequence with genuine structure

A half-periodic, half-random binary string gives a clearly nonzero EMC and 17 summands.

import random
import lzcomplexity as lz

r = random.Random(42)
s = "01" * 300 + "".join(r.choice("01") for _ in range(400))   # n = 1000
value, summands = lz.emc(s)

>>> value
2.2024383269103214
>>> len(summands)
17
>>> summands
[0.5082549985177665, 0.049828921423310524, 0.488323429948442, 0.0,
 0.34880244996317344, 0.039863137138647975, 0.06976048999263496, 0.0,
 0.2840248521128692, 0.0, 0.08969205856195894, 0.0, 0.14201242605643505,
 0.0, 0.0, 0.0, 0.1818755631950828]
>>> min(summands) >= 0.0
True
>>> abs(sum(summands) - value) < 1e-12
True
>>> lz.factorization(s)[0]
51

The four 0.0 entries are scales the projection pooled with their left neighbour, not scales that were skipped — all 17 surrogates were factorized.

Every block size contributes, and the terms cannot go negative

Each block size l yields a rung Ê(l) = l · g · (C_LZ(u^RS(l)) − C_LZ(u)), an estimate of the excess entropy accumulated up to scale l. Excess entropy is a mutual information between past and future, so the true ladder is non-negative and non-decreasing; the raw rungs are neither, because each rests on its own surrogate draw. The library projects the ladder onto that shape and returns the increments:

Ê_fit = argmin_v Σ_l (v_l − Ê(l))² subject to 0 ≤ v₁ ≤ … ≤ v_mm

summands[l−1] = Ê_fit(l) − Ê_fit(l−1)  ·  Ê = Ê_fit(mm)

mm

the effective max_block_size, which also equals len(summands)

g

log_b(N) ⁄ N

C_LZ(u)

complexity of the original sequence

C_LZ(u^RS(l))

complexity after block-shuffling at scale l

Isotonic regression by pool-adjacent-violators. Reading the total off the projection rather than summing the raw first differences is what keeps the intermediate scales in play — the raw sum telescopes to its own top rung. Derived on Effective measure complexity.

Read the value as an ordinal index at fixed length, not as a bit count

The total is non-negative and every scale informs it, but two biases survive: the estimator overshoots the analytic excess entropy of order-1 Markov sources by 2–3×, and it scales with max_block_size, which is itself derived from len(seq). Never compare emc between sequences of different length.

An exact 0.0 now means one thing: no scale showed a monotone rise, which is what structureless input looks like. Constant sequences return 0.0 with all-zero summands, and so does i.i.d. binary at n = 20 000. It is no longer produced by resonance between the block grid and the source period — lz.emc("01" * 1024) returned 0.0 under 1.0.2 because mm = 18 is a multiple of the period, and now returns 1.0196126302083333, while lz.emc("01" * 500) returns 2.033019994071066 (mm = 17). A small positive value is more likely to be the noise floor than a weak signal; Effective measure complexity tabulates that floor by length.

Automatic max_block_size

With max_block_size <= 0 the block size is derived from the length. Measured values of len(summands):

len(seq)	5	9	10	20	47	50	51	100	1 000	10 000
auto mm	1	1	2	3	4	4	14	14	17	20

The jump at n = 51 is a +10 step applied above 50 symbols.

A large max_block_size costs a full re-factorization per unit

Runtime grows linearly in max_block_size, and each unit re-factorizes the whole sequence, so a large value is only expensive when len(seq) is also large. Measured on a 20 000-symbol binary string (one machine, ratios are the durable part): auto — mm = 21 — 0.007 s, max_block_size=50 0.013 s, max_block_size=200 0.045 s. There is no upper clamp — emc("abababab", max_block_size=100000) really does return a 100 000-element list and perform 100 000 shuffle-and-refactorize rounds.

emc is reproducible run to run with no seeding, because the shuffle seed is derived from the sequence content. See Determinism and Effective measure complexity.

nid

lz.nid(seq1, seq2, partitions=1, alphabet=None, log_base=None, jobs=0) → float stable

Normalized information distance between two sequences, from four LZ76 factorizations.

seq1, seq2

two sequences, each accepted in any of the input forms. They need not be the same length or over the same alphabet.

partitions

int — ignored.

alphabet

int | None — no effect.

log_base

int | None — no effect; the result is a ratio of raw counts.

jobs

int — ignored.

returns

a Python float. A zero-denominator guard exists but never fires, because every factorization returns at least 1; nid("", "") is 0.0 because the numerator is zero, not because of the guard.

NID(X, Y) = max[C(XY) − C(X), C(YX) − C(Y)] ⁄ max[C(X), C(Y)]

C(·)

LZ76 complexity of a sequence

XY

concatenation of X followed by Y

The four factorizations run concurrently inside one call, on the rayon pool.

Background: Information distance.

>>> lz.nid("abcd", "abce")
0.25
>>> lz.nid("ABRACADABRA", "ABRACADABRZ")
0.3333333333333333
>>> lz.nid("ABRACADABRA", "ZYXWVUTSRQP")
0.9090909090909091
>>> lz.nid("aaaa", "bbbb")
1.0
>>> lz.nid("", ""), lz.nid("a", "")
(0.0, 0.0)

nid(x, x) is not always 0

A sequence compared with itself can return a nonzero distance, so a zero test is not a valid identity check. The result is 0 only when appending a copy of x to itself adds no new factors.

>>> lz.nid("abc", "abc")
0.0
>>> lz.nid(SEQ, SEQ)
0.1111111111111111

NID is symmetric by construction, and nid(A, B) == nid(B, A) held on every pair tested. Identity of indiscernibles fails on finite sequences, so it is not a metric in the strict sense.

The [0, 1] range is empirical, not guaranteed

Every input tried returned a value in [0, 1], but the numerator has no formal bound below the denominator for finite sequences. Treat the range as an observation.

lz76

lz.lz76(seq, partitions=1, alphabet=None, log_base=None, max_block_size=-1, jobs=0) → dict stable

Everything the library computes about one sequence, in a single pass.

seq

as for factorization.

partitions, jobs

int — ignored.

alphabet

int | None — this is the one function where it matters; it changes ["epsilon"].

log_base

int | None — affects h, epsilon, the EMC scaling and both error estimates.

max_block_size

int — as for emc.

returns

a dict with exactly 9 top-level keys, insertion-ordered as listed below. Two of them are nested dicts. A fresh object is built on every call; nothing is cached.

The complete return value

Real output from lzcomplexity 1.0.0, unedited:

>>> lz.lz76("ABRACADABRA")
{'complexity': 5,
 'h': 0.6772255010931718,
 'factors': [0, 1, 2, 3, 5, 7, 12],
 'emc': {'value': 0.5417804008745377,
         'summands': [0.27089020043726886, 0.27089020043726886],
         'max_block_size': 2,
         'multi_information': 0.27089020043726886},
 'epsilon': 2.10324356088793,
 'factors_stddev': 1.6187674827674623,
 'normal_error': 0.3320213997117467,
 'poison_error': 0.061565954644833804,
 'extras': {'rajski_distance': 0.3999999999999999,
            'redundancy': 0.375,
            'fh_uncertainty': 0.75,
            'lh_uncertainty': 0.75,
            'pearson_coefficient': 0.75}}

Top level — 9 keys

Key	Type	Meaning
complexity	int	complete LZ76 components; identical to factorization(seq)[0]
h	float	normalized entropy density; identical to h(seq)
factors	list[int]	boundary indices; identical to factorization(seq)[1]
emc	dict	4 keys, below
epsilon	float	Lempel–Ziv finite-size correction ε. Computed and reported, but never applied to h or inside the EMC path
factors_stddev	float	spread of factor lengths — not a textbook standard deviation, see below
normal_error	float	sqrt(h**3) * factors_stddev / sqrt(n / log_b(n))
poison_error	float	h / n. The key name is a frozen misspelling of Poisson; it is not corrected because that would break callers
extras	dict	5 keys, below

["emc"] — 4 keys

Key	Type	Meaning
value	float	the same float as emc(seq)[0]; always >= 0.0
summands	list[float]	the same list as emc(seq)[1]; its length equals max_block_size, every entry is >= 0.0, and they sum to value
max_block_size	int	the block size actually used, after auto-selection
multi_information	float	the raw l = 1 rung, reported before the projection. Usually equals summands[0]; differs when the first rung was clamped or pooled

["extras"] — 5 keys. All five compare the first half of the sequence (fh, split at n // 2) with the second half (lh), using mi = C(fh) + C(lh) − C(seq).

The core guards against a zero complexity by returning all-zero extras, but that guard never fires: the factorizer returns at least 1 for every input, the empty sequence included. So the extras are always populated, meaningful or not — see the empty-input example below.

Key	Type	Definition
rajski_distance	float	2 − (C(fh) + C(lh)) / C(seq)
redundancy	float	mi / (C(fh) + C(lh))
fh_uncertainty	float	mi / C(fh)
lh_uncertainty	float	mi / C(lh)
pearson_coefficient	float	mi / sqrt(C(fh) * C(lh))

factors_stddev divides by the longest factor, not the factor count

The value is not comparable with a standard deviation you compute yourself. The core evaluates sqrt( Σ (len_i − mean)**2 / max_factor_len ), where mean is (factors[-1] − 1) / len(factors) — divided by the number of boundaries, one more than the number of gaps. It is a heuristic dispersion measure inherited from the C++ implementation, and normal_error is built on top of it, so it inherits the same caveat.

The lz76 docstring names a key that does not exist

help(lz.lz76) lists entropy_density. The key is h. The rename landed in 0.12.0 and the Rust docstring was missed; the type stub and the README are both correct.

Cross-consistency with the standalone functions

Exact float equality, verified — the implementation factorizes once and reuses the count:

>>> s = "ABRACADABRA"
>>> d = lz.lz76(s)
>>> d["complexity"] == lz.factorization(s)[0]
True
>>> d["factors"] == lz.factorization(s)[1]
True
>>> d["h"] == lz.h(s)
True
>>> (d["emc"]["value"], d["emc"]["summands"]) == lz.emc(s)
True

lz76 costs several times more than factorization or h on the same input, because it pays for mm extra shuffled re-factorizations — about 6× on a 100 000-symbol binary string, where mm is 23. The multiplier tracks mm, so it grows slowly with length. If you only need the complexity, call factorization.

Degenerate input still returns the full dict

No exception is raised for an empty sequence. Every key is present, and some values are numerically meaningless.

>>> lz.lz76("")
{'complexity': 1,
 'h': 0.0,
 'factors': [0, 1, 0],
 'emc': {'value': 0.0, 'summands': [0.0], 'max_block_size': 1, 'multi_information': 0.0},
 'epsilon': 0.0,
 'factors_stddev': 1.3743685418725535,
 'normal_error': 0.0,
 'poison_error': 0.0,
 'extras': {'rajski_distance': 0.0,
            'redundancy': 0.5,
            'fh_uncertainty': 1.0,
            'lh_uncertainty': 1.0,
            'pearson_coefficient': 1.0}}

Note factors_stddev = 1.374… and a full set of nonzero extras for a sequence with no data. If "no data" should be an error in your pipeline, check len(seq) yourself.

Errors

There are no custom exception classes in this package. Everything raised is a builtin.

Wrong sequence type → TypeError

Call	Exception
lz.factorization(None)	TypeError: expected str, bytes, list[str], list[int], or iterable of ints
lz.factorization(42), lz.factorization(3.14)	same message
lz.factorization({0, 1}), lz.factorization({0: 1})	same message — sets and dicts are rejected
lz.factorization(iter([0, 1])), lz.factorization(x for x in [0, 1])	same message — iterators and generators are rejected
lz.factorization([1.5, 2.5]), lz.factorization(np.array([0.0, 1.0]))	same message — floats are rejected
lz.factorization(np.array([[0, 1], [1, 0]]))	same message — 2-D arrays are rejected
lz.factorization([1, "a"])	same message
lz.factorization(["a", 1])	TypeError: 'int' object cannot be converted to 'PyString' — a different message, because a list whose first element is a string commits to the string path
lz.nid(None, "a")	same message as the first row

The module docstring's "any iterable of ints" is inaccurate. What is accepted is anything that can be extracted into a vector of integers, which requires the sequence protocol. Materialise lazy iterables into a list first. See Inputs for the full coercion table, including the bytearray trap.

Wrong argument type or out-of-range integer

Call	Exception
lz.factorization("ab", "x")	TypeError: argument 'partitions': 'str' object cannot be interpreted as an integer
lz.h("abab", partitions=None)	TypeError: argument 'partitions': 'NoneType' object cannot be interpreted as an integer
lz.h("abab", alphabet=2.5)	TypeError: argument 'alphabet': 'float' object cannot be interpreted as an integer
lz.h("abab", log_base=2.718)	TypeError: argument 'log_base': 'float' object cannot be interpreted as an integer
lz.emc("abab", max_block_size="x")	TypeError: argument 'max_block_size': 'str' object cannot be interpreted as an integer
lz.factorization("ab", alphabet=-1)	OverflowError: out of range integral type conversion attempted
lz.h("abab", log_base=-2)	OverflowError: out of range integral type conversion attempted
lz.factorization("ab", jobs=-1)	OverflowError: out of range integral type conversion attempted
lz.h(SEQ, alphabet=2**32)	OverflowError: out of range integral type conversion attempted
lz.factorization(SEQ, partitions=2**31)	OverflowError: out of range integral type conversion attempted
lz.h(SEQ, log_base=2**63)	OverflowError: Python int too large to convert to C long

alphabet, log_base and jobs are unsigned 32-bit: negatives and values ≥ 2³² raise OverflowError, never ValueError. partitions and max_block_size are signed 32-bit, so negatives are accepted there.

Arity and keyword errors

Call	Exception
lz.factorization()	TypeError: factorization() missing 1 required positional argument: 'seq'
lz.nid("ab")	TypeError: nid() missing 1 required positional argument: 'seq2'
lz.factorization("ab", nonsense=1)	TypeError: factorization() got an unexpected keyword argument 'nonsense'
lz.h("abab", 1, None, None, 0, 9)	TypeError: h() takes from 1 to 5 positional arguments but 6 were given

What does not raise

Call	Result
lz.factorization("ab", alphabet=0)	(2, [0, 1, 2]) — clamped to 2
lz.factorization("ab", log_base=0) / log_base=1	clamped to 2
lz.factorization("ab", partitions=-5)	ignored
lz.emc("abab", max_block_size=0) / -100	(0.0, [0.0]) — auto-selected
lz.h(""), lz.lz76("")	0.0, and the full dict

Nothing in the Python layer validates sequence length; every degenerate input returns a value. LZ76 entropy estimates are not meaningful below a few hundred symbols, and the shuffle-based EMC needs n > 50 before the block size leaves single digits.

Module attributes and packaging

Name	Value	Notes
lzcomplexity.__version__	'1.0.2'	comes from the Rust crate version, not from pyproject.toml. Not in __all__, so from lzcomplexity import * does not bind it
lzcomplexity.__all__	['lz76', 'factorization', 'h', 'emc', 'nid']	a star-import binds exactly these five
lzcomplexity.__doc__	the package docstring	what help(lzcomplexity) shows

The package is typed. python/lzcomplexity/py.typed is a git-tracked, zero-byte PEP 561 marker, and __init__.pyi carries the full signatures, so mypy and pyright resolve the API without loading the compiled extension.

lz.lzcomplexity raises AttributeError — that is deliberate

The compiled extension is a submodule named lzcomplexity.lzcomplexity. The package __init__.py deletes the attribute after importing from it, so dir(lzcomplexity) shows only the five public names. The submodule stays in sys.modules and import lzcomplexity.lzcomplexity still works — but the attribute is never re-bound.

>>> hasattr(lz, "lzcomplexity")
False
>>> "lzcomplexity.lzcomplexity" in sys.modules
True

It is namespace hygiene, not encapsulation. Do not depend on the submodule; it is not part of the public API.

inspect.signature reports max_block_size=Ellipsis

Documentation generators render the default as ... rather than -1, because the negative default cannot be expressed as a literal in the generated text signature. The runtime default really is -1 — lz.emc(SEQ) == lz.emc(SEQ, max_block_size=-1) is True. Prefer the .pyi stub or a hand-written signature when generating docs.

>>> lz.emc.__text_signature__
'(seq, partitions=1, alphabet=None, log_base=None, max_block_size=..., jobs=0)'

Wheels are tagged cp39-abi3-<platform>: one wheel per platform covers every Python from 3.9 up. The module built for this page loaded unmodified under CPython 3.14. See Releases.

Threading and the GIL

A call blocks every other Python thread for its whole duration

The extension never releases the GIL — there is no allow_threads call anywhere in the workspace. All heavy work (suffix array, longest-previous-factor table, factorization, the shuffle loop) runs with the GIL held. Measured here: a background Python thread ticking at roughly 17 700 iterations per second managed one single tick during the 3.2-second lz76 call on a 1 500 000-symbol sequence — total starvation. Ctrl-C is not serviced until the call returns, and a GUI or asyncio loop sharing the interpreter will freeze.

Practical consequences:

• concurrent.futures.ThreadPoolExecutor gives no speedup when batching sequences, and sometimes costs. Measured here: four lz76 calls on 500 000-symbol sequences took 2.14 s across four threads against 2.18 s run back to back — four threads bought nothing. Under load the threaded version can run slower still, because the rayon pools of the concurrent calls oversubscribe the cores while the GIL serialises the Python-visible work.
• For Python-level parallelism use multiprocessing or ProcessPoolExecutor, and set RAYON_NUM_THREADS=1 in the workers so the two levels of parallelism do not fight.
• Within a single call, parallelism already happens: the EMC shuffles fan out across the rayon pool, nid runs its four factorizations concurrently, and extras runs its two half-sequence factorizations concurrently. Control it with RAYON_NUM_THREADS; results are bit-identical across thread counts.
• Because the GIL is held throughout and no mutable state is shared, the functions are trivially thread-safe. They are not thread-parallel.

See Performance and Batch and distance workflows.

Migrating from the pre-1.0 API

Pre-1.0	1.0.0	What changed
lz.entropy_density(seq, …)	lz.h(seq, …)	renamed — the function and the lz76(...) dict key
lz.metrics.nid(a, b, …) (v0.10.1) lz.metrics.information_distance(a, b, …) (v0.10.2)	lz.nid(a, b, …)	hoisted to the top level. Which old name you used depends on which release you came from
lz.factors(seq, …)	lz.factorization(seq, …)	folded in — factorization now returns the boundary list too
lz.factorization(seq, …) -> int	-> tuple[int, list[int]]	take [0] for the old value
lz.lz76(seq, …) -> 4-tuple	-> dict with 9 keys	EMC promoted to a nested dict; ε, factors_stddev, the two error estimates and extras are new
lz.emc(seq, …) -> (max_block_size, value, multi_information)	-> (value, summands)	different arity and different element meaning; the other two are now under lz76(...)["emc"]
lz.metrics.rid(a, b, …)	removed	reachable only through v0.10.1; already disabled in the last C++ release
lz.metrics submodule	removed	one metric did not justify a submodule
lz.spectral.psd / entropy / semc	removed	moved to a separate package

Defaults changed too

Parameter	Pre-1.0 default	1.0.0 default	Effect
alphabet	2	None (auto-detect, minimum 2)	non-binary data auto-detects instead of silently assuming binary
log_base	2	None (= detected alphabet size)	h is normalized by default, not reported in bits
jobs	hardware_concurrency()	0	now ignored
partitions	1	1	now ignored
max_block_size	-1	-1	unchanged

The log_base default change silently rescales published numbers

An h value computed with 1.0.0 defaults is not comparable with one from the C++ era. Old entropy_density(seq) was always base 2; new h(seq) uses the detected alphabet size, so for DNA (k = 4) the value is log(2)/log(4) = 0.5× the old one. Pass log_base=2 to reproduce the old numbers.

Separately, EMC numbers are not comparable across three generations of the estimator. Before 0.13.0 it summed absolute complexity differences. From 0.13.0 to 1.0.2 it summed block-entropy first differences, which telescoped to a single block size and could return negative values. Since then it projects the whole block-entropy ladder and returns a non-negative total. Values agree between the last two only where the raw ladder was already monotone. See Effective measure complexity.

A drop-in shim for old call sites:

import lzcomplexity as lz


def entropy_density(seq, **kw):        # was base-2 by default
    kw.setdefault("log_base", 2)
    return lz.h(seq, **kw)


def factorization(seq, **kw):          # was -> int
    return lz.factorization(seq, **kw)[0]


factors = lz.factorization             # was -> (int, list[int])


def emc(seq, **kw):                    # was -> (max_block_size, value, multi_information)
    d = lz.lz76(seq, **kw)["emc"]
    return (d["max_block_size"], d["value"], d["multi_information"])


nid = lz.nid
information_distance = lz.nid          # for code written against 0.10.2 / 0.11.0

lz.metrics.rid and lz.spectral.* have no replacement in this package.

Where to go next

What the inputs accept

The coercion order, the bytearray trap, UTF-8 bytes versus code points, and why [0, 1, 10] is not injective.

lz.factorization(seq)

Reading the numbers

Which of these values to report, which to ignore, and what a given magnitude means.

lz.lz76(seq)

The Rust API

The same five measures from the core crate, where get_shuffle_terms is a real option instead of a hard-coded true.

lzcomplexity_core::lz76