CellDSL Tokens⚓︎

CellDSL lets a project give special meaning to authored cell text.

Talika does not ship built-in meanings for words like random, today, current user, or default. Those words are project vocabulary. A CellDSL lets you define that vocabulary once and attach it to the fields where it is allowed.

A table with a project token

Feature: CMS generated content

  Scenario: Use project tokens in content setup
    Given the content items:
      | IDs      | A-1    |
      | Headline | random |
      | Status   | random |
    Then only the headline token is expanded

In this table, random should generate a headline, but it should not generate a status. The same visible cell text can mean something in one field and stay literal in another field.

A token is exact project vocabulary

Use token rules for short, exact cell values such as random, today, none, current user, or published. If the value has parameters, a pattern rule is usually a better fit.

Create a Token DSL⚓︎

Start by creating one CellDSL instance and registering exact tokens on it.

A field-scoped random token

from talika import CellDSL, ColumnTable, RowTable, field, id_field


content_cells = CellDSL()


@content_cells.token("random", fields=("headline",))
def random_headline(context):
    generator = context.user_data["generator"]
    return generator(context.item_id)

The decorated function receives a CellContext. It can read parse-time project data from context.user_data, source identity from context.row and context.column, and record identity from context.item_id when the table has an ID.

The token value is exact. This rule matches the cell text random. It does not match Random, random, random headline, or any other spelling.

Tokens do not normalize text

If your project wants case-insensitive or more flexible syntax, normalize the table before parsing or use a pattern/predicate rule in the later DSL topics. Exact tokens stay exact.

Attach the DSL to Fields⚓︎

A CellDSL is a field parser. Attach it with field(..., parser=...).

A content table using token parsing

class ContentTable(ColumnTable):
    id = id_field("IDs")
    headline = field("Headline", required=True, parser=content_cells)
    status = field("Status", parser=content_cells)

The same parser is attached to headline and status, but the token itself is scoped to headline.

A table using the same word twice

content_table = [
    ["IDs", "A-1"],
    ["Headline", "random"],
    ["Status", "random"],
]

Parsing with project context

record = ContentTable.parse(
    content_table,
    context={
        "generator": lambda item_id: f"Generated headline for {item_id}",
    },
)[0]

Token parsing result

>> record
ContentTable(id='A-1', headline='Generated headline for A-1', status='random')

>> record.headline
'Generated headline for A-1'

>> record.status
'random'

The headline token matched, so the handler generated a value. The status cell also contains random, but the token is not scoped to status, so the value passes through unchanged.

A DSL may be shared by several fields

The DSL can be attached to multiple fields. Field scopes decide which rules apply to which schema attribute.

Use Cell Context⚓︎

Token handlers receive CellContext, not only the raw value. For exact tokens, the value is already known because it is the token being handled.

Inspecting parser context

context_cells = CellDSL()


@context_cells.token("where")
def where(context):
    return {
        "field": context.field_name,
        "label": context.field_label,
        "row": context.row,
        "column": context.column,
        "item_id": context.item_id,
        "source": context.source_value,
    }

A schema for context inspection

class ContextTable(ColumnTable):
    id = id_field("IDs")
    headline = field("Headline", parser=context_cells)

Cell context seen by a token

>> ContextTable.parse([["IDs", "A-1"], ["Headline", "where"]])[0].headline
{'field': 'headline', 'label': 'Headline', 'row': 2, 'column': 2, 'item_id': 'A-1', 'source': 'where'}

The context gives token handlers enough information to generate values that belong to the current record and field. This is why a CMS headline token can use the current item ID, or a date token can read a clock supplied through parse context.

Pass dependencies through parse context

Put generators, clocks, fixtures, configuration, and policy values in parse(..., context={...}). Do not close over test-local mutable state when the context can make the dependency explicit.

Scope Tokens by Python Field Name⚓︎

The fields= argument uses schema attribute names, not table labels.

Field name used for token scope

headline = field("Headline", parser=content_cells)

The field scope is "headline", not "Headline".

This distinction matters because table labels are user-facing vocabulary. They may contain spaces, punctuation, aliases, or old names. Field scopes belong to Python schema code and use the stable attribute name.

Use field names, not labels

A field declared as headline = field("Headline*") is scoped as "headline". Passing "Headline*" to fields= will not match that field.

Scoped Tokens Beat Global Tokens⚓︎

You can register a global token and a field-scoped token with the same exact value.

Global and field-scoped tokens

scoped_cells = CellDSL()


@scoped_cells.token("random")
def global_random(context):
    return "global value"


@scoped_cells.token("random", fields=("headline",))
def scoped_random(context):
    return "headline value"

Fields using the same DSL

class ScopedTable(RowTable):
    headline = field("headline", parser=scoped_cells)
    category = field("category", parser=scoped_cells)

Scoped token precedence

>> ScopedTable.parse([["headline", "category"], ["random", "random"]])[0]
ScopedTable(headline='headline value', category='global value')

For headline, the scoped token wins. For category, no scoped token applies, so the global token handles the value.

This is useful when a broad project token has a sensible default meaning, but one field needs a more specific meaning.

Token precedence is narrow

This precedence rule is for exact tokens with the same value. Pattern, predicate, fallback, and composed DSL precedence are covered separately.

Unmatched Values Pass Through⚓︎

If no token matches, a CellDSL returns the original cell value.

A DSL with no matching tokens

literal_cells = CellDSL()


class LiteralTable(RowTable):
    value = field("value", parser=literal_cells)

Literal value passthrough

>> LiteralTable.parse([["value"], ["literal"]])[0].value
'literal'

This lets one parser accept both special project tokens and ordinary literal values. A headline field can support random while still accepting a normal authored headline.

Keep literals valid

A token DSL does not force every value to be a token. Use it when a field should accept a small set of special values plus ordinary table text.

Registration Is Validated⚓︎

An exact token cannot be empty.

Empty token registration

cells = CellDSL()
cells.token("")

Empty token error

Cell DSL token cannot be empty

The same token and same scope cannot be registered twice.

Duplicate token registration

cells = CellDSL()


@cells.token("random")
def first_random(context):
    return "first"


@cells.token("random")
def second_random(context):
    return "second"

Duplicate token error

Cell DSL token 'random' is already registered for this scope

Duplicate detection is per scope. A global random token and a fields=("headline",) random token can coexist because they do not describe the same rule.

Registration errors happen early

These errors happen when the DSL is defined, not when a feature table is parsed. That helps catch ambiguous project vocabulary during import.

Token Handler Errors Keep Source Context⚓︎

If a token handler raises an exception, Talika wraps it as a field parser failure and keeps the source location of the cell that triggered the token.

A token handler that fails

broken_cells = CellDSL()


@broken_cells.token("broken")
def broken(context):
    raise RuntimeError("generator unavailable")


class BrokenTable(ColumnTable):
    id = id_field("IDs")
    headline = field("Headline", parser=broken_cells)

Parsing the failing token

BrokenTable.parse([["IDs", "A-1"], ["Headline", "broken"]])

Source-aware token diagnostic

Field parser failed: generator unavailable (code=parser_failed, schema=BrokenTable, field='Headline', row=2, column=2, item_id='A-1', value='broken'). Hint: Check the cell value or adjust the field parser for this syntax.

The failure points to Headline row 2, column 2, the authored cell containing broken. That is the right place for the table author or test maintainer to start looking.

Keep token handlers deterministic

A token handler is still a parser. It should return a value for the current cell or raise a clear exception. Avoid hidden side effects that make the same table parse differently across runs.

Choose Tokens for Stable Vocabulary⚓︎

Use exact tokens for values that should read like named project vocabulary:

random
today
current user
none
published
default

Avoid exact tokens for values that contain parameters, counts, IDs, or ranges. Those belong in pattern, predicate, or transformation rules where the variable part can be parsed deliberately.

The best token names are short, predictable, and documented by the schema that uses them. A reader should be able to see random in a field and know that the project has assigned that word a specific meaning for that field.

Prefer clear token names

A good token is small, predictable table language. It removes repeated setup code and keeps authored examples readable.