Dictionary

Dictionary functions allow you to reuse transformation values across different entities and fields, ensuring consistency in your anonymized data. When a value is transformed once, the dictionary stores this transformation so that identical input values will always produce the same output, regardless of where they appear in your data.

How Dictionary Works

The dictionary maintains a mapping between original values and their transformed counterparts using a hashing mechanism. When enabled, transformations are stored in scopes that determine when and how values should be reused.

Dictionary Modes

There are several modes that control how dictionary values are reused:

Dictionary Modes Summary

The following table summarizes all available dictionary modes, when to use them, and links to their detailed explanations:

Mode	Description	When to Use	Link
Field	Same entity and field will be transformed in the same way	When you need different transformations for the same value in different fields	Field Mode
Label	Fields labelled with the same label will be transformed in the same way	When you want consistent anonymization across related data types (default mode)	Label Mode
Field Name	Values with the same field name across different entities will be transformed consistently	When fields with identical names should have consistent transformations regardless of labels	Field Name Mode
Global	All values stored in the dictionary will be reused regardless of entity, field, or label	When you want all identical values in your project to transform consistently	Global Mode
Row	Uses row-based scoping for transformations	When values within the same row should be consistent but different across rows	Row Mode
Related	Uses relationship-based scoping for transformations	When you want consistency across related tables according to defined relationships	Related Mode
User Scope	Allows you to define custom scoping for dictionary transformations	When you need precise control over which specific fields share transformation consistency	User Scope Mode
Disabled	Dictionary is disabled - transformations will not be stored or reused	When you want maximum randomness and no consistency in transformations	Disabled Mode

Field Mode

Matches in the same entity and field will be transformed in the same way. Even if there are matches in other entities or fields, they will be ignored.

In Field mode, the dictionary scope is limited to the specific combination of entity and field. This means that if the same value appears in different fields or entities, it will be transformed differently each time.

Example: If you have a "Customers" table with a "Name" field and an "Orders" table with a "Name" field, the value "John" in Customers/Name will be transformed differently than "John" in Orders/Name.

Visual representation:

Customers Table             Orders Table
┌─────────────┐             ┌─────────────┐
│ Name        │             │ Name        │
├─────────────┤             ├─────────────┤
│ John        │ ──────────▶ │ Mark        │  (Different transformation)
└─────────────┘             └─────────────┘

┌─────────────┐             ┌─────────────┐
│ Name        │             │ Name        │
├─────────────┤             ├─────────────┤
│ John        │ ──────────▶ │ Sarah       │  (Different transformation)
└─────────────┘             └─────────────┘

In Field Mode, each entity-field combination maintains its own dictionary scope, so the same input value can produce different outputs depending on where it appears.

To use this mode, select "Field" from the dictionary mode dropdown in the UI.

Label Mode (Default)

Matches on fields labelled with the same label will be transformed in the same way. This is the most common mode as it ensures consistent anonymization across related data types.

In Label mode, the dictionary scope is based on field labels. This means that if two fields have the same label (e.g., "person/name"), they will consistently transform identical values to the same output, regardless of which entity they belong to.

Example: If you have a "Customers" table with a "Name" field labelled as "person/name" and an "Employees" table with a "Full Name" field also labelled as "person/name", the value "John Smith" will be transformed to the same anonymized value in both fields.

Visual representation:

Customers Table             Employees Table
┌────────────────┐          ┌─────────────────┐
│ Name           │          │ Full Name       │
│ [person/name]  │          │ [person/name]   │
├────────────────┤          ├─────────────────┤
│ John Smith     │ ────────▶ │ John Smith      │
└────────────────┘          └─────────────────┘
       │                           │
       ▼                           ▼
┌────────────────┐          ┌─────────────────┐
│ Mark Johnson   │ ◀──────── │ Mark Johnson    │  (Same transformation)
└────────────────┘          └─────────────────┘
                           │ (Dictionary     │
                           │  scope shared)  │
                           └─────────────────┘

In Label Mode, fields with the same label share a dictionary scope ensuring consistent transformation across different entities.

To use this mode, select "Label" from the dictionary mode dropdown in the UI.

Field Name Mode

Values with the same field name across different entities will be transformed consistently.

In Field Name mode, the dictionary scope is based on the field name itself, regardless of which entity it belongs to. This means that fields with identical names (e.g., "email") across different tables will transform the same values consistently.

Example: If you have an "email" field in a "Users" table and an "email" field in a "Subscribers" table, the value "john@example.com" will be transformed to the same anonymized value in both fields, even if they don't share the same labels.

Visual representation:

Users Table             Subscribers Table
┌────────────────────┐   ┌────────────────────┐
│ email              │   │ email              │
├────────────────────┤   ├────────────────────┤
│ john@example.com   │ ─▶ │ john@example.com   │
└────────────────────┘   └────────────────────┘
        │                       │
        ▼                       ▼
┌────────────────────┐   ┌────────────────────┐
│ user123@email.com  │ ◀─ │ user123@email.com  │ (Same transformation)
└────────────────────┘   └────────────────────┘
        (Dictionary scope shared based on field name)

In Field Name Mode, any fields with the same name share a dictionary scope regardless of their entity, ensuring consistent transformation of identical field names across different tables.

To use this mode, select "Field Name" from the dictionary mode dropdown in the UI.

Global Mode

All values that have already been stored in the dictionary will be reused regardless of the entity and field in which they have been found or the label assigned to it.

In Global mode, there is a single shared dictionary scope for all transformations in your project. This means that once a value is transformed anywhere in your project, all subsequent occurrences of that same value will be transformed identically, regardless of entity, field, or label.

Example: If "John Smith" is first transformed to "Mark Johnson" in a "Customers" table "Name" field, then all future occurrences of "John Smith" anywhere in your project (whether in "Employees" table "Full Name" field or "Orders" table "Customer Name" field) will also be transformed to "Mark Johnson".

Visual representation:

Project Dictionary Scope (Global)
┌──────────────────────────────┐
│ John Smith ──▶ Mark Johnson  │
└──────────────────────────────┘
            ▲
            │
            ▼
Customers Table        Employees Table        Orders Table
┌──────────────┐       ┌──────────────┐       ┌──────────────┐
│ Name         │       │ Full Name    │       │ Customer Name│
├──────────────┤       ├──────────────┤       ├──────────────┤
│ John Smith   │ ──▶   │ John Smith   │ ──▶   │ John Smith   │
└──────────────┘       └──────────────┘       └──────────────┘
       │                       │                       │
       ▼                       ▼                       ▼
┌──────────────┐       ┌──────────────┐       ┌──────────────┐
│ Mark Johnson │ ◀──   │ Mark Johnson │ ◀──   │ Mark Johnson │
└──────────────┘       └──────────────┘       └──────────────┘

In Global Mode, there is a single dictionary scope for the entire project. Once a value is transformed, all identical values throughout your project will be transformed to the same output, regardless of entity, field, or label.

To use this mode, select "Global" from the dictionary mode dropdown in the UI.

Row Mode

Uses row-based scoping for transformations.

In Row mode, the dictionary scope is based on entire rows of data. This means that transformations within the same row will be consistent, but the same value in different rows may be transformed differently.

Example: If you have a row with "John Smith" in the "Name" field and "123 Main St" in the "Address" field, both values will be consistently transformed within that row. However, "John Smith" in a different row might be transformed to a different value.

Visual representation:

Customer Table (Row Mode)
┌─────────────────────────────────┐
│ Row ID │ Name           │ Addr │
├────────┼────────────────┼──────┤
│ 1      │ John Smith     │ 123  │ ◀─ Same row, consistent transformation
│ 2      │ John Smith     │ 456  │ ◀─ Different row, different transformation
└────────┴────────────────┴──────┘
         │                │
         ▼                ▼
┌─────────────────────────────────┐
│ Row ID │ Name           │ Addr │
├────────┼────────────────┼──────┤
│ 1      │ Mark Johnson   │ 789  │ ◀─ Values within same row are consistent
│ 2      │ Sarah Williams │ 456  │ ◀─ Same input, different output due to row scope
└────────┴────────────────┴──────┘

In Row Mode, dictionary scope is limited to each individual row. Values within the same row are transformed consistently, but identical values in different rows may have different transformations.

To use this mode, select "Row" from the dictionary mode dropdown in the UI.

Uses relationship-based scoping for transformations.

In Related mode, the dictionary scope is based on relationships between entities in your data model. This ensures that values maintain consistency across related tables according to their defined relationships.

Example: If you have a "Customers" table and an "Orders" table with a relationship between them, customer names will be consistently transformed across both tables based on their relationship. When "John Smith" is transformed in the Customers table, any related orders will reference the same transformed name.

Visual representation:

Customers Table           Orders Table
┌─────────────────┐       ┌──────────────────┐
│ ID │ Name       │       │ OrderID │ CustID │
├────┼────────────┤       ├─────────┼────────┤
│ 1  │ John Smith │ ◀──── │ 101     │ 1      │
│ 2  │ Jane Doe   │ ◀──── │ 102     │ 2      │
└────┴────────────┘       └─────────┴────────┘
     │                         │
     ▼                         ▼
┌─────────────────┐       ┌──────────────────┐
│ ID │ Name       │       │ OrderID │ CustID │
├────┼────────────┤       ├─────────┼────────┤
│ 1  │ Mark Johnson │       │ 101     │ 1      │
│ 2  │ Sarah Williams │       │ 102     │ 2      │
└────┴────────────┘       └─────────┴────────┘
                               │
                               ▼
                      ┌────────────────────┐
                      │ Customer Name      │
                      ├────────────────────┤
                      │ Mark Johnson       │ ◀─ Consistent with Customers table
                      │ Sarah Williams     │ ◀─ Consistent with Customers table
                      └────────────────────┘

In Related Mode, dictionary scope follows defined relationships between entities. When a value is transformed in one table, all related occurrences in connected tables will use the same transformation.

To use this mode, select "Related" from the dictionary mode dropdown in the UI.

User Scope Mode

Allows you to define custom scoping for dictionary transformations.

In User Scope mode, you can create your own custom dictionary scopes that can be shared across specific fields of your choice. This gives you precise control over which fields should share transformation consistency.

Example: You might create a custom scope called "customer-identifiers" and apply it to fields like "Customer Name", "Billing Contact", and "Shipping Contact". This ensures that identical names in these fields are consistently transformed, while other fields use different scopes.

Visual representation:

User Defined Scope: "customer-identifiers"
┌─────────────────────────────────────────────┐
│ Cust Name   Bill Cont   Ship Cont          │
├─────────────────────────────────────────────┤
│ John Smith  John Smith  John Smith         │
└─────────────────────────────────────────────┘
      │           │           │
      ▼           ▼           ▼
┌─────────────────────────────────────────────┐
│ Mark Johnson Mark Johnson Mark Johnson      │ ◀─ Consistent transformation
└─────────────────────────────────────────────┘

Other Fields (Different Scopes)
┌────────────────┐    ┌─────────────────┐
│ Email          │    │ User ID         │
├────────────────┤    ├─────────────────┤
│ John Smith     │    │ John Smith      │
│ (different     │    │ (different      │
│  transformation)│   │  transformation)│
└────────────────┘    └─────────────────┘

In User Scope Mode, you define custom scopes that can span across any fields you choose, providing precise control over which values should share consistent transformations.

To use this mode, select "User Scope" from the dictionary mode dropdown and enter a custom scope name in the scope text field.

Disabled Mode

Dictionary is disabled - transformations will not be stored or reused.

In Disabled mode, each transformation is generated independently without checking or storing results in the dictionary. This means that identical values may be transformed differently each time they appear, providing maximum randomness but no consistency.

Example: If the name "John Smith" appears multiple times in your data, it might be transformed to "Mark Johnson", "Sarah Williams", and "Robert Davis" in different occurrences.

Visual representation:

Disabled Mode - No Dictionary Scope
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│ Customer Name   │    │ Employee Name   │    │ Order Name      │
├─────────────────┤    ├─────────────────┤    ├─────────────────┤
│ John Smith      │    │ John Smith      │    │ John Smith      │
└─────────────────┘    └─────────────────┘    └─────────────────┘
       │                      │                      │
       ▼                      ▼                      ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│ Random Output 1 │    │ Random Output 2 │    │ Random Output 3 │
├─────────────────┤    ├─────────────────┤    ├─────────────────┤
│ Mark Johnson    │    │ Sarah Williams  │    │ Robert Davis    │
└─────────────────┘    └─────────────────┘    └─────────────────┘

In Disabled Mode, each transformation is generated independently without any dictionary scope, meaning identical input values can produce completely different output values each time they appear.

To use this mode, select "Disabled" from the dictionary mode dropdown in the UI.

Configuration Options

With Options

When enabled, function options are included in the dictionary key, ensuring that identical values with different transformation options are stored separately.

Example: If you have a Name field being transformed with a person function, and you use the same input "John Smith" but with different options (e.g., one with gender: 'male' and another with gender: 'female'), enabling "With Options" will store these as separate dictionary entries, allowing for different transformations based on the options provided.

Visual representation:

With Options Enabled
┌────────────────────────────────────┐
│ "John Smith" + {gender: 'male'}    │ ──▶ "Mark Johnson"
│ "John Smith" + {gender: 'female'}  │ ──▶ "Sarah Williams"
└────────────────────────────────────┘

Without Options Enabled
┌────────────────────────────────────┐
│ "John Smith"                       │ ──▶ "Mark Johnson"
│ "John Smith"                       │ ──▶ "Mark Johnson"  (Same transformation)
└────────────────────────────────────┘

To use this option, check the "With Options" checkbox in the dictionary configuration panel.

Nulls Handling

When enabled, null values will also be stored and reused in the dictionary.

When disabled (default), null values are not stored in the dictionary and will be processed independently each time they are encountered, potentially resulting in different transformations or handling behaviors.

Example: With Nulls Handling enabled, all NULL values in a "Last Name" field might be consistently transformed to "Anonymous", whereas with it disabled, they might be left as NULL or handled differently based on the transformation function.

Visual representation:

Nulls Handling Enabled
┌─────────────────┐
│ Last Name       │ ──▶ "Anonymous"
│ NULL            │ ──▶ "Anonymous"
│ NULL            │ ──▶ "Anonymous"  (Consistent transformation)
└─────────────────┘

Nulls Handling Disabled
┌─────────────────┐
│ Last Name       │ ──▶ "Anonymous"
│ NULL            │ ──▶ "Unknown"
│ NULL            │ ──▶ "N/A"        (Different transformations)
└─────────────────┘

To use this option, check the "Nulls Handling" checkbox in the dictionary configuration panel.

Scope

Defines a custom scope name for dictionary transformations when using User Scope mode.

This option allows you to create named scopes that can be shared across different fields, ensuring consistent transformations for values within the same scope while maintaining different transformations for values in different scopes.

Visual representation:

User Scope: "customer-data"
┌────────────────────┐
│ Customer Name      │ ──▶ "Mark Johnson"
│ Billing Contact    │ ──▶ "Mark Johnson"   (Same scope)
└────────────────────┘

User Scope: "employee-data"
┌────────────────────┐
│ Employee Name      │ ──▶ "Sarah Williams"
│ Manager Name       │ ──▶ "Sarah Williams" (Same scope)
└────────────────────┘

To use this option, select "User Scope" from the mode dropdown and enter your custom scope name in the scope text field.

Label

Explicitly defines which label to use for dictionary scoping, overriding the field's automatic label detection.

This option is useful when you want a field to be treated as a different data type for dictionary purposes, allowing you to group fields with different names but similar semantic meaning under the same transformation scope.

Visual representation:

Automatic Label Detection
┌────────────────────┐
│ Company Name       │ [company/name] ──▶ "GloboCorp"
│ Client Name        │ [client/name]  ──▶ "John Smith"
└────────────────────┘

Override Label to "person/name"
┌────────────────────┐
│ Company Name       │ [person/name]  ──▶ "Mark Johnson"
│ Client Name        │ [person/name]  ──▶ "Mark Johnson" (Consistent with override)
└────────────────────┘

To use this option, enter the desired label in the "Label" text field in the dictionary configuration panel.

Dictionary Function Access

The dictionary functionality can be accessed directly through UI options in your transformation rules:

Using Dictionary Scopes

You can specify which dictionary scope to use for storing and retrieving transformations through dropdown selections and text fields in the UI.

Visual representation:

Dictionary Configuration Panel
┌─────────────────────────────────┐
│ Mode:     [▼ Label]            │
│ Replace Label: [___________]   │
│ Scope:    [___________]       │
│ With Options: [✓]              │
│ Nulls Handling: [ ]            │
└─────────────────────────────────┘

Setting Custom Transformations

You can explicitly set custom transformations for specific values using the dictionary configuration panel.

Visual representation:

Custom Transformation Setting
┌─────────────────────────────────┐
│ Original Value: "John Smith"   │
│ Transformed Value: "Mark Johnson" │
│ Scope: "customer-identifiers"  │
│ [ Save Transformation ]        │
└─────────────────────────────────┘

Retrieving Stored Transformations

Previously stored transformations can be retrieved through the automatic dictionary matching process when the same values are encountered.

Visual representation:

Dictionary Lookup Process
┌─────────────────────────────────┐
│ Input: "John Smith"            │
│ Check Dictionary:              │
│   Scope: "person/name"         │
│   Hash: abc123...              │
│ Found: "Mark Johnson"          │
│ Output: "Mark Johnson"         │
└─────────────────────────────────┘

Replace Label

The Replace Label option allows you to specify which label should be used for dictionary operations instead of automatically detecting the field's label.

This is useful when you want a field to be treated as a different data type for dictionary purposes, ensuring consistent transformations across semantically similar fields regardless of their automatic label detection.

Visual representation:

Before Replace Label
┌─────────────────────────────────┐
│ Field: Company Name             │
│ Auto Label: [company/name]      │ ──▶ "GloboCorp"
│ Field: Client Name              │
│ Auto Label: [client/name]       │ ──▶ "John Smith"
└─────────────────────────────────┘

After Replace Label with "person/name"
┌─────────────────────────────────┐
│ Field: Company Name             │
│ Override Label: [person/name]   │ ──▶ "Mark Johnson"
│ Field: Client Name              │
│ Override Label: [person/name]   │ ──▶ "Mark Johnson" (Consistent)
└─────────────────────────────────┘

To use this option, enter the desired label in the "Replace Label" text field in the dictionary configuration panel. This is useful when you want a field to be treated as a different data type for dictionary purposes.

Examples

Basic Dictionary Usage

In the UI, you can enable dictionary functionality by selecting an appropriate mode from the dictionary mode dropdown. The default "Label" mode is suitable for most use cases where you want consistent anonymization across related data types.

Visual representation:

Dictionary Mode Selection
┌─────────────────────────────────┐
│ Dictionary Mode: [▼ Label]      │ ◀─ Default mode
│           Options:              │
│           - Field               │
│           - Label (default)     │
│           - Field Name          │
│           - Global              │
│           - Row                 │
│           - Related             │
│           - User Scope          │
│           - Disabled            │
└─────────────────────────────────┘

Custom Scope with Options

To create custom scopes for your dictionary transformations, select "User Scope" from the mode dropdown and enter a custom scope name in the scope text field. You can also use the "With Options" checkbox to include function options in the dictionary key.

Visual representation:

User Scope Configuration with Options
┌─────────────────────────────────┐
│ Mode: [▼ User Scope]           │
│ Scope: "customer-identifiers"  │
│ With Options: [✓]              │ ◀─ Includes function parameters
│ Nulls Handling: [ ]            │
└─────────────────────────────────┘

Global Dictionary

To reuse all transformations globally regardless of entity, field, or label, select "Global" from the dictionary mode dropdown.

Visual representation:

Global Mode Selection
┌─────────────────────────────────┐
│ Dictionary Mode: [▼ Global]    │ ◀─ Single scope for entire project
└─────────────────────────────────┘

Global Dictionary Scope
┌─────────────────────────────────┐
│ "John Smith" ──▶ "Mark Johnson"│ ◀─ Used everywhere in project
│ "jane@email.com" ──▶ "sarah@email.com"│
│ "Company ABC" ──▶ "GloboCorp"  │
└─────────────────────────────────┘

Using Replace Label

In the UI, you can override the automatic label detection by entering a value in the "Replace Label" text field. This is useful when you want a field to be treated as a different data type for dictionary purposes.

Visual representation:

Replace Label Configuration
┌─────────────────────────────────┐
│ Mode: [▼ Label]                │
│ Replace Label: "person/name"   │ ◀─ Override automatic detection
│ Scope: ""                      │
│ With Options: [ ]              │
│ Nulls Handling: [ ]            │
└─────────────────────────────────┘

On this page