Forgettable Data

In event sourcing, events are immutable — they form the permanent audit log. But regulations like GDPR require the ability to permanently delete personal data on request.

es-entity solves this with the Forgettable<T> wrapper type. Fields marked as Forgettable have their values stored separately from the event data, so they can be deleted independently without rewriting event history.

How It Works

1. When an event is persisted, any Forgettable<T> field values are extracted and stored in a separate _forgettable_payloads table
2. The event itself is stored with null in place of those fields
3. When loading, the payload values are injected back into the event before deserialization
4. Calling forget() deletes the payloads — events remain intact but with null for forgotten fields

Database Setup

You need one additional table alongside your events table:

CREATE TABLE customers (
  id UUID PRIMARY KEY,
  -- `name` is indexed as a forgettable column (see below): it must be
  -- nullable so `forget()` can set it to NULL.
  name VARCHAR,
  email VARCHAR UNIQUE NOT NULL,
  created_at TIMESTAMPTZ NOT NULL
);

CREATE TABLE customer_events (
  id UUID NOT NULL REFERENCES customers(id),
  sequence INT NOT NULL,
  event_type VARCHAR NOT NULL,
  event JSONB NOT NULL,
  context JSONB DEFAULT NULL,
  recorded_at TIMESTAMPTZ NOT NULL,
  UNIQUE(id, sequence)
);

-- The forgettable payloads table
CREATE TABLE customer_forgettable_payloads (
  entity_id UUID NOT NULL REFERENCES customers(id),
  sequence INT NOT NULL,
  payload JSONB NOT NULL,
  UNIQUE(entity_id, sequence)
);

Defining Forgettable Fields

Wrap any personal data fields in Forgettable<T> in your event enum:

extern crate es_entity;
extern crate sqlx;
extern crate serde;
use serde::{Deserialize, Serialize};
use es_entity::*;

es_entity::entity_id! { CustomerId }

#[derive(EsEvent, Debug, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
#[es_event(id = "CustomerId")]
pub enum CustomerEvent {
    Initialized {
        id: CustomerId,
        // `name` is personal data — wrap it in Forgettable
        name: Forgettable<String>,
        // `email` is NOT forgettable — it stays in the event
        email: String,
    },
    NameUpdated {
        name: Forgettable<String>,
    },
    EmailUpdated {
        email: String,
    },
}
fn main() {}

The EsEvent derive macro detects Forgettable fields and generates the extraction/injection code automatically.

Current Limitations

A Forgettable<T> is only handled as a direct, top-level field of an event variant, spelled Forgettable<...>. Anywhere else — nested inside another struct, inside a Vec or Option, or behind a type alias — its value is silently written as null and never stored: permanent data loss. Likewise, renaming a forgettable field’s serialized key (e.g. via #[serde(rename = "...")]) breaks payload recovery and the value reads back as forgotten. Keep forgettable fields top-level and un-renamed. (Compile-time enforcement of these two rules is deferred.)

Accessing Forgettable Values

Forgettable<T> is an opaque type. You cannot pattern-match the inner value directly. Instead, use .value() which returns an Option<ForgettableRef<T>>:

extern crate es_entity;
use es_entity::Forgettable;

// Create with Forgettable::new() or .into()
let name: Forgettable<String> = Forgettable::new("Alice".to_string());
let also_name: Forgettable<String> = "Alice".to_string().into();

// .value() returns Option<ForgettableRef<T>>
// ForgettableRef derefs to T but does NOT implement Serialize
if let Some(val) = name.value() {
    assert_eq!(&*val, "Alice");
}

// Forgettable::forgotten() or Default::default()
let forgotten: Forgettable<String> = Forgettable::forgotten();
let also_forgotten: Forgettable<String> = Default::default();
assert!(forgotten.value().is_none());
fn main() {}

ForgettableRef intentionally does not implement Serialize, preventing accidental re-serialization of personal data into secondary stores.

Hydrating Entities

Keep forgettable data as Forgettable<T> on the entity too, so the entity can faithfully represent the forgotten state (and so the field can back a forgettable index column). Hydration just carries the event’s value through:

extern crate es_entity;
extern crate sqlx;
extern crate serde;
extern crate derive_builder;
use serde::{Deserialize, Serialize};
use derive_builder::Builder;
use es_entity::*;
es_entity::entity_id! { CustomerId }
#[derive(EsEvent, Debug, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
#[es_event(id = "CustomerId")]
pub enum CustomerEvent {
    Initialized { id: CustomerId, name: Forgettable<String>, email: String },
    NameUpdated { name: Forgettable<String> },
    EmailUpdated { email: String },
}
#[derive(EsEntity, Builder)]
#[builder(pattern = "owned", build_fn(error = "EntityHydrationError"))]
pub struct Customer {
    pub id: CustomerId,
    pub name: Forgettable<String>,
    pub email: String,
    events: EntityEvents<CustomerEvent>,
}

impl TryFromEvents<CustomerEvent> for Customer {
    fn try_from_events(events: EntityEvents<CustomerEvent>) -> Result<Self, EntityHydrationError> {
        let mut builder = CustomerBuilder::default();
        for event in events.iter_all() {
            match event {
                CustomerEvent::Initialized { id, name, email } => {
                    builder = builder
                        .id(*id)
                        // Carry the Forgettable through; it is `Some` while live
                        // and `None` (forgotten) once `forget()` has run.
                        .name(name.clone())
                        .email(email.clone());
                }
                CustomerEvent::NameUpdated { name } => {
                    builder = builder.name(name.clone());
                }
                CustomerEvent::EmailUpdated { email } => {
                    builder = builder.email(email.clone());
                }
            }
        }
        builder.events(events).build()
    }
}
impl IntoEvents<CustomerEvent> for NewCustomer {
    fn into_events(self) -> EntityEvents<CustomerEvent> {
        EntityEvents::init(self.id, [CustomerEvent::Initialized {
            id: self.id, name: Forgettable::new(self.name), email: self.email,
        }])
    }
}
pub struct NewCustomer { id: CustomerId, name: String, email: String }
fn main() {}

Repository Configuration

Enable forgettable on the repository with the forgettable attribute:

#[derive(EsRepo)]
#[es_repo(
    entity = "Customer",
    // `name` is a forgettable index column (queryable, NULLed on forget);
    // `email` is a normal column.
    columns(name = "Forgettable<String>", email = "String"),
    forgettable,
)]
pub struct Customers {
    pool: sqlx::PgPool,
}

This generates a forget method on the repository that:

1. Deletes all forgettable payloads for the entity from the database
2. Sets any forgettable index columns to NULL
3. Rebuilds the entity in-place with forgotten fields set to Forgettable::forgotten()

// Load the entity
let mut customer = customers.find_by_id(id).await?;
assert_eq!(customer.name.value().map(|v| v.clone()), Some("Alice".to_string()));

// Forget personal data — updates `customer` in place
customers.forget(&mut customer).await?;

// The entity immediately reflects the forgotten state
assert!(customer.name.is_forgotten());

Custom Queries with es_query!

If you write custom queries using es_query!, you must pass the forgettable_tbl parameter so the generated SQL includes the LEFT JOIN for forgettable payloads:

let query = es_query!(
    entity = Customer,
    sql = "SELECT * FROM customers WHERE email = $1",
    args = [email as String],
    forgettable_tbl = "customer_forgettable_payloads",
);

If you omit forgettable_tbl on an event type that has Forgettable<T> fields, you get a compile-time error:

error: es_query! requires `forgettable_tbl` parameter when the event type has Forgettable<T> fields

This prevents silently loading events without their forgettable data.

Delete and Forgettable

When forgettable is enabled and delete = "soft" is configured, calling delete() also auto-forgets: it deletes all forgettable payloads for the entity and sets any forgettable index columns to NULL (instead of re-persisting the live value). This prevents orphaned personal data from remaining in the database after a soft delete.

#[derive(EsRepo)]
#[es_repo(
    entity = "Customer",
    columns(name = "Forgettable<String>", email = "String"),
    delete = "soft",
    forgettable,
)]
pub struct Customers {
    pool: sqlx::PgPool,
}

// Soft-delete also cleans up forgettable payloads and nulls forgettable columns
customers.delete(customer).await?;

Forgettable Index Columns

forgettable scrubs the event stream (the payloads table). It does not touch columns you materialise into the lookup table via columns(...). If you index a field that is also personal data as a plain column, the value survives forget() in that column — a leak. Declaring the column Forgettable<Inner> (as in the configuration above) closes the gap.

A Forgettable<Inner> column:

• Stores a nullable index column (Inner while live, NULL once forgotten) — the database column must therefore be nullable.
• Is queried by Inner, exactly like a naked column: find_by_name("Alice") / list_by_name(...) behave the same as for name = "String".
• Is set to NULL by forget() — in the same transaction as the payload delete and the in-place rebuild.
• Is set to NULL by soft delete() (auto-forget), instead of re-persisting the live value.

The types must line up: the entity field carries the Forgettable, while the New entity field holds the plain inner value (a freshly-created entity always has it set):

pub struct Customer {
    pub id: CustomerId,
    pub name: Forgettable<String>, // hydrated field is Forgettable
    // ...
}

pub struct NewCustomer {
    pub id: CustomerId,
    pub name: String,              // New entity holds the raw value
    // ...
}

After forget(), the rebuilt entity’s field is Forgettable::forgotten() and the name column in the lookup table is NULL, so the value is retained nowhere the framework materialises it.

Important: The payloads are hard-deleted even when the entity is only soft-deleted. If the entity is later restored, the forgettable fields will remain permanently forgotten.