Your Control Plane on Postgres
The bfx.* run-audit schema and functions, now native plpgsql you own
June 22, 2026
Every BimlFlex load that runs leaves a trail: when it started, whether it finished, how many rows moved, what the running config variables were, and what blew up if anything did. That trail lives in the BimlCatalog, the control framework BimlFlex deploys to orchestrate execution and record the audit history of every run. For years that meant SQL Server. If your platform of record is PostgreSQL, that was a wall: your warehouse could run anywhere, but the framework watching it could not.
That wall is gone. BimlFlex now ships a complete PostgreSQL BimlCatalog, a parallel implementation written in native plpgsql that you deploy into a Postgres database you own. Same bfx.* schema, same execution and logging tables, same config and audit semantics as the SQL Server build. Not a connector, not a hosted service you rent access to. A schema and a set of functions that live in your database.
What you actually deploy
The Postgres build creates two schemas: bfx for the live control tables and archive for the long-term copies. Inside bfx you get the same operational tables the SQL Server framework has always had. The execution table tracks every run, including a parent/child hierarchy so a batch and its children stay linked:
CREATE TABLE IF NOT EXISTS bfx.execution (
execution_id BIGSERIAL PRIMARY KEY,
parent_execution_id BIGINT,
external_execution_id VARCHAR(100),
package_id INTEGER,
execution_status CHAR(1),
next_load_status CHAR(1),
start_time TIMESTAMPTZ DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
end_time TIMESTAMPTZ,
batch_start_time TIMESTAMP(7),
duration INTEGER GENERATED ALWAYS AS (EXTRACT(EPOCH FROM (end_time - start_time))) STORED
);
That duration column is a nice touch the Postgres version leans into: it is a stored generated column, so the elapsed seconds compute themselves from start_time and end_time. You query it, you never maintain it.
Alongside execution you get execution_error for failures, package for the package and project definitions, process and process_log for in-flight status, config_variable and its log table for dynamic configuration, settings for framework behavior toggles, and task_execution for task-level timing. Row counts land in bfx.row_count, and this is the table worth reading closely:
CREATE TABLE IF NOT EXISTS bfx.row_count (
row_count_id BIGSERIAL PRIMARY KEY,
execution_id BIGINT NOT NULL,
component_name VARCHAR(200) NOT NULL,
object_name VARCHAR(200) NOT NULL,
count_type VARCHAR(20) NOT NULL,
row_count INTEGER NOT NULL,
column_sum NUMERIC(38, 4),
column_name VARCHAR(500),
audit_date TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP
);
The column_sum NUMERIC(38,4) is the same reconciliation control the SQL Server catalog carries. A row count tells you how many rows moved. A column control total tells you whether the values inside them are intact. Two loads with matching row counts but a drifted sum are a reconciliation problem you can query, and the Postgres build gives you that column with the same precision.
The functions do the work
A schema is just a place to put rows. The logic that BimlFlex calls at runtime lives in plpgsql functions, and they carry the same names and the same responsibilities as the SQL Server procedures.
bfx.log_execution_start is the heaviest of them. When a package kicks off, it resolves or creates the package record, reads the retry limit from bfx.settings, looks at the last execution to decide whether this run is a fresh start, a continuation, or a retry, writes the new execution row, and returns the execution context the caller needs. It is the function that decides, on a failed prior load, whether the next run resumes or restarts. That restart-or-resume decision is the same state machine we wrote about in the BimlCatalog restart and recovery walkthrough, now running on Postgres.
bfx.log_execution_end closes a run: it stamps the end time, flips status to success, aborts any stale executions still marked running for the same package, and on a batch completion sets the children up for the next load. bfx.log_execution_error does the failure path: mark the run failed, set the next load to reload, abort older unclosed runs, and insert the error description into bfx.execution_error. bfx.log_row_count upserts into the row-count table, inserting on first sight of a component and updating on a repeat. bfx.set_config_variable and bfx.set_config_variable_by_id write runtime configuration and, when the framework's audit setting is on, snapshot the prior value into the audit log.
One detail in log_execution_end and its siblings is worth calling out because it shows the framework is genuinely configuration-driven, not hardcoded. The timestamp every function records is governed by a setting:
SELECT CASE
WHEN COALESCE((
SELECT setting_value
FROM bfx.settings
WHERE setting_code = 'BimlFlex' AND setting_key = 'UseGETUTCDATE'
LIMIT 1
), 'N') = 'Y' THEN now() AT TIME ZONE 'UTC'
ELSE now()
END
Flip UseGETUTCDATE in your settings and every logged time switches between UTC and local. The behavior follows the setting, the same way it does on SQL Server.
Archival is built in
Operational metadata grows without bound if you let it. A busy warehouse can write millions of execution, row-count, and config-variable rows a year, and a control framework that never prunes itself eventually becomes the slow part of your morning. The Postgres BimlCatalog handles this directly.
There is an archival function per table family: bfx.archive_execution, bfx.archive_process, bfx.archive_row_count, bfx.archive_task_execution, and bfx.archive_config_variable. Each reads two settings, a retention period and an archive period, and moves aging rows out of the live bfx tables into the archive schema, then prunes the archive itself once rows pass the longer archive window. The archive_all procedure runs the full set in order:
CREATE OR REPLACE PROCEDURE bfx.archive_all()
LANGUAGE plpgsql
AS $procedure$
BEGIN
CALL bfx.archive_row_count();
CALL bfx.archive_task_execution();
CALL bfx.archive_config_variable();
CALL bfx.archive_execution();
CALL bfx.archive_process();
END;
$procedure$;
And it is wired to run on its own. When a batch finishes, log_execution_end checks the AutoArchive setting; if it is on, the batch completion triggers archive_all. Your operational history trims itself as part of the normal run cycle, with the boundary controlled by a number you set rather than a maintenance job you have to remember to write. The retention logic has a floor built in: a period under seven days is ignored, so a fat-fingered setting can't quietly start deleting yesterday's runs.
Why customer-owned matters
This is the part the architecture decision turns on. The control plane is not in someone else's cloud. It is a schema and a set of functions in a database you administer, back up, query, and keep. You can read your own execution history with plain SQL. You can join bfx.execution to bfx.row_count to build whatever reconciliation report your auditors want. You can point a dashboard at it. You can move the database between Postgres hosts. Nothing about your run history depends on a vendor keeping a service online or on a subscription staying current.
That is a different posture from the SaaS data-automation tools that keep orchestration and run metadata locked inside their platform. We have made the broader argument before, that generated, portable assets beat rented runtimes, in the asset-versus-liability piece. The Postgres BimlCatalog is that principle applied to the control plane: the framework that records and governs your loads is yours, on the database engine you already standardized on. And because the catalog is a deployable artifact, it versions like the rest of your project. Track it in source control the way you track the rest of a BimlFlex project, promote it through environments, and you have a control plane that moves with your code.
If you have followed how BimlFlex integrates with the BimlCatalog, nothing about the model changes here. The contract is identical: packages log starts and ends, row counts and column sums get captured, config variables get tracked and audited, failures get recorded and recovered from. What changed is the engine underneath. The same control framework, the same bfx.* semantics, now native to Postgres and entirely in your hands.
