91 Sources, One Model
Native Source Connectors Are Generated, Not Hand-Wired
June 25, 2026
Most data platforms treat the source side as someone else's problem. They transform what already landed in the warehouse and assume a separate replication tool dragged it in. That assumption is fine until you count the sources. A real enterprise estate is not one clean Snowflake feed. It is SAP for finance, Salesforce for the pipeline, Dynamics for service, ServiceNow for tickets, an Oracle ERP nobody is allowed to touch, and a Teradata box still running the nightly batch from 2011.
Each of those wants its own connector, its own auth, its own dataset shape. Wire one by hand and it works. Wire forty by hand and you have a maintenance liability that grows every time someone rotates a key or a vendor ships a new connector version. The interesting question is not whether a tool can read SAP. It is whether the source layer is something you author once as metadata, or something you keep hand-building forever.
The source side is the hard part
The modern data stack quietly outsourced ingestion. dbt is explicit about this: it transforms SQL inside your warehouse and, by its own description, does not import source metadata and does not do change data capture. You bring the data; dbt reshapes it. Coalesce is the same story one layer up. It generates native warehouse SQL but does not extract from SAP. SAP data has to land in the warehouse through a separate replication tool first, and then Coalesce transforms it.
That split is a real cost, not a technicality. It means the SAP connector, the Salesforce connector, and the Oracle extractor live in a different product with a different bill, a different metadata model, and a different team who owns the breakage. Your warehouse automation tool never sees the source. It only sees the landing zone someone else filled.
BimlFlex takes the opposite position. The source connector is part of the same metadata model that generates your staging, your Data Vault, and your marts. You describe the connection once, and the pipeline that reads it is generated output you own.
What 91 connectors actually means
When you add a source connection in BimlFlex, you are not picking from a short list of the usual suspects. The native source library covers 91 connector types, and the breadth is the point. The SAP family alone is six distinct connectors: SAP ECC, SAP BW, SAP HANA, SAP Table, SAP Open Hub, and SAP Cloud for Customer. Each one is a different extraction surface, and SAP shops know they are not interchangeable. Pulling from a BW Open Hub is a different job than reading an ECC table, and the connector list reflects that instead of flattening it into a single "SAP" checkbox.
The SaaS sources are just as granular. Salesforce is three connectors, not one: core Salesforce, Salesforce Marketing Cloud, and Salesforce Service Cloud. Dynamics is three as well: Dynamics, Dynamics AX, and Dynamics CRM, alongside Common Data Service for Apps. Then the long tail most projects actually hit: ServiceNow, Oracle, Teradata, Snowflake, Google BigQuery, Amazon Redshift, MongoDB, Cassandra, Db2, Netezza, and dozens more across files, OData, REST, and the big object stores.
This is a source library, not a marketing bullet. The reason it matters is what comes next: you do not wire any of them by hand.
Generated datasets and linked services
In Azure Data Factory, every source needs two artifacts. A linked service holds the connection: the host, the auth method, the credentials reference. A dataset describes the shape of the thing you are reading: the table, the object, the file path. Hand-author these and you are writing JSON for every source, keeping the auth blocks consistent, and re-checking them every time ADF revises a connector version.
BimlFlex generates both from your metadata. You define the connection once in the connection editor, choosing the connection type and the authentication method the source supports. Teradata, for example, offers Windows Authentication or Teradata Authentication. Oracle offers Windows Authentication or Oracle Authentication. Dynamics CRM prompts you for the organization URL in its native form, https://<organization-name>.crm[x].dynamics.com. You fill in what the source needs, and BimlFlex emits the native ADF linked service and dataset JSON for it.
The generated dataset is ordinary ADF JSON, the kind your data engineers already read and your Git history already tracks:
{
"name": "ds_Salesforce_Account",
"properties": {
"linkedServiceName": {
"referenceName": "ls_Salesforce_Source",
"type": "LinkedServiceReference"
},
"type": "SalesforceObject",
"typeProperties": {
"objectApiName": "Account"
}
}
}
Nothing about that file is exotic. That is the point. It is the artifact a platform engineer would have written by hand, except you did not write it, and you will not maintain it. When the model changes, you regenerate. When you add the next forty Salesforce objects, you add forty rows of metadata, not forty JSON files.
The same model that builds everything else
The reason this works is that the source connector is not bolted onto a separate ingestion product. It is the front of the same pipeline that builds the rest of the warehouse. The connection you defined feeds the staging load, which feeds the Data Vault, which feeds the marts, all from one metadata model and all regenerable to the target you choose. We covered the cloud side of that flow in building and deploying for Azure Data Factory, and the warehouse-specific version in the walkthrough of BimlFlex loading Snowflake from source to staging.
Breadth does not mean every source maps to a native ADF connector forever. When a source sits outside the native list, BimlFlex still has a path: land it first, then process it with mapping data flows, which is exactly the pattern in connecting to unsupported data sources. And the same metadata discipline that handles relational tables also handles files and semi-structured payloads, which is the subject of one framework, all formats.
Why generated beats wired
Hand-wired connectors are fine in the demo and expensive in year three. Every one is a small artifact someone has to keep current: the auth block, the dataset shape, the connector version ADF just deprecated. Multiply that by the real source count in an enterprise and the source layer becomes its own backlog.
Generated connectors invert the economics. The 91-source library means the connector you need is usually already in the box, defined as a type you select rather than JSON you write. Defining it as metadata means the ADF datasets and linked services are output, not assets you babysit. And because that metadata sits in the same model that generates your staging and your Data Vault, the source side is not a separate product with a separate bill. It is the first step of a pipeline you regenerate whenever the model changes. The breadth is generated. So is the maintenance you no longer do.
