Mapping to data sources
This is where you connect features to actual data, whether that data is inlined directly in your query or persisted in external databases and services.
All column based mappings follow the same pattern: you declare columns, bind key columns to inputs, and specify a data source via FROM. The only thing that changes is the operator and the source type.
Cardinality constraint: the result of any mapping must return 0 or 1 row per unique input combination. This is enforced through the BOUND TO mechanism. If your source can return multiple rows for a given key, your query will run but return impredictable results.
@fql-playground-hidden(mapping_data_sources_teardown)
The general pattern
FEATURE_NAME := <OPERATOR>(
key_column TYPE BOUND TO SOME_INPUT,
value_column TYPE
FROM <SOURCE>(...)
) You access columns of the result using the [] operator:
FEATURE_NAME[value_column] Column based mappings
These operators return a row. You pick columns from that row with [].
| Operator | Source | What it does | Context |
|---|---|---|---|
INLINE_COLUMNS() | CSV()JSON() | Embeds data directly in the query | Analytics or realtime serving |
EXTERNAL_COLUMNS() | TABLE()SQL() | Maps to columns in external database tables or queries | Analytics |
EXTERNAL_COLUMNS() | VIEW() | Maps to views in external databases | Realtime serving |
The only difference between INLINE_COLUMNS() and EXTERNAL_COLUMNS() is where the data lives. The syntax inside the parentheses is identical.
Text based mappings
These operators return a VARCHAR instead of a row. You parse the result yourself (to JSON or whatever format you need).
| Operator | What it does | Context |
|---|---|---|
EXTERNAL_REDIS() | Maps to a Redis key | Realtime serving |
EXTERNAL_HTTP() | Maps to an HTTP endpoint | Realtime serving |
Sources for INLINE_COLUMNS
CSV
WITH
CUSTOMER_ID := INPUT(BIGINT),
DIM_CUSTOMERS := INLINE_COLUMNS(
customer_id BIGINT BOUND TO CUSTOMER_ID,
name VARCHAR
FROM CSV(
customer_id,name
101,Alice
102,Bob
103,Charly
104,Daniel
105,Elody
106,Francis
)
)
SELECT
CUSTOMER_ID,
DIM_CUSTOMERS,
CUSTOMER_NAME := DIM_CUSTOMERS[name]
FOR
CUSTOMER_ID := BIND_VALUES(ARRAY[101, 102, 103, 104, 105, 106]);| CUSTOMER_ID BIGINT | DIM_CUSTOMERS ROW | CUSTOMER_NAME VARCHAR |
|---|---|---|
| 101 | {customer_id: 101, name: Alice} | Alice |
| 102 | {customer_id: 102, name: Bob} | Bob |
| 103 | {customer_id: 103, name: Charly} | Charly |
| 104 | {customer_id: 104, name: Daniel} | Daniel |
| 105 | {customer_id: 105, name: Elody} | Elody |
| 106 | {customer_id: 106, name: Francis} | Francis |
JSON
WITH
CUSTOMER_ID := INPUT(BIGINT),
DIM_CUSTOMERS := INLINE_COLUMNS(
customer_id BIGINT BOUND TO CUSTOMER_ID,
name VARCHAR
FROM JSON(
[
{"customer_id": 101, "name": "Alice"},
{"customer_id": 102, "name": "Bob"},
{"customer_id": 103, "name": "Charly"},
{"customer_id": 104, "name": "Daniel"},
{"customer_id": 105, "name": "Elody"},
{"customer_id": 106, "name": "Francis"}
]
)
)
SELECT
CUSTOMER_ID,
DIM_CUSTOMERS,
CUSTOMER_NAME := DIM_CUSTOMERS[name]
FOR
CUSTOMER_ID := BIND_VALUES(ARRAY[101, 102, 103, 104, 105, 106]);| CUSTOMER_ID BIGINT | DIM_CUSTOMERS ROW | CUSTOMER_NAME VARCHAR |
|---|---|---|
| 101 | {customer_id: 101, name: Alice} | Alice |
| 102 | {customer_id: 102, name: Bob} | Bob |
| 103 | {customer_id: 103, name: Charly} | Charly |
| 104 | {customer_id: 104, name: Daniel} | Daniel |
| 105 | {customer_id: 105, name: Elody} | Elody |
| 106 | {customer_id: 106, name: Francis} | Francis |
Sources for EXTERNAL_COLUMNS
For the examples below, assume this table exists:
/* SQL */
CREATE SCHEMA IF NOT exists ext;
--
CREATE OR REPLACE TABLE ext.dim_customers AS
SELECT customer_id, name FROM (
VALUES
(101, 'Alice'),
(102, 'Bob'),
(103, 'Charly'),
(104, 'Daniel'),
(105, 'Elody'),
(106, 'Francis')
) as t(customer_id, name);| Count BIGINT |
|---|
| 6 |
TABLE
WITH
CUSTOMER_ID := INPUT(BIGINT),
DIM_CUSTOMERS := EXTERNAL_COLUMNS(
customer_id BIGINT BOUND TO CUSTOMER_ID,
name VARCHAR
FROM TABLE(ext.dim_customers)
)
SELECT
CUSTOMER_ID,
DIM_CUSTOMERS,
CUSTOMER_NAME := DIM_CUSTOMERS[name]
FOR
CUSTOMER_ID := BIND_VALUES(ARRAY[101, 102, 103, 104, 105, 106]);| CUSTOMER_ID BIGINT | DIM_CUSTOMERS ROW | CUSTOMER_NAME VARCHAR |
|---|---|---|
| 101 | {customer_id: 101, name: Alice} | Alice |
| 102 | {customer_id: 102, name: Bob} | Bob |
| 103 | {customer_id: 103, name: Charly} | Charly |
| 104 | {customer_id: 104, name: Daniel} | Daniel |
| 105 | {customer_id: 105, name: Elody} | Elody |
| 106 | {customer_id: 106, name: Francis} | Francis |
SQL
When you need to transform or filter data before mapping, use SQL() instead of TABLE():
WITH
CUSTOMER_ID := INPUT(BIGINT),
DIM_CUSTOMERS := EXTERNAL_COLUMNS(
customer_id BIGINT BOUND TO CUSTOMER_ID,
name VARCHAR
FROM SQL(SELECT customer_id, name FROM ext.dim_customers)
)
SELECT
CUSTOMER_ID,
DIM_CUSTOMERS,
CUSTOMER_NAME := DIM_CUSTOMERS[name]
FOR
CUSTOMER_ID := BIND_VALUES(ARRAY[101, 102, 103, 104, 105, 106]);| CUSTOMER_ID BIGINT | DIM_CUSTOMERS ROW | CUSTOMER_NAME VARCHAR |
|---|---|---|
| 101 | {customer_id: 101, name: Alice} | Alice |
| 102 | {customer_id: 102, name: Bob} | Bob |
| 103 | {customer_id: 103, name: Charly} | Charly |
| 104 | {customer_id: 104, name: Daniel} | Daniel |
| 105 | {customer_id: 105, name: Elody} | Elody |
| 106 | {customer_id: 106, name: Francis} | Francis |
VIEW
Views work the same way as TABLE() but are used in a realtime serving context:
WITH
CUSTOMER_ID := INPUT(BIGINT),
DIM_CUSTOMERS := EXTERNAL_COLUMNS(
customer_id BIGINT BOUND TO CUSTOMER_ID,
name VARCHAR
FROM VIEW(ext.v_dim_customers)
)
SELECT
CUSTOMER_ID,
DIM_CUSTOMERS,
CUSTOMER_NAME := DIM_CUSTOMERS[name]
FOR
CUSTOMER_ID := BIND_VALUES(ARRAY[101, 102, 103, 104, 105, 106]);