DataSources

DataSources are JavaScript objects that dropdown fields and some of the transformers can query at runtime. They enable dynamic, dependent dropdowns without manual event handling, and run in web workers for blazing fast performance.

The Problem

Consider a Country + State dropdown. Rendering country options is easy, but what about states? You can't render all states from all countries—the list should change based on the selected country.

Traditional solutions require event handlers that run on the main thread, which becomes a bottleneck with large datasets (100k+ rows).

Note how each country should have a different set of states

Loading importer...

import { IvtSchema } from "@ivandt/importer";

import { countries } from "./countries";
import { defaultSchemaProps } from "@/app/schemas/defaultSchemaProps";

export const countryStateWithDataSourcesScheme: IvtSchema = {
  ...defaultSchemaProps,
  key: "countryStateWithDataSourcesScheme",
  dataSources: {
    countries,
  },
  fields: [
    {
      type: "dropdown",
      label: "Country",
      key: "country",
      order: 0,
      options: {
        query: `
        .countries
          | map({label:.name, value:.code3})
      `,
      },
    },
    {
      type: "dropdown",
      label: "State",
      key: "state",
      order: 1,
      options: {
        query: `
        .countries
          | filter(.name == "{country}")
          | pick(.states)
          | map(.states)
          | flatten()
          | map({label:.name, value:.code})
      `,
      },
    },
  ],
  initialData: [
    {
      country: "Australia",
      state: "",
    },
    {
      country: "United States",
      state: "",
    },
    {
      country: "Canada",
      state: "",
    },
  ],
};

import { IvtSchema } from "@ivandt/importer";import { countries } from "./countries";import { defaultSchemaProps } from "@/app/schemas/defaultSchemaProps";export const countryStateWithDataSourcesScheme: IvtSchema = {  ...defaultSchemaProps,  key: "countryStateWithDataSourcesScheme",  dataSources: {    countries,  },  fields: [    {      type: "dropdown",      label: "Country",      key: "country",      order: 0,      options: {        query: `        .countries          | map({label:.name, value:.code3})      `,      },    },    {      type: "dropdown",      label: "State",      key: "state",      order: 1,      options: {        query: `        .countries          | filter(.name == "{country}")          | pick(.states)          | map(.states)          | flatten()          | map({label:.name, value:.code})      `,      },    },  ],  initialData: [    {      country: "Australia",      state: "",    },    {      country: "United States",      state: "",    },    {      country: "Canada",      state: "",    },  ],};

The Solution

DataSources solve this elegantly using JSON query language. Queries run in web workers, keeping performance fast even with millions of cells.

Basic Usage

Define your data (or fetch it from your server), however complex

const countries = [
  {
    code2: 'AU',
    code3: 'AUS',
    name: 'Australia',
    states: [
      { code: 'NSW', name: 'New South Wales' },
      { code: 'VIC', name: 'Victoria' }
    ]
  },
  {
    code2: 'US',
    code3: 'USA',
    name: 'United States',
    states: [
      { code: 'CA', name: 'California' },
      { code: 'NY', name: 'New York' }
    ]
  }
];

Add to Schema

const schema: IvtSchema = {
  title: 'Address Import',
  publicKey: 'pk_live_xxx',
  sessionToken: 'session_xxx',

  dataSources: {
    countries  // Add your data here
  },

  fields: [
    // Fields defined next
  ]
};

Query in Fields

fields: [
  {
    type: 'dropdown',
    label: 'Country',
    key: 'country',
    order: 0,
    options: {
      query: `
        .countries
          | map({label:.name, value:.code3})
      `
    }
  },
  {
    type: 'dropdown',
    label: 'State',
    key: 'state',
    order: 1,
    options: {
      query: `
        .countries
          | filter(.name == "{country}")
          | pick(.states)
          | map(.states)
          | flatten()
          | map({label:.name, value:.code})
      `
    }
  }
]

How It Works

Query Syntax

DataSource queries use JSON Query Language. Here are the key operations:

Accessing Data

.countries  // Access the 'countries' key from dataSources

Mapping

Transform each item in an array:

.countries | map({label:.name, value:.code3})

Result:

[
  { "label": "Australia", "value": "AUS" },
  { "label": "United States", "value": "USA" }
]

Filtering

Filter items based on conditions:

.countries | filter(.name == "Australia")

Template Variables

Use {fieldKey} to reference other field values in the same row:

.countries | filter(.name == "{country}")

The {country} placeholder is replaced with the actual value from the country field.

Picking

Extract a specific property:

.countries | filter(.name == "Australia") | pick(.states)

Flattening

Flatten nested arrays:

.countries | map(.states) | flatten()

Benefits

DataSources provide significant advantages over event handlers:

Performance - Queries run in web workers, not the main thread
Simplicity - No manual event handling code
Reusability - Same data source can be used across multiple fields
Consistency - Declarative JSON queries are easier to maintain
Scalability - Handles millions of cells without performance degradation

Advanced Patterns

Three-Level Cascade

Country → State → City:

Loading importer...

import { IvtSchema } from "@ivandt/importer";
import { defaultSchemaProps } from "@/app/schemas/defaultSchemaProps";

export const threeLevelCascadeDataSourcesSchema: IvtSchema = {
  ...defaultSchemaProps,
  key: "threeLevelCascadeDataSourcesSchema",
  dataSources: {
    locations: [
      {
        country: "Australia",
        states: [
          {
            state: "New South Wales",
            cities: ["Sydney", "Newcastle", "Wollongong"],
          },
          {
            state: "Victoria",
            cities: ["Melbourne", "Geelong", "Ballarat"],
          },
        ],
      },
    ],
  },

  fields: [
    {
      type: "dropdown",
      label: "Country",
      key: "country",
      order: 0,
      options: {
        query: ".locations | map({label:.country, value:.country})",
      },
    },
    {
      type: "dropdown",
      label: "State",
      key: "state",
      order: 1,
      options: {
        query: `
          .locations
            | filter(.country == "{country}")
            | map(.states)
            | flatten()
            | map({label:.state, value:.state})
        `,
      },
    },
    {
      type: "dropdown",
      label: "City",
      key: "city",
      order: 2,
      options: {
        query: `
          .locations
            | filter(.country == "{country}")
            | map(.states)
            | flatten()
            | filter(.state == "{state}")
            | map(.cities)
            | flatten()
            | map({label:get(), value:get()})
        `,
      },
    },
  ],
  initialData: [
    {
      country: "Australia",
      state: "New South Wales",
      city: "Melbourne",
    },
    {
      country: "Australia",
      state: "New South Wales",
      city: "Sydney",
    },
    {
      country: "Australia",
      state: "",
      city: "",
    },
  ],
};

import { IvtSchema } from "@ivandt/importer";import { defaultSchemaProps } from "@/app/schemas/defaultSchemaProps";export const threeLevelCascadeDataSourcesSchema: IvtSchema = {  ...defaultSchemaProps,  key: "threeLevelCascadeDataSourcesSchema",  dataSources: {    locations: [      {        country: "Australia",        states: [          {            state: "New South Wales",            cities: ["Sydney", "Newcastle", "Wollongong"],          },          {            state: "Victoria",            cities: ["Melbourne", "Geelong", "Ballarat"],          },        ],      },    ],  },  fields: [    {      type: "dropdown",      label: "Country",      key: "country",      order: 0,      options: {        query: ".locations | map({label:.country, value:.country})",      },    },    {      type: "dropdown",      label: "State",      key: "state",      order: 1,      options: {        query: `          .locations            | filter(.country == "{country}")            | map(.states)            | flatten()            | map({label:.state, value:.state})        `,      },    },    {      type: "dropdown",      label: "City",      key: "city",      order: 2,      options: {        query: `          .locations            | filter(.country == "{country}")            | map(.states)            | flatten()            | filter(.state == "{state}")            | map(.cities)            | flatten()            | map({label:get(), value:get()})        `,      },    },  ],  initialData: [    {      country: "Australia",      state: "New South Wales",      city: "Melbourne",    },    {      country: "Australia",      state: "New South Wales",      city: "Sydney",    },    {      country: "Australia",      state: "",      city: "",    },  ],};

Conditional Options

Show different options based on multiple field values:

{
  type: 'dropdown',
  label: 'Car Model',
  key: 'model',
  order: 2,
  options: {
    query: `
      .cars
        | filter(.brand == "{brand}")
        | filter(.year >= {minYear})
        | map({label:.model, value:.id})
    `
  }
}

Computed Labels

Create custom labels from multiple properties:

{
  type: 'dropdown',
  label: 'Product',
  key: 'product',
  order: 0,
  options: {
    query: `
      .products
        | map({
            label: (.name + " - $" + (.price | tostring)),
            value: .id
          })
    `
  }
}

Query Language Reference

For complete documentation on the JSON query language, see jsonquerylang.org/docs.

Common operations:

.key - Access property
| map(expr) - Transform each item
| filter(condition) - Filter items
| flatten() - Flatten nested arrays
| pick(.property) - Extract property
| sort_by(.property) - Sort by property
| unique() - Remove duplicates
| first() / | last() - Get first/last item
| length() - Get array length

Performance Notes

DataSource queries run in web workers
Queries are cached and only re-run when dependent field values change
No performance impact even with 500k+ rows
Much faster than event handlers on the main thread

Comparison: DataSources vs Event Handlers

eventHandlers: {
  onBeforeRowsChange: async (changedRows) => {
    for (const tr of changedRows) {
      if (tr.country.currentValue === 'Australia') {
        tr.state.newOptions = countries
          .find(c => c.name === 'Australia')
          .states.map(state => ({
            value: state.code,
            label: state.name
          }));
      } else if (tr.country.currentValue === 'United States') {
        tr.state.newOptions = countries
          .find(c => c.name === 'United States')
          .states.map(state => ({
            value: state.code,
            label: state.name
          }));
      }
    }
    return { changedRows };
  }
}

Event handles issues:

Runs on main thread (performance bottleneck)
Manual code for each country
Not reusable
Hard to maintain

DataSources benefits:

Runs in web workers (fast)
Declarative, clean
Reusable across schemas
Easy to maintain

Note that whether you use Event Handlers or DataSources is completely up to you, both solutions perfectly work. In some cases, where you need to fetch certain items from your remote server, using an Event Handler is unavoidable. But you could also use remote options for dropdowns in such cases, which offers advanced caching and validation out of the box.

DataSources

On this page