BETA: Working with dynamic data sources

A dynamic data source is a data source which uses one or more variables in its query instead of hard-coded values, thereby enabling the ability to dynamically retrieve data. Dynamic data sources offer a solution for users who need more flexibility in working with large amounts of data or data that requires multiple instances.

Important notes

Please note the following when working with dynamic data sources:

  • Dynamic data sources are typically used for pagination, combining data sources, dynamic dashboards, and date queries.
  • Dynamic data sources should NOT be used to implement security. They are variable driven and variable values can be accessed through different features, for example, permalinks, where all the variables and values are included in the URL. If you want to implement security to prevent users from accessing your data, create a specific data source with the parameters and values set in the data source definition, without using variables.
  • Modelling dynamic data sources is not currently supported.

Setting up a dynamic data source

A common use for dynamic data sources is accessing a list from a data source, for example, Google Analytics profile IDs, Facebook pages, or Google Adwords MCC client IDs, and displaying the data for each item in the list. You can do this effectively with dynamic data sources instead of creating a separate data source for every profile or page ID in your list.

You can also use dynamic data sources with SQL databases. Any parameter in the WHERE clause can be used as the dynamic variable and also any value in the host, port numbers, usernames, and password fields.

This example describes how to create a dynamic data source that displays data for each Google Analytics ID:

  1. Create a new variable in the Klip Editor and enter a value, for example, a variable called gaID with the value set to 1234567.

    Note: The variable used in a dynamic data source must be set to a value that will return valid results when used in the data source query.

  2. Create a new data source and use the variable you created in the Query URL with the correct formatting. In this example, we use the Google Analytics connector and enter the variable in place of a hard-coded value, using the format {props.gaID}. For example:

https://www.googleapis.com/analytics/v3/data/ga?ids=ga:1234567&start-date=date.startOfYear&end-date={date.today}

Replace 1234567 with {props.gaID}.

https://www.googleapis.com/analytics/v3/data/ga?ids=ga:{props.gaID}&start-date=date.startOfYear&end-date={date.today}

  1. To create a list of values like profile IDs, build a drop-down User Input Control component to select the profile IDs from. Enter the list of values you want to select from in the Values sub-component.
  2. Set the variable in the User Input Control to the same variable you entered in the Query URL. In this example, we use gaID.
  3. Build a Klip for example, a Table component, with the data source that was returned when you created the query. Select the columns of data and reference the data as you would with any data source.

When the Google Analytics ID is selected from the User Input Control, a dynamic data source instance is created and the table or any component automatically responds to display the data for the Google Analytic ID selected.

Troubleshooting dynamic data sources 

When creating a dynamic data source, you have to specify query parameters directly in the Query URL. You cannot use the Query Parameter fields.

If you have a Klip that uses dynamic data sources which is not displaying the expected data, you can investigate the dynamic data sources using the About this Data Source and debug_info pages:

  1. Click Data Sources in the left navigation bar.
  2. From the list of data sources displayed, select the dynamic data source to open its About this Data Source page. A data source can be identified as dynamic if the Download Data option is not displayed.
  3. In the URL displayed in the browser address bar, replace view with debug_info.
  4. Scroll down to the Instances section. This is a list of the individual dynamic data source instances, each with its unique ID, variable (prop:<variable>) setting and refresh information.
  5. Check for dynamic data source instances with the following issues:
    • invalid or missing variable setting
    • empty last refresh value
    • small data size
    • refresh fail count with a non-zero value
  6. For each invalid dynamic data source instance, select one of the [ x | refresh | data ] actions displayed next to its identifier.
    • click data to view the dynamic data source instance and assess the contents
    • click x to delete the dynamic data source instance (this will delete only the specific instance, not the entire data source)
    • click refresh to refresh the dynamic data source instance

Dynamic data sources with multiple variables

Dynamic data sources can have multiple variables associated with them. If these variables are set to an invalid combination, an invalid data source instance will be created. To prevent invalid combinations of variables, the User Input Control components that set these variables should be associated with a Button component configured to co-ordinate the setting of these variables. This enables better Klip performance and avoids unnecessary formula execution.

For additional assistance debugging dynamic data sources, contact support@klipfolio.com.

Learn more

Now that you know more about dynamic data sources, check out the following:

Have more questions? Submit a request