Skip to main content

How does data source refresh work in Klips?

Updated

You can set a refresh queuing interval on a data source to automate data updates in your Klips.

Data sources refresh only if there is at least one active user in the account. Otherwise, the data sources are queued for refresh the next time a user logs in, or when a published link is being viewed. If the next scheduled refresh time is due or past due, the data source is sent to a global refresh queue.

The time it takes for a data source to refresh depends on:

  • The number of data sources already in the queue
  • The number of data sources in your account being refreshed simultaneously
  • The time it takes for the source system to respond to the query defined in the Klipfolio data source
  • Whether or not the refresh needs to be deferred in order to avoid reaching an API call limit, for an account, for a selected data source.

Notes:

  • The minimum refresh interval for any type of data source is one minute.
  • If you're interested in the Warm Up Refresh option for data sources (which enables you to set a daily time for all of your data sources to refresh), go here.

This article includes the following sections:

How do my data sources get added to the refresh queue?

A data source continues to be added to the refresh queue depending on the interval set in the data source definition, and as long as there is some activity on the dashboard. For example, a data source set to refresh every hour will be refreshed as long as one person is active in the account (viewing the dashboard, working in the Klip Editor, or viewing their list of data sources).

Data sources are refreshed on a first-in-first-out order of priority. This ensures that resources are made available to active users. The only exception is deferred refreshes. When a refresh is deferred past the scheduled refresh time frame, the deferred refresh request is automatically added to the front of the queue.

Note for Partner accounts: For a client account to be recognized as active, a person must be signed into the client account, not the partner account. If no-one is signed in, the next person to sign in, will send data sources that have a next refresh time of before that login, to the global refresh queue.

About data refresh and service API limits

Because some APIs, like Google Analytics, impose a limit on the number of requests per profile, number of concurrent requests per profile, and the number of tokens per user account, Klipfolio monitors the number of API calls for each OAuth token and profile. This is most commonly seen when you have a large number of queries set to refresh in close succession. In this case, Klipfolio can defer a new refresh request from executing for a short period of time in order to prevent an account from violating an enforced limit. This typically has little impact on queue refresh times.

API limits can affect the refresh interval for services. For example, due to the API limits set by Google Analytics and Adobe Analytics, the shortest refresh interval that can be selected for these services in Klipfolio is every 15 minutes.

Note: As an Admin, when you set a data source refresh rate, make sure the refresh rate complies with any limitations set by the service you're connecting to.

Data refresh for embedded Klips

Embedded Klips provide a simple way to expose your Klips on an external web site. Each time you view an embedded Klip on a web page, Klipfolio checks to see if the Klip has been rendered within the last three hours. If the Klip has been rendered in three hours or less, the cached Klip data is presented to the viewer. If the Klip has not been rendered, the Klip will now render using the current cached data from your dashboard. You must log in to your dashboard in order for the data source cache to refresh.

Data refresh for scheduled email snapshots

When you schedule an email snapshot, Klipfolio automatically refreshes all data sources for the Klips contained in the email just before the scheduled email is due to be sent out.

Using Klipfolio's API to refresh your data

If you want to refresh your data in real-time, you can use the Klipfolio API to push data to the dashboard.

For more information on using the Klipfolio API see Klipfolio API Reference Guide.

Why should I use published links to share my dashboards?

To ensure users are seeing current data, consider using published links to share a dashboard. These links are automatically refreshed as the data is refreshed and can be protected by a password if required.

For more information on published links see How do I share my dashboards using published links?

Where can I find more information on troubleshooting refresh issues?

For more information on handling data source refresh failures see Understanding data source refresh intervals, failures, and errors.

Related articles