About data source refresh

Klipfolio allows you to set a refresh queuing interval on a data source to automate the process of updating the data in a Klip. This ensures the data in your Klip is up-to-date. 

Data sources refresh only if there is at least one active user in the account. Otherwise, the data sources are queued for refresh the next time a user logs in, or when a published link is being viewed. If the next scheduled refresh time is due or past due, the data source is sent to a global refresh queue. 

The time it takes for a data source to refresh depends on:

  • the number of data sources already in the queue,
  • the time it takes for the source system to respond to the query defined in the Klipfolio data source, and
  • whether or not the refresh needs to be deferred in order to avoid reaching an API call limit, for an account, for a selected data source.

The Refresh Queue

A data source then continues to be added to the refresh queue depending on the interval set in the data source definition, and as long as there is some activity on the dashboard. For example, a data source set to refresh every hour will be refreshed as long one person is active in the account (viewing the dashboard, working in the Klip Editor, or viewing data sources in the Library). 

Data sources are refreshed on a first-in-first-out order of priority. This ensures that resources are made available to active users. The only exception is deferred refreshes. When a refresh is deferred, passed the scheduled refresh time frame, the deferred refresh request is automatically added to the front of the queue.

Note for Partner accounts: For a client account to be recognized as active, a person must be signed into the client account, not the partner account. If no-one is signed in, the next person to sign in, will send data sources that have a next refresh time of before that login, to the global refresh queue.

The Deferred Queue

Because some APIs, like Google Analytics, impose a limit on the number of requests per profile, number of concurrent requests per profile, and the number of tokens per Google user account, Klipfolio monitors the number of API calls for each OAuth token and Google Analytics profile. This is most commonly seen when you have a large number of Google Analytics queries set to refresh in close succession. In this case, Klipfolio can defer a new refresh request from executing for a short period of time in order to prevent an account from violating an enforced limit. This typically has little impact on queue refresh times.

Embedded Klips

Embedded Klips provide a simple way to expose your Klips on an external web site. Each time you view an embedded Klip on a web page, Klipfolio checks to see if the Klip has been rendered within the last three hours. If the Klip has been rendered in three hours or less, the cached Klip data is presented to the viewer. If the Klip has not been rendered, the Klip will now render using the current cached data from your Dashboard. You must log in to your Dashboard in order for the data source cache to refresh.

Scheduled email snapshots

When you schedule an email snapshot, Klipfolio automatically refreshes all data sources for the Klips contained in the email just before the scheduled email is due to be sent out.

Related links: