Understanding data source refresh intervals, failures, and errors

This article contains answers to the following questions:

About data source refresh intervals

Klipfolio attempts multiple times to connect to a data source before disabling it. The re-check interval is designed to ensure that the maximum amount of time is given to retrieving new data and to enable errors to be cleared.

Refresh attempts use the following intervals:

  1. After one failure, Klipfolio attempts to refresh again every 5 minutes until success.
  2. After three failures, Klipfolio attempts to refresh again every 30 minutes until success.
  3. After five failures, Klipfolio attempts to refresh again every 6 hours until success.
  4. After 20 failures, the data source is disabled.

Modifying the queue for refresh settings

You can manually place a data source in the queue for refresh or change its refresh interval settings.

To manually queue a data source for refresh:

  1. Click Data Sources in the left navigation bar.
  2. Select the checkbox(es) for the data source(s) you want to manually refresh.
  3. Click Queue for Refresh.

Note: You can only manually queue raw, unmodelled data sources for refresh. For modelled data sources, select the data source from the list, click the link beside Uses Data Source (to access the unmodelled data source it is based on), then click Queue for Refresh on the About this Data Source page.

To change the refresh interval for a data source:

  1. Click Data Sources in the left navigation bar.
  2. Click the data source you want to modify.
  3. For a modelled data source, under Connection, at Uses Data Source, click the link to the raw, unmodelled data source. Go to step 4.
  4. Under Connection, at Last Refresh, click View Refresh Info.
  5. Under Refresh Information, click Edit and select an alternate refresh interval from the drop-down beside Queue for Refresh.
  6. Click Save.

Investigating data source failures

If there is a red icon beside Data Sources in the left navigation bar, that means one or more of your data sources are failing. To access your list of failed data sources, click the red icon.

To view error details or to reconfigure the data source connection, click the red exclamation mark beside the Data Source Name.

If it’s a modelled data source, under Connection, click the link beside Uses Data Source (as any changes need to be made in the raw, unmodelled data source).

In a raw, unmodelled data source, depending on the red warning text, at Refresh Status, click Reconfigure or Re-enable.

Finding data source error logs

If a data source is failing or disabled, you can consult the error log to help diagnose the problem.

To find the error log for a data source:

  1. Click the red icon beside Data Sources in the left navigation bar.
  2. Select the relevant data source.
  3. For modelled data sources, under Connection, at Uses Data Source, click the link to the raw, unmodelled data source. Go to step 4.
  4. Under Connection, at Last Refresh, click View Refresh Info to display the Error Log.

See the following section to learn more about how to use the Error Log.

Analyzing the error log

The following information about a data source is available in the error log:

  • The last Refresh Date/Time
  • The length of time (Duration) to refresh the data
  • The current size of the data source
  • The result of refreshing the data source. (Click the link for error information.)
  • The kind of data source
  • Whether the refresh was successful or not (Updated?)

Note: Data in the error log is purged weekly and is current for the last ten minutes.

Troubleshooting data sources that fail to refresh

  1. Some APIs, like Google Analytics, impose a limit on the number of requests per profile, number of concurrent requests per profile, and the number of tokens per Google user account. For more information on the Google Analytics API, see Limits and Quotas on API requests.
  2. Some OAuth tokens expire and must be re-authorized on a regular basis. For example, a Facebook OAuth token must be re-authorized every two months. Go here for more information.
  3. Some APIs impose a limit to the size of file you can upload. 
  4. Some data sources, such as SQL data sources inside your firewall, might require you to whitelist our Klipfolio dashboard IP addresses. For more information, see Allow list IP addresses.
  5. A query might be incorrect. For example, a Google Analytics query returns a "Connection Service" error if there is a line break at the end of a query.
  6. A file name might be incorrect. For example, if a file in Dropbox has been replaced with a file using a different file name.

Learn more

Have more questions? Submit a request