Skip to main content

Creating Google BigQuery data sources and custom Klips

Updated

You can set up a connection between Klipfolio and your Google BigQuery data to display your Google Storage data on a dashboard.

This article includes:

Connecting Klipfolio to your Google BigQuery account

To retrieve your data and visualize it as custom metrics, you first need to connect PowerMetrics to your Google BigQuery account.

Here are a couple things to note:

Your data is safe with us. The first time you connect your Google BigQuery data to Klipfolio, you'll be prompted to enter your Google login credentials and give permission to allow access to the data within your Google BigQuery account. We are serious about protecting your data privacy.

Managing your connections. By default, your connection name looks like this: yourname@Google<date and time created>. You can modify the default connection name when you create a new connection or from your list of connections, accessed by clicking your Account Name (located at the bottom of the left navigation sidebar) and selecting Account > Connected Accounts. Each time you connect to Google BigQuery, you can use the same account (or switch to a different account if needed). If you have trouble accessing an existing connection, your OAuth token may have expired. If that happens to you, go here for help.

Adding a Google BigQuery data source

Here are a few things to note before creating your Google BigQuery data source:

If you choose to create a custom Google BigQuery data source, you first need to create an SQL query using the Google BigQuery Query editor. You'll copy and paste that query into Klipfolio when choosing the data to retrieve for your data source.

To help you get to your data faster, when creating your Google BigQuery data source, the data preview will only display up to 2MB of data by default. After creating the data source, if you navigate to the About this data source page, the preview there will include all of the Google BigQuery data you requested.

To add a Google BigQuery data source:

  1. In the left navigation sidebar, click the 3-dot menu beside Data Sources and select +New data source.
  2. On the Where is your data? page, select Google BigQuery.
  3. On the Choose a pre-built or custom connection page, select either a pre-built or custom connection to use for your data source.
  4. If this is your first time connecting to Google BigQuery:
    • Click Connect an account. Enter your Google login credentials and click Next. If prompted, click Allow Access to enable Klipfolio to securely access your Google BigQuery data. Click Continue.
  5. If you've connected to Google BigQuery before:
    • Select an existing connection (token) from the drop-down list and click Continue.
  6. On the Configure your data source page:
    • If you selected the List of Projects pre-built option:
      • Click Get data.
    • If you selected the Sample Query - Life Expectancy by Country pre-built option:
      • Select a Project from the drop-down menu (or, optionally, manually enter the Project name/ID in the text field).
      • At Request Option, select either BigQuery SQL or Raw data (JSON), depending on the desired output format.
      • Click Get data.
    • If you selected Create a custom Google BigQuery data source:
      • Select a Project from the drop-down menu (or, optionally, manually enter the Project name/ID in the text field).
      • At Request Option, select BigQuery SQL, paste the SQL query you copied from the Google BigQuery Query Editor into the SQL Query text box.
        Note: Be sure to include #standardSQL as the first line in your query, otherwise you may receive a date format error.
      • Click Get data.

For tips on working with the Google BigQuery API, see "More about Google BigQuery's API and queries".

 The image below displays a Google BigQuery SQL query and its results in Klipfolio:

  1. Ensure this is the data you’re looking for. Optionally select the checkbox to Model your data. Learn about modelling data sources.
  2. Click Continue.
  3. Name, choose refresh settings, and, optionally, share your data source.
  4. Click Save.
    Note: To ensure your data source continues to update and refresh correctly, after saving, don't change its name or ID.

More about Google BigQuery's API and queries

Klipfolio connects to hundreds of services in the cloud that have a REST API. If you take the time to learn your service’s API to create queries for the data you want, your possibilities are endless. Here’s a quick overview video on APIs and Klipfolio:

  Klipfolio API's 101

If you need assistance using APIs, enlist a data analyst or developer.

See the reference table below for API related information and sample queries.

API Documentation Google BigQuery API documentation

Authentication Method

 OAuth Token Authentication
Response Format CSV or JSON
Sample Queries

Life Expectancy by Country in Raw data (query in JSON format):

{
"query": "#standardSQL\r\nSELECT\r\n age.country_name,\r\n age.life_expectancy,\r\n size.country_area\r\nFROM (\r\n SELECT\r\n country_name,\r\n life_expectancy\r\n FROM\r\n `bigquery-public-data.census_bureau_international.mortality_life_expectancy`\r\n WHERE\r\n year = 2016) age\r\nINNER JOIN (\r\n SELECT\r\n country_name,\r\n country_area\r\n FROM\r\n `bigquery-public-data.census_bureau_international.country_names_area` where country_area > 25000) size\r\nON\r\n age.country_name = size.country_name\r\nORDER BY\r\n 2 DESC"
}

Creating Google BigQuery custom Klips

Now that you've created a Google BigQuery data source, you can return to it and use it to make custom Klips. A single data source can be used to create a single or multiple custom Klips.

To create a Google BigQuery custom Klip:

  1. In the left navigation sidebar, click Data Sources to open your Data Source Library.
  2. Select the Google BigQuery data source you want to use for your custom Klip.
  3. On its details page, click Create a Klip.
  4. In the Klip Editor, configure your custom Klip.
    If you need help, go to this article. (You can skip the “Choose the data source” step in the linked article. You’ve already chosen it.) You'll also find lots of articles on working with Klips in this section of our Knowledge Base.
  5. Click Save.

Your new Klip is ready to add to a dashboard. Learn about dashboards.

Related articles