Hello Klipfolio Gurus,
How you get the data from a API which returns data in multiple pages and depending upon the page_num parameter in the API query.
I do not want to create like 15 different datasource for my 15 pages and use them in the Klip as there is limit on the number of datasources per formula. Is there a way around this?
How would I manipulate data from these different pages in the Klip?
Thank you for your excellent support as always! :)
You can use our dynamic data sources with variables and the MAP function to get a set of pages from an api that requires paging. This approach requires the api to return information on the total number of pages or entries returned by the query. This approach can be used if you have a limited number of pages to return. We typically recommend less than 10 pages.
Here are links to dynamic data sources, variables and MAP documentation.
https://support.klipfolio.com/hc/en-us/articles/216183237-BETA-Working-with-dynamic-data-sources
https://support.klipfolio.com/hc/en-us/articles/215547108-Variables
https://support.klipfolio.com/hc/en-us/articles/216182487-MAP-function
Here is a specific example based on the service desk.com
Create 2 datasources:
1. a static datasource to return the first batch of data. This is your existing query with ?per_page =100&status=pending,open. From this data source, the 'total_entries' field tells you how many records are returned by your query, although the API can only return 100 at a time.
2. a dynamic data source query that assigns a "page" variable to the page parameter. It will look mostly like your static data source
?per_page=100&status=pending,open&page={props.page}
You will need to create the "page" variable and assign it a valid value (e.g., 2) before you create the dynamic data source.
Then your formula would be something like (where total_entries is a field retrieved from your static data source):
MAP ( NUMBERFORMAT ( CUMULATIVE( REPEAT( 1, CEILING( total_entries/100 ) ) ) ),
page,
<reference fields in the dynamic datasource> )
The MAP function creates a dynamic data source instance for each page value. For example, if total_entries = 629 records, CUMULATIVE( REPEAT( 1, CEILING ( total_entries /100 ) ) ) would return 1, 2, 3, 4, 5, 6, 7, so 7 dynamic datasource instances would be created.
Thanks Keely,
This is so cool, I will try this solution. U rock!
Thanks Keely so much - this is really very well explained!
I've followed all your steps and almost there in solving this - but not able to understand the entire thing within a klip and its execution basically - so some follow up questions
1. How many data sources are you finally loading in the klip? Static or dynamic or both of them?
2. Where is the final query with "MAP" exactly added?
Thank you so much Keely.
Whatever the questions posted by Contractor Publishing I also would like to know them
Could you please help us to understand them completely.
I would also like to know the answer to "Contractor Publishing"'s post - particularly "Where is the final query with "MAP" exactly added?"
Could anyone from the support team please assist with this? I work for Terem Technologies and am building a dashboard
Thank you,
David Georgy
Hi David,
Please see Data Sources With Multiple Pages for details. If you have further questions, please email support@klipfolio.com and let us know what kind of datasource you are using.
Thank you,
Janice