Pipedrive pagination is tricky because we do not know how many pages or how many total elements are there before writing the API query. That makes the Klip maintainability a bit of a trouble.
The maximum number of items in the results are limited to 500 in the pagination API. You can use normal queries to get the data where data elements are less than 500. If the data items are more than 500 you can use dynamic data source.
In the dynamic datasource there will be a dynamic property which is defined using a variable, each value change in the dynamic property will create new instance of the datasource using that value.
You can also use filters to filter your data so that you do not always have to use dynamic datasource. For example, deals have 89 types of filters.
You can get filters for different types using queries mentioned here.
https://developers.pipedrive.com/v1
filter_id = 73 is for deals created last month.
Where non dynamic datasource query will be like, If the data has less than 500 deals you can use this,
https://api.pipedrive.com/v1/deals?api_token=YOUR-API-TOKEN&limit=500&filter_id=73
Pipedrive has a parameter called start which is actually an offset of the data. If there are more than 500 items then, dynamic datasource API query is,
https://api.pipedrive.com/v1/deals?api_token=YOUR-API-TOKEN&start={props.offset}&limit=500&filter_id=73
Please ensure that you create a variable called offset and assign a default value to the same in Klip editor and save it before you try to use it in the query.
{props.offset} is a variable and will be assigned values in the formula. Filter id is 73 to pull last months deals.
Consider for example you know there are 3 pages for this data with 1051 elements, so you will create a list of values for the start parameters as 0, 500, 100 with following formula. If there are 4 pages change 2 --> 3 in below formula.
CUMULATIVE(ARRAY(0, REPEAT(500, 2) ) ).
You can use MAP function which acts as a loop and assigns these values to the start parameter in the query and create 3 instances of the dynamic data source with different start value and pull the Id from all the datasource instances.
MAP( CUMULATIVE(ARRAY(0, REPEAT(500, 2) ) ), "offset" , JOIN (id ))
You can use JOIN to join the Ids from one instances in a single value separated by comma, as MAP will return one value for each input value of the variable, Then you can join all the three Ids from three instances of the datasource and convert them in to array.
ARRAY (JOIN( MAP( CUMULATIVE(ARRAY(0, REPEAT(500, 2) ) ),"offset" , JOIN (id ) ) ) )
New Information : Now instead of going all this trouble of joining all the results together and then making an array you can use MAPFLAT function, thanks to Klipfolio. So formula is much simpler now.
MAPFLAT( CUMULATIVE(ARRAY(0, REPEAT(500, 2) ) ),"offset" , (id ) ) )
All these formulas and features explained here.
https://support.klipfolio.com/hc/en-us/articles/216183237-BETA-Working?r=uv&i=258886
https://support.klipfolio.com/hc/en-us/articles/216182487-MAP-function?r=uv&i=193827
https://support.klipfolio.com/hc/en-us/articles/215548918-Troubleshooting-dynamic-data-sources
Please create a new dynamic data source from the scratch. You should not modify non dynamic data source to convert it to dynamic as it might make it unusable.
