Map multiple pages with Harvest


I created a datasource for a harvest app with v2


I want to combine paged data for this one URL. I am not understanding where to put the map or what it would look like to create a single result set for multiple pages of results.

I had created a number of other paged urls that call directly.


I don't mind using this as an option, but I still need to join these datasets.


Does anyone have an example I could consider, or a walk through how to make it work? I feel I am getting just barely enough to get me close using MAP/MAPFLAT, but there isn't really information on how to get the result set pulled together.

1 comment

  • 0
    Rahul Bura

    Hey Chris, thanks for reaching out! :) This is a similar community post that I'd like to refer you to that shows how you should set up your datasource with the variable as a page number. The comments provide further context as well:


    Once you have that dynamic datasource set up, you will be using the MAPFLAT in your Klip formula to iterate through the page numbers. 

    The MAPFLAT function has the following definition: MAPFLAT( values, variable name, expression)

    In your case here are what should be used:

    - Values: You will need to calculate the number of pages worth of data you have to iterate through here.Harvest only allows a maximum of 100 records per page so if you have 500 records you need to bring back in total, you'll have to do some arithmetic: 500/100 records per page = 5 Total pages to iterate through.  

    - Variable Name: This is the name of the variable you will be passing in the expression - in this case it would be the variable that represents the page number. You need to create this variable by following Advanced Tools > Create New Variable in the editor view and assign it a valid value like 1. Once you have done so you can create your dynamic datasource.

    - Expression: Here, you're going to reference the data in your dynamic datasource. So now when the MAPFLAT evaluates, you're going to get this referenced data across all the pages.

    Please give that a shot and if you need further assistance, I would recommend writing into us directly via  ticket so we can see what you have so far and provide further steps and clarifications :)



Please sign in to leave a comment.