2

Finer control on refresh of data sources

We should be able to set changing refresh frequency of data sources during weekdays vs weekends and working hours vs non-working hours.

Right now we have to set the frequency to the fastest rate and it applies for the whole week.

This not only loads Klipfolio servers (data transfer) with fetching information but also taxes the app (other) server that is serving this information to the Klip server.

 

Finer control on refresh schedules across each data source could help reduce the data fetching by around 33% in our case.

With clients across the globe, the peak working hours would help spread the workload further. I will leave the math to you guys.

3 comments

  • 0
    Avatar
    Scott Lawrence

    Hi Darshan,

    Thank you for raising this and contributing to the community.

    I would love to understand your situation more.  

    • Can you describe the kinds of data sources you are using and what would be driving the variations on refresh frequency?  
    • Would there be exceptions to the variations?
    • Would you see executing the variations by data source or for your account as a whole? (My suspicion is by data source.)
    • Do all your data sources need a specific interval(s) during your working day vs weekends/non-business hours?  Or would some be more specific times vs. an interval?

    Thank you again for raising this.  I'll raise this as an enhancement request and associate with you for context.  (I'll add any additional responses as more context.)

    Cheers,

    Scott.

     

     

  • 0
    Avatar
    Admin Old Darshan

    There are 2 different clients, we have - one of them has a MySQL database, the other has Excel spreadsheets.

    Both need refreshes at like 15-30 minutes on weekdays between 8 am - 6pm IST. Rest of the times, it could suffice to have once in 2 - 4 hours.

    For the MySQL database, which collects response to a survey every fortnight. Activity is at its peak in the first 2 days after sending the invitation and reminder (week later).

    In those first 2 days (each), refresh rate needs to be 15 mins, next 2 days 30 mins and last 3 days 1 hour. Sometimes if a new survey is not initiated in 2 weeks, there is 0 activity and we should be able to turn off refresh. Also we should be able to turn off or slow down (once in 2 hours on weekdays and once in 4 hours on weekends) refresh between 11pm - 7am.

    For the other client, using spreadsheets from Google Drive, it would be better if refresh takes place on update of the source file rather than a set frequency.

    We would like the variations by data source, not by account as a whole.

    Yes, there will be exceptions to variations but less than 5%.

  • 0
    Avatar
    Scott Lawrence

    Hi Darshan,

    Thank you for the additional context.

    Cheers,

    Scott.

Please sign in to leave a comment.