During the meeting, the State Changers discussed the technical aspect of running job queues, with emphasis on optimizing job scheduling and execution using a platform called Xano. They explored different strategies on managing the strict one-minute granular of the scheduling feature in Xano, which they deemed to be too infrequent for their requirements.
A proposed solution involved setting up an Application Programming Interface (API) with some sort of loop for triggering these APIs more frequently. Another viable solution was to use background tasks with a different approach instead of triggering one job and going quiet, multiple jobs would be scheduled and all executed within a particular duration. The State Changers discussed how the system could execute a pending jobs list until all tasks were completed without consuming too much system resources. If a job takes an hour, the ‘every minute’ indication doesn’t necessarily space them apart but refers to the inner clock of the machine which activates the scheduled run every time it hits zero. A key recommendation was to use custom functions alongside background tasks to execute all pending jobs from the jobs table. The tasks would be carried out until completion noting the completion time for each task to optimize scheduling. The group also discussed the importance of setting limits for how much data to return from the database through pagination. In conclusion, the meeting spotlighted how to optimize the use of Xano for job queues through efficient scheduling, custom functions, and prudent use of system resources. The participants also stressed the importance of control in running background tasks to prevent resource runaway and ensuring the system runs at peak efficiency.
(Source: Office Hours 10/26/2023 )
