In this meeting, the State Changers discuss how to handle long-running processes within their project concerning over a million data records. The key technical tools that they are using include Xano from which they are extracting queries, while Integromat and JavaScript power their operations. They discussed technical challenges experienced during a "burndown" process, where they are trying to delete a large set of query records without overloading the system memory.
They found the task of authentication to be complex, indicating that once it is solved, it is indeed solved. They have utilized a while loop coupled with a conditional break in JavaScript to automate the deletion of queried records until the data count reaches zero. Here, they're only loading 10,000 records at a time into memory to prevent overwhelming the system. The team seemed to be somewhat unfamiliar with using pagination and where it fits into this process. The senior member clarified that the pagination essentially acted to prevent loading all the 1.5 million records into memory, allowing the system to continuously delete batches of records and retrieve new ones without running out of memory. The team concluded the meeting by recommending posting their progress, challenges, and next steps on a community forum to ensure continuous support and progress updates until their next meeting. Improvements were reported in property and image posting through mobile, though a challenge was noted in pulling data from Xano to be addressed in the next stages.
(Source: Office Hours 9/13 )
