The State Changers meeting focused on the optimization and future development strategy of a data caching system. They reviewed a task which downloads about three hundred thousand records a day, with a topic on how time could be reduced, speeding up the process by refactoring how the functions work—from looping through records to identifying and implementing more efficient alternatives.
The meeting referred to the sports of football and baseball as they discussed the functionality and performance issues of their data processing system. They expressed concerns about a loop that didn't run as expected, and discussed why the system was so slow. At the core of the conversation was how to optimize the automated tasks carried out by the system, reduce the duration it takes to download records (which can take up to 30 minutes per sport), and how to implement data caching to improve system speed. One of the attendees flagged an issue with the caching system, suggesting that it was running too fast, likely due to the cache intercepting the process, which prevented the task from executing as it should. To address this, it was proposed to disable the cache, shift focus to make the system work more efficiently before improving speed, and potentially create a single function to replace the current three. Future steps will include testing and debug iterations to ensure the optimisation is working as expected. Ultimately, the summarised goal is to pinpoint the slowest parts of the system and make them faster, starting with ensuring that the system is running as intended in the first place. Interestingly, they indicated using YouTube tutorials to assist in learning and implementing data caching. No specific technologies or platforms like Xano, WeWeb, Zapier, etc., were mentioned in the discussion.
(Source: Office Hours 9/5 )
