In this meeting, the State Changers discussed resolving a memory management issue in their coding workflow. The problem resulted from the current system's approach of going through records one at a time, leading to a significant number of 'wrappers' or excessive data processing.
The team proposed solutions using the tool "Xano" to optimize memory usage, making changes to how data chunking is implemented. Instead of going through every line one by one (which creates too many wrappers and thus, memory problems), the group proposed experimenting with 'pages' - chunks of 1000 lines at once. The functionality of 'slice' in Xano is explored, dividing the original data set into smaller 'pages' by multiplying the index number by a thousand, thereby considerably reducing the memory volume being processed at once. This could potentially keep the operation under the memory ceiling, but the group acknowledged uncertainty as Xano's memory management is a bit obscure. The State Changers also implemented the 'return' function to address inconsistency of error codes, reducing the possibly memory-heavy operation of sending back error information. Finally, they proposed sending over 10,000 items at once or pulling an entire 'page' of data instead of one line at a time. However, these solutions would take time to implement and test. Overall, the meeting dealt with advanced concepts of data chunking, memory management, and error handling using the Xano platform, and offered potential solutions to the mentioned issue.
(Source: Office Hours 3/23 )
