During the meeting, several important insights were discussed. Here are the key points:
1. The issue with the background task in Zano was highlighted. It was mentioned that sometimes the task runs out of credits per minute, and there was an uncertainty about how to approach this problem. 2. It was suggested to create an event logging system to track the progress of the task and identify any errors or delays. This can be implemented using a table with columns for source, message, and data. Logging the information as it progresses will help debug any issues or delays. 3. To mitigate memory usage, it was recommended to load records in a streaming fashion rather than loading all 20,000 records into memory at once. This reduces the memory footprint and can improve performance. 4. It was also suggested to consider launching an endpoint for the for-each loop instead of a function. This would separate the memory space and potentially improve performance. 5. Another recommendation was to reduce the number of writes by checking if any changes need to be made before performing updates. This can optimize the power profile and speed up the task. 6. To avoid overwhelming the system, it was advised to include resting periods during the task. After processing a certain number of records, taking a short break can allow the underlying database system to clean up and be ready for subsequent operations. 7. The idea of offloading the heavy workload to a separate service called Langa was mentioned. This service could handle the orchestration and reduce the memory usage in Zano. Overall, the meeting provided valuable insights on improving the background task and addressing performance issues in Zano.
(Source: Office Hours 4/26 )
