In this meeting, there were discussions around two main topics.
1. Improving the efficiency of operations design when working with large amounts of data: The participant shared frustrations about facing memory issues when slicing and dicing data. They explained a specific example related to retrieving data about cruise ships from an API. They have divided the data into smaller sections but are still struggling with memory issues. Possible solutions discussed included using a separate process or Lambda function to handle the initial API request and data segmentation, and optimizing the code for better memory management. 2. Manipulating JSON data and maintaining order: The participant had concerns about the order of the JSON objects they were creating. They wanted to ensure that specific keys were in a certain order. It was discussed that JSON serialization and encoding processes automatically determine the order of keys, and trying to manipulate it could lead to compatibility issues. It was advised to focus on machine readability rather than human readability for JSON data.
(Source: Office Hours 5/31 AM )
