Exploring Synchronization Process: Transferring Data from Xano to Azure with Enhanced Transaction Logging
In the meeting, State Changers primarily discussed the creation of a system to transfer data from Xano into Azure for analytics purposes. They focused on producing a transaction log that tracks all the operations (add, update, remove) happening in tables. This transaction log could be used to rebuild the database in Azure.
They also suggested adding a timestamp field (receipt), indicating when Azure collects the data. This would prevent running the same transaction multiple times and would also help track synchronization progress, providing an audit log for future troubleshooting.
They also talked about the potential complexities of data synchronization, pointing out the possibility of errors that could cause data discrepancies. Similarly, they discussed maintaining a backup plan to rectify these errors - resetting and realigning the system when necessary.
The meeting concluded with a suggestion to keep the Azure log table slightly larger than the data yet to be transferred to Azure, to keep a buffer, allowing for a review of the transactions for at least a day or a week. This is expected to help with diagnosing issues, especially during the early stages of the process the State Changers are building.
Keywords mentioned in the meeting include: Xano, Azure.