In this meeting, the State Changers discussed Xano and OpenAI, primarily focusing on an issue where responses weren't coming back from OpenAI endpoints when using Xano. They found out that the issue was due to timeouts set by Xano, which default to ten seconds for its API requests. They concluded that this was due to the computationally intensive process of predicting what the next word, or token, will be using language models, and hence the operation is slow. The participant realized that they needed to increase the timeout to approximately sixty seconds for nontrivial number of request inputs. They also felt that given the nature of the technology, making live calls to OpenAI might result in slower user experiences.
(Source: Office Hours 9/26/2023 - Plus an Extra! )
