In the meeting, the State Changers discussed various tools and vendors for web scraping and how to incorporate them into no-code/nonprogrammatic operations. The main concern of one of the participants was getting started with these tools without getting lost in the complex coding tutorials.
Three potential solutions were discussed: ScrapingBee, Axiom.ai, and BrowserBear: 1. ScrapingBee is a known entity in the scraping ecosystem, appreciated for its determinism and reliability. It's been widely used by State Changers members in the past and recommended for the same reason. However, it involves a fair amount of coding and scripting. 2. Axiom.ai is a newer tool that uses AI technology to automate the scraping process. This tool comes with a browser extension that enables the user to record their actions on a site, converting those actions into a set of instructions for the tool. 3. BrowserBear was mentioned as a less familiar, indie alternative that is also no-code and potentially worth exploring, though no one in the meeting had extensive personal experience using it. The overall recommendation was to embrace a combination of these tools, where each does the job they are best at. In the case of Xano, which one of the participants asked about, it should not necessarily be used for the scraping itself, but rather for triggering requests and saving data. The choice between these tools ultimately depends on the user’s specific needs and situation. While they touched on the use of AI technology in these tools, the full potential of AI for structured and unstructured data handling wasn't explored in depth during the meeting.
(Source: Office Hours 7/26/23 )
