--- title: "How to build Voice-based AI Action Agents App to Execute Tasks, Automate Reports, and Analyze Data …and more." slug: how-to-build-voice-based-ai-action-agents-app-to-execute-tasks-automate-reports-and-analyze-data date_published: 2024-10-24T06:07:54.541Z original_url: https://www.tigzig.com/post/how-to-build-voice-based-ai-action-agents-app-to-execute-tasks-automate-reports-and-analyze-data source: migrated processed_at: 2025-12-02T10:00:00.000Z --- # How to build Voice-based AI Action Agents App to Execute Tasks, Automate Reports, and Analyze Data …and more. In this article I show how to build a voice-powered AI agentic app to automate business tasks and analyze live data warehouses. **NEW** My open-source platform with a ton of micro-apps and tooling's for AI driven analytics and data science This 5 part series has very detailed video guides for numerous reusable components that can be used across many apps This app has a non-realtime implementation of voice AI agents. For realtime voice AI agent app implemented with OpenAI's new WebRTC check out the posting below Chat is just one of the many things LLMs can do…They can act and execute. **VTEXER: Voice-Enabled LLM Action Agent App for Task Execution Automation Query and Research.** I demonstrate how you can, with natural language voice instructions: * Update Excel Sheet * Update Google Sheet * Update records in remote Data Warehouses * Query, Analyze and Transform data in remote Data Warehouses * Generate and email print formatted report in PDF format * Generate and email deck format * Carry out custom web search * Provide user menu option to choose amongst multiple LLM Agents And yes, it can also chat. Detailed, step-by-step, hands-on implementation guide, source code, GitHub repo, Schemas, Blueprints shared in sections below. The source deployed as-is on Vercel etc. will give you a fully functional voice-bot right off the bat (with restricted app features) **By leveraging platforms like Flowise AI and [Make.com](http://make.com/), you can interact with and execute tasks in voice and natural language text with:** your business data warehouses, thousands of platforms (Zoho, QuickBooks, Salesforce, AWS, Google, Microsoft, Azure, Slack HubSpot, Stripe, Woo Commerce, Shopify….) and most importantly your own existing API's The implementations are relatively rapid, with part of it being no-code, and any coding being done by AI-assisted coding tools. As part of the series, I show you how to use AI Assisted Coding Tools like Claude Dev and Cursor AI to develop LLM Apps with natural language commands. And deploy to open internet. **The app uses LLM ReAct Agents, Reasoning and Action Agents.** Called as such since they can execute tasks via API calls, also called Function Calls / Tool Calling. Fairly easy to setup in Flowise. Flowise has a marvelous functionality to access an agent / chat flow via an API endpoint. Allows you to fully customize the UI and response flow. Using that extensively here * Custom Frontend: React.js * Rapid Deploy Frontend: Flowise * LLM ReAct Agents: Flowise ([Flowiseai.com](http://flowiseai.com/)) * Workflow: [Make.com](http://make.com/) * Automation Scripts: Google Script * AWS MySQL DB Connection: Flowise ReAct Agent & Custom built FastAPI Server **ALL coding done by GPT, LLMs & AI Assisted Coding Tools.** For Google Scripts and Python I like GPT-4o. For React in general it is Claude Sonnet. For React.js apps like this one, my current favorites are Claude Dev, a VS Code extension and Cursor AI IDE, a VS Code fork. Claude Dev works on Github Codespaces also… I have a bit of preference for cloud over local. Both are able to create new files as well as modify files across whole codebase. Claude Dev even has terminal access, so does pretty much everything. ## Hands on Implementation Guide ### Video guide * 📌 Part 1: Demo and Agent Process Flow * 📌 Part 2 : How to update Excel, Google Sheets and Databases with AI / LLM Voice Agents * 📌 Part 3: How To Automate Excel to PDF, Excel to Slides, Automatic Email with AI Agents and Google Scripts * 📌 Part 4: How to build AI App with natural language with Claude Dev and Cursor AI * 📌 Part 5: How to create AI/ LLM Agents that query databases, do web search and take action Source code, JSON Schemas and blueprints on [GitHub](https://github.com/amararun/vtexer-shared): The repo, deployed ASIS to Vercel / Netlify etc will give you a fully functional voice-bot (with restricted app features) **Top Flowise AI Resource:** Leon Van Zyl's videos are a must-watch - from simple chatbots to complex agentic flows. [Leon van Zyl - YouTube](https://www.youtube.com/@leonvanzyl) ## AI Assisted Coding: Must Read ## Implementation Videos with Time-Stamped Sections Complete Playlist at: [YouTube](https://www.youtube.com/watch?v=Efn2hdLOPiU&list=PL4CpDSkww6gAuwuA-0CxJguDUnleDbn50) ### Part 1: Demo and Agent Process Flow * 00:02 - Introduction and LLM Agent Capabilities * 01:10 - LLM Agent Demo: Updating Excel and Google Sheets * 03:31 - Generating Report and Deck Formats * 05:12 - Data SQL Analyst and Other Functionalities * 08:06 - Overview of Architecture and Flow * 09:47 - Tracing the Flow of the Agent * 14:01 - Details on Data SQL Analyst and General Analyst * 17:50 - Build Process and Implementation Overview * 18:51 - Breakdown of the Four-Part Guide ### Part 2 : How to update Excel, Google Sheets and Databases with AI / LLM Voice Agents * 00:00 - Overview and Introduction * 03:55 - Setting up Flowise and Learning Resources * 05:10 - Top Tips for Working with GPT * 10:10 - Building the Chat Flow in Flowise * 11:30 - Custom Tool for Function Calls in Flowise * 17:15 - Connecting Flow Wise to [Make.com](http://make.com/) * 20:10 - Testing the Flow and Updating Google Sheets * 22:54 - Connecting and updating Google Sheets and Excel Sheets * 29:10 - Connecting Flow Wise to AWS RDS MySQL * 35:00 - Adding Voice Functionality to the Flowise Bot * 38:00 - Connecting Flow Wise to a React Frontend * 41:00 - Deploying the React Frontend Using Vercel * 43:00 - Conclusion and Next Steps ### Part 3: How To Automate Excel to PDF, Excel to Slides, Automatic Email with AI Agents and Google Scripts * 00:03 - Introduction and Recap * 00:36 - Creating Automation Scripts in Google Script * 07:38 - Deploying Google Script and connecting to [Make.com](http://make.com/) * 10:38 - Configuring Triggers and Filters in [Make.com](http://make.com/) * 14:26 - Demonstrating the Functioning Web App and Chatbot * 17:29 - Connecting React Frontend to Flowwise * 21:10 - Deploy and Go Live with Vercel ### Part 4: How to Build AI Voice Action Agent App with natural language with Claude Dev and Cursor AI * 00:04 - Introduction to Part Four and Custom Frontend * 00:36 - Deploying the Voicebot on a Website and Repository Setup * 01:36 - Overview of Voicebot Components: Voice to Text, Chat Completion, Text to Speech * 04:18 - Building the Demo App with Claude Dev and Cursor AI * 15:26 - Deploying the Voicebot App Live on Vercel * 16:38 - Connecting GitHub Repository to Vercel for Deployment * 17:12 - Final Testing of Live Voicebot App * 19:03 - Demonstrating Multilingual Support in Voicebot * 20:21 - Differentiating Between Transcription and Translation in API Usage * 21:22 - Overview of API Endpoints for Speech and Text Processing * 21:49 - Routing Text to LLM Agent for Response Generation * 23:58 - Finalizing API Integrations for Voicebot Functionality * 25:48 - Conclusion and Next Steps for Voicebot Enhancement ### Part 5: How to create LLM Agents that query databases, do web search and take action * 00:04 - Introduction and Overview * 00:32 - Analyst Team and LLM Agents * 04:16 - Configuring Perplexity Search Tool * 15:09 - Implementing Session IDs in React * 16:08 - Demonstrating Session Memory * 18:07 - Integrating Data SQL and Document PDF Analysts * 23:50 - Pushing to GitHub and Deploying to Vercel * 25:42 - Conclusion Hope you find the guide useful and enjoy using it as much as I enjoyed making it.