Welcome to the IDinsight Tech Blog
Check in regularly or sign-up to the IDinsight newsletter to find out about IDinsight's latest data science and engineering work.
All Posts
Comparing Ask-a-Metric and Vanna.ai
TL;DR: We are comparing Ask-a-Metric (AAM) and vanna.ai performance, on metrics that we find ourselves regularly testing for AAM use-cases. We find that Ask-a-Metric performs on-par with vanna.ai for straightforward queries, but struggles with more complex queries. Vanna.ai also struggles with complex queries and lacks guardrails, but has a greater range of features than AAM.1
Using Agents to Not Use Agents: How we built our Text-to-SQL Q&A system
Ask-a-Metric is a WhatsApp-based AI data analyst that uses LLMs to answer SQL database queries, facilitating data access for decision-making in the development sector (GitHub). Initially, we used a simple pipeline for rapid feedback but faced challenges in accuracy and building it for scale. We tested an agentic approach with CrewAI, improving accuracy but ending up with high costs and slow response speeds. We used these results to develop a pseudo-agent pipeline that combines the best of both approaches, reducing costs and response times while maintaining accuracy.1
Search is all you need... probably
It’s the GenAI age. Every person and their grandpa is creating AI chatbots based on RAG. For farmers, for mothers, for teachers, for bureaucrats. Hey, we’re doing it too! But here’s a hot take: you don’t need a RAG AI chatbot. Definitely not at the start. Probably not ever.
Clustering algorithms for grid-based sampling
TL;DR: In this blog post, we will describe a custom clustering algorithm we designed to efficiently cluster grids into enumeration areas for grid-based sampling The DSEM team at IDinsight is the technical workhorse for project teams, and nearly every piece of technical work we do involves grouping things by some measure of similarity. Let me explain.
Making satellite imagery easy-to-use: speeding up computations
In our previous post, we examined how satellite imagery can be used in the social sector and how the MOSAIKS algorithm enables us to draw out “features” from these images without needing complex image-processing models. But the story doesn’t end with the algorithm.
