My board says do AI. Halp plz

Over the last year or so, the data science team at IDinsight has been busy building AI products like Ask-A-Question and Ask-A-Metric. A few months back, I was on a panel on GenAI for Social Impact and was asked if they should be investing in AI or not. I talked about how AI is a tool and we want to be problem driven. I talked about thinking about your use case and finding the tool that fits it best instead of starting with the AI hammer and looking for a nail.

With all the hoopla around AI of late, these are questions on the minds of almost all social sector organizations. Below is how I see it. I’d love to hear your thoughts.

Should I be investing in AI?

The short answer is “yes” but it’s also the wrong question.

There was a time when people asked “Should I be investing in computers?” In hindsight, the answer was obviously “yes.” They are now used in almost all aspects of life and business. But even then the right question to ask was “What should I use computers for?” And that was very subjective. It required deep knowledge of your organization and the population you served to understand how it might be useful.

AI is similar. There will be different ways in which AI can improve various aspects of your organization. Identifying what you should use it for requires understanding your current pain points, binding constraints, and impact channels. To cut through the hype, you need to remain problem driven - you may not need the latest model from OpenAI, something simpler may be an even better fit.

The right use case is at the intersection of feasible (technically, financially) and a well-defined problem statement. After you have articulated the problem statement, you need to know if the current state of AI offers a well-established solution or are you hitting the frontiers of research. You’ll probably need a data scientist for this; knowing what is easy and what is difficult for AI requires understanding the methods and their limitations. By this we mean a hundred percent accurate - zero hallucinations and no misleading responses.

What kind of AI use cases should I consider?

There are three categories of AI use-cases:

1. Improve how your organization works.
2. Improve the experience of your current channels.
3. Create new channels of impact through AI.

Improve how your organization works

You know your organization well and understand its pain points. Is it that knowledge is scattered and it takes forever to find anything? Is it that data is locked in databases and it is not readily available for decision-making? You may be able to deploy an AI solution for that.

Since it is an internal user base, your risk is low. We all know AI hallucinates. But it is easier to educate an internal team on its risks and correct usage than the population you serve. You can also abandon it easily (especially if you “adopted” – more on that below) and you don’t take on any reputational risks since it’s all internal.

Improve your current services through AI

One of the reasons why MomConnect was successful in deploying AI was that they already had an existing WhatsApp platform with millions of active users and they used AI to just improve their experience. They didn’t need to invest in changing user behavior or do large campaigns to get users on the platform. Users continue to use the platform the way they did previously, except now they just get a nicer experience when looking up information.

Think about if there are parts of your user’s journey that can be improved. Where are you losing your users? What are frictions they experience?

Create new channels of impact through AI

The third kind of use-case is creating an entirely new channel to support your beneficiaries. Examples may be a new app to connect citizens with benefits, or a new WhatsApp hotline that answers questions from farmers, or an AI-diagnostics assistant for community health workers.

These can have a large, outsized impact if done right but requires you to think deeply about user journeys, outreach and comms, and do extensive user testing. Like any software solution, how well you know your users will determine how successful it is.

Ok. How should I get started?

There are three ways to get started. If you are not familiar with AI, unsure if it will be useful, or don’t have a strong engineering team, start with “Adopt”.

Adopt

This is the cheapest place to start and hence least risky. Identify an “off-the-shelf” AI-powered SaaS product that seems like a good fit. Pay the small cost to try it out. If it doesn’t create that value you thought, you can easily abandon it and try another. You’ll learn a tonne along the way on what works and will be able to articulate your requirements better.

Pros:

• Cheap to try.
• No need for an engineering team. Someone else is managing the app for you.
• You can start using AI tomorrow! You get new features for free.

Cons:

• You may not get exactly what you want.
• You may not get to dictate which direction the product goes in.

Customize

Identify an open-source product that fits 80% of your use case. Deploy it on your infrastructure and try it out. Invest in making the changes you need to get to the remaining 20%.

Pros:

• Not that expensive.
• Start using AI in weeks.
• You get new features for free as the open-source product evolves

Cons:

• You need an engineering team.
• You may find that extending it is not trivial especially if the product was not built to do what you want.
• You need to invest in maintenance as the product improves and bugs are fixed.

Build

Build exactly what you need from scratch. But keep in mind that if you are building something that will scale and is easy to maintain, it will not be quick or cheap.

Pros:

• You get exactly what you want.

Cons:

• You need a strong engineering team with some AI knowledge.
• This will take way longer than you think (and therefore quite expensive).
• You need to maintain the product and add features and fix bugs all by yourself.

What kind of gotchas do I need to be aware of?

GenAI is expensive to run at scale

Traditionally, the financial model for software projects was that you pay a large fixed cost and the marginal cost of bringing on a new user is practically zero. Now your variable cost scales with the number of users –– you pay the token cost for every new question your users ask. The cost model is not sustainable if you have massive user base.

One way around it is to fine-tune smaller models to replace parts of your pipeline. Instead of using all the free credits you got from OpenAI or Google to subsidize your running costs, you should instead be using it to fine-tune an open-source model and reduce your variable cost.

Evaluating GenAI is tricky and expensive

LLMs generate text. Knowing if it is correct, complete, and not misleading is not trivial and requires a lot of investment –– in validation sets, in engineering time, and in tokens!

GenAI is easy to try, hard to scale

Going from an idea to a proof-of-concept is easy. A good engineer can throw together code in a few weeks to demonstrate a use case. Building a solution that will be used by thousands of users, be safe and not hallucinate, be stable, and be cost-effective takes months.

“We live in the year 5000”

That’s what my colleague Tony said when sharing yet another incredible open source AI library. It’s a wild world out there and what is feasible is changing everyday. Stay safe and have fun.

And yes. I know that Daleks are not robots.