Are you letting an AI chatbot decide what your most sensitive data is?
It’s a decision a lot of organizations are making (or not making) right now, whether they'd describe it that way or not. You connect a model to your fundraising tool or database, click "Allow" (aka you hand it a set of credentials), and from that moment on, the question of what should and shouldn't surface gets answered by the model, in real time, behind the curtain.
No LLM will make that call the way you would. They can't. The models have no idea that your recurring donor list, your prospect research notes, and your membership roster are a different category of thing than your event attendee counts. The model just sees columns.
Four ways this goes wrong, even without a bad actor:
Nobody specified “return names” in their prompt, but names were surfaced anyway. A staffer asks an LLM how many members we have in Ohio, a totally routine question. To answer it, the model queries the member table, gets rows back, counts them - and then, because models like to show their work, returns the count along with a sample of the records behind it. In this sample are names, phones, employers, and whether they signed a card. That answer then gets pasted into Slack, and it's searchable forever. Because no one exported anything there's no record that a member list ever left the database, but during an active membership drive that list is now one screenshot away from the employer.
A paste is an export, but you promised you wouldn't. Your development director has a call in an hour, pulls 400 prospect records, pastes them into a chatbot, and asks for a one-page brief. It's a good brief! Alas, that paragraph on your website that says you don't share donor information with third parties... you just did. On a free or personal account, that content can be retained and used to train the model, which means the candid note your development director wrote about somebody's divorce is now in a place you can't reach into. And if that donor ever asks you to delete their record, that’s not even possible.
Chat will hand you a number and sound certain about it. Ask an LLM for recurring donor retention over eighteen months, and you'll get a number. What you won't get is "heads up, I picked a join that dropped everyone who changed their email, so this is off by about a third." Then that number goes in the board deck. Then it sets next year's budget. A confidently wrong number is worse than no number - a missing number makes you go searching for the right answer, but a wrong number leads you astray.
An agent will take instructions from anyone, not just you. The moment you give a tool access to your inbox and your CRM so it can triage donor email and log the replies, you've built something with three properties at once: it can see private data, it can send things outward, and it reads content written by strangers. That's the whole recipe. A hidden line of text in an inbound email, or in a web page you asked it to summarize, can redirect it - nothing sophisticated is required, just instructions written in text the model reads and your staff never see.
It's important to call out: shadow operating (i.e. staffers using AI on their personal accounts in the shadows) is a symptom, not a discipline problem. The standard management reflex is to ban the tools, or to throw up your hands and give the models everything. Neither works and both make your position worse. A ban doesn't stop the behavior - it moves it somewhere you can't see. Nobody is going around you because they're careless. They're going around you because they have a report due, a dataset that's a mess, and not enough hands to do it. If your AI policy is a ban, you don't have an AI policy - you have shadow operating with compliance document wrapping paper around it.
What governance actually looks like
Good governance should cover seven areas, roughly in order of importance:
Decide what's off-limits before you connect anything to a model. Three tiers is enough. Open: aggregates, published metrics, anything you'd put in a newsletter. Restricted: individual-level records, available with a named approver and a stated purpose. Never: donor notes, and anything where exposure is a safety problem rather than an embarrassment - member lists during an active drive, program participation in immigration or health services, anyone whose association with you could cost them something. Write it down, date it, give it an owner. This core data inventory is essential to everything else.
Enforce it in the data layer, not the prompt. "Please don't query the donor notes" is a request, and a request is not a control. Controls are permissioned views, row-level security, column masking, and a defined set of approved metrics the model queries instead of raw tables. An easy test: if the only thing between a curious question and your donor notes is the model's judgment, you don't have sufficient governance. Build your data layer so the model is structurally unable to reach what it shouldn't. While you're setting your data layer, you’ll probably realize that half your staff has more access than they need for their work - fix that too.
Give the AI its own account, scoped to the job. Do not use an admin service account, and do not use an individual’s credentials. Instead set up a separate identity with the narrowest permissions the task requires. This is what makes your logs legible later - you can tell a person from a tool - and it means shutting it off is one action you can actually confidently track.
Log every query, and have a human read the logs. Reviewing monthly is plenty. The point isn't catching people; the point is "who touched the donor table in September" needs to have an answer, and prompt history in somebody's personal account won’t get you that. In that review you're looking for three things: queries that returned individual-level rows, anything that hit a restricted table, and volume that doesn't match the work. Devote fifteen minutes once a month, assign this to someone who knows what’s in which tables, and ensure your structures are functioning correctly.
Put a name on every output that leaves the building. “Human in the loop” is just a slogan until it's a specific person who read the thing, checked the number against something else, and owns it if it's wrong. Be precise about what review means, too. Reading the prose isn't review, looking at the query, the row count, and whether the number is in the range you'd expect is review.
Build the path people will actually use, and make asking safe. Choose your organization’s approved tool, set a short list of what's allowed and isn’t, and identify one person to field questions. Reiterate these three repeatedly so they’re engrained with your team. Then say the harder part explicitly: if you put something in the wrong place, tell us today and nothing happens to you. Honesty from your team without risk of repercussion is the difference between hearing about a bad paste today versus learning about its consequences a year from now.
Decide now what happens the day it goes wrong. Who gets the call, who has the authority to pull the connection, how do you classify what happened, and when does a partner or funder need to hear it from you rather than from someone else? Most of the hard questions here aren't technical - they're organizational. Decide the plans now while the worst case scenarios are only hypotheticals.
After you work through the above, go read your agreements. Review things like data-sharing terms, vendor contracts, your own privacy policy, donor privacy language, and terms on your website. A lot of it will likely cover third-party processing, was written well before any of this existed, and will apply anyway.
None of this is an argument for slowing down. Our side is outnumbered and outspent, our teams are stretched well past reasonable, and dollar for dollar these tools are one of the few real multipliers available to us this cycle. I want organizations moving fast.
Governance is what makes fast survivable. Skip it and the first serious breach in our ecosystem - and there will be one - doesn't only cost that organization. It becomes the story every board and every funder cites for the next five years, and somewhere inside it is a person whose name got out at the worst possible moment.
If you need help working through AI governance for your organization, we can help! Let me know at AIHelp@techallies.org .
