Over the past few months I have sat across the table from managed service providers in Milwaukee and across Wisconsin, and the conversation keeps arriving at the same place. Their clients are already using AI at work, often through personal accounts on free tools, and the MSP is the one who gets the call when a client file lands in the wrong place. Many of these MSPs would like to offer a governed alternative, and they want a clear way to judge the vendors who ask to partner with them.
This checklist is the one I would want an MSP to use on us, and on any other vendor offering a managed intelligence platform. It builds on the vendor review habits MSPs already use for security tools, and it adds the questions that are specific to AI: who decides what the model sees, how a model change gets approved, and what keeps running on the day the vendor goes away. I have grouped it into ten parts, each with the questions to ask and the evidence a strong answer comes with.
A quick note on language. A managed intelligence platform, as I use the term, is an AI system a vendor configures and maintains for a client while the MSP keeps ownership of the infrastructure and the client relationship. The questions apply whether the platform runs on the client's own machine, on the MSP's server or in a private cloud.
Part 1. Vendor viability and continuity
Many AI vendors are young companies, ours included, so continuity deserves a direct conversation early. The goal is to know exactly what keeps running for the client if the vendor is acquired, changes direction or closes.
Questions to ask:
- Who are the people behind the platform, and who covers for them during an outage or an absence?
- If the managed service ends, which parts keep working on the client's infrastructure, and which parts stop?
- Is the source code held in escrow, or is an equivalent arrangement written into the contract?
- How does the client get a full export of their data, configuration and logs?
- How often does the vendor release, and does each release come with notes and a version number?
Strong evidence: a written continuity plan, escrow or equivalent terms in the draft agreement, and a demonstration of the platform running with the vendor's connection switched off.
Part 2. Security posture and supply chain
MSPs already ask security vendors for proof, and an AI vendor belongs in the same review. An early vendor may still be working toward a SOC 2 report, and a credible substitute, such as a completed CSA CAIQ Lite questionnaire or a CIS Controls IG1 self-assessment, gives the MSP something concrete to review in the meantime.
Questions to ask:
- Which third-party attestation does the vendor hold, or which substitute will they complete for the MSP?
- Is multi-factor authentication enforced on the vendor's code hosting and admin accounts?
- Does the vendor keep any standing access into client deployments?
- Does each release ship with a software bill of materials, and are the installers code-signed?
- How does the vendor receive vulnerability reports, and what patch timelines do they commit to?
- Which suppliers does the platform depend on, including model publishers and model runtimes?
Strong evidence: completed questionnaire responses, a supplier list, signed installers and a published disclosure policy.
Part 3. AI governance quality
This is the part of the review that separates a governed platform from a general chat tool. I recommend asking to see each control working in a live demonstration, because a policy document describes intent and the running system shows behavior. The NIST AI Risk Management Framework and ISO/IEC 42001 give both sides a shared vocabulary for this conversation.
Questions to ask:
- Does every model request pass through one checkpoint that applies the client's policy, and can other programs on the host reach the model directly?
- Is the client's policy tied to the running configuration, so that any configuration change shows up as an unsigned policy?
- Where in the workflow does a person approve the AI's output, and can that step be skipped?
- What does the approver see when they make the call: the draft alone, or the draft with its sources and the single check that matters?
- Is there a kill switch, who holds it, and how quickly does it stop every agent and every outside call?
- How does the platform handle prompt injection, meaning instructions hidden inside documents or emails the AI reads? The OWASP Top 10 for LLM Applications is a useful reference here.
- What does the decision log record for each call, and how is the log protected from editing?
Strong evidence: a live demonstration of each control, plus the test results the vendor runs on every release.
Part 4. Data handling and isolation
Data questions are where an MSP's existing expertise pays off, because they map closely to the work MSPs already do around backups, encryption and access.
Questions to ask:
- Is each client in a separate, isolated deployment, and how has cross-client isolation been tested?
- Can the vendor draw a data flow map showing every place client data travels, including any outside model provider?
- How does the platform classify data, and how does it treat information that arrives unclassified? A strong answer treats it as sensitive by default.
- Is any client data used to train or tune a model, for this client or for anyone else?
- How are retention and deletion handled, and what does offboarding look like step by step?
- Which party holds the encryption keys, and which party runs backups and restore tests?
Strong evidence: a data flow map, a written responsibility split for backups and encryption, and a restore drill the MSP and vendor run together.
Part 5. Model management
Models change often, and every change can shift the answers a client relies on. A strong vendor treats a model change the way an MSP treats a major patch.
Questions to ask:
- Does the vendor keep a register of every model in use, with versions pinned and licenses recorded?
- How is a new model evaluated before it reaches a client, and is the evaluation run on that client's own examples?
- Who approves a model change, and where is the approval recorded?
- Is the model runtime limited to the local machine, with model download and deletion functions blocked?
- How is the runtime itself patched, and is its version recorded?
- For white-label arrangements, which license obligations pass to the MSP?
Strong evidence: the model register, an evaluation report and the change record for the last model update.
Part 6. Supportability for the MSP
The platform has to fit into the tools the MSP already runs every day, or it becomes a separate island that costs more to support than it earns.
Questions to ask:
- Does the installer support silent deployment through the MSP's RMM?
- How are updates delivered, and can the MSP schedule them and roll them back?
- Does the platform expose a health signal the RMM can monitor?
- Does sign-in work with the identity provider the MSP already manages?
- How are the support tiers split, and what does the severity matrix say?
- Which runbooks exist for install, upgrade, restore and offboarding?
- How many hours does onboarding a new client take, measured on a real deployment?
Strong evidence: a test install through the MSP's RMM, runbooks the MSP can read before signing, and a support matrix in the draft agreement.
Part 7. Commercial model
Good partnerships leave room for both businesses to earn, with terms clear enough for both sides to plan around.
Questions to ask:
- Which partner models does the vendor offer: referral, wholesale, white label or licensing?
- Who sets the client price, and how much margin room does the wholesale price leave?
- How are price changes handled, and with how much notice?
- Are there minimum commitments, and how does billing fit the MSP's current cycle?
- Which services does the vendor sell directly, and does that list overlap with what the MSP sells?
- What are the term, renewal and exit conditions?
Strong evidence: a written partner price book and a draft agreement that covers channel conflict, branding and exit.
Part 8. Go-to-market enablement
Clients buy AI from their MSP once they understand what it does for their work, so the vendor's enablement matters as much as the platform itself.
Questions to ask:
- Which workloads has the vendor defined for the MSP's client verticals?
- Is there an assessment-led first offer the MSP can lead with?
- Is there a demo environment the MSP's team can run with the vendor out of the room?
- How much co-selling time can the vendor commit each month?
- Does the vendor's marketing stay within claims it can prove, especially around compliance? A vendor who calls a platform HIPAA or GDPR compliant on a client's behalf is handing that risk to the MSP.
Strong evidence: a partner one-pager, a demo the MSP can run alone, and collateral the MSP's own compliance lead is comfortable sharing.
Part 9. Liability and insurance
AI introduces new questions for insurers, and both businesses need coverage that lines up.
Questions to ask:
- Which technology errors and omissions and cyber policies does the vendor carry, and will they share certificates?
- Do either party's policies contain exclusions for AI-related claims?
- Do the liability caps in the vendor agreement line up with the caps in the MSP's client agreements?
- How are indemnities and carve-outs written?
- Is human approval of AI output written into the contract as well as built into the product?
Strong evidence: certificates of insurance, a conversation with the MSP's own broker, and caps that line up across both contracts.
Part 10. Pilot and proof
A well-run pilot answers every question above with evidence from the MSP's own environment. I recommend agreeing on the pass and fail criteria in writing before the first install.
Questions to ask:
- What are the written success criteria, and who signs the go or no-go decision?
- What value will the client see in the first week?
- Will the pilot be deployed with the MSP's own tools?
- Which governance tests will the vendor run in front of the MSP's team? A strong pilot includes an isolation test, a direct runtime access test, an unsigned policy test and a timed kill switch test.
- Will the pilot include a support drill and an offboarding and export drill?
Strong evidence: a pilot plan with criteria, test scripts and a dated go or no-go review.
How we use this checklist
We built this checklist from the reviews MSPs have walked us through, and we hold our own platform to it. In a partner conversation we share where each item stands for us today, part by part, and we encourage MSPs to expect the same openness from every vendor they consider. If you lead an MSP and would like to walk through the checklist together, we would be glad to start with a conversation.