Third-Party AI Vendor Due Diligence

The question a security review does not ask

Vendor diligence for AI has to test recordkeeping capability, not only security posture. Those are different systems inside the vendor and a strong answer on one implies nothing about the other.

The FINRA 2026 Annual Regulatory Oversight Report describes testing third-party vendors' ability to meet SEC Rules 17a-3 and 17a-4 obligations through simulated regulatory examinations. That is a production test, not a questionnaire.

Recordkeeping questions

These establish whether the vendor can support your Rule 204-2 and 17a-4 obligations.

• Can you produce all outputs generated for a named client across a date range, and in what format?
• Are outputs retained in the form they existed at the time, or regenerated on request? Regeneration is not production.
• What is your retention period, and what happens to records if we terminate?
• Can you supply records in a format suitable for a regulatory production without manual reconstruction?
• Are records held in a manner that meets the preservation requirements our firm is subject to?

Model governance questions

These establish whether you can answer the model-version question the FINRA report raises.

• Which model or models generate output in our instance, and are they yours or a third party's?
• Do you record and expose the model version attached to each output?
• How are we notified of a model change, and how far in advance?
• Are prompts and outputs logged, and can we access those logs?
• Is our data used to train models serving other customers?

The notification question is the one that separates vendors. A silent model upgrade changes output behaviour mid-period, and a firm that learns about it afterwards cannot explain a change in output quality to an examiner or a client.

Supervision questions

These establish whether the vendor's design supports the supervision obligation that sits with you.

• What controls exist to prevent output reaching a client without human review?
• Is there an audit trail of who reviewed, approved or edited an output, and when?
• How are errors surfaced, and what is the notification path to us?
• What happens when the model cannot answer — does it decline, or does it produce something?

Disclosure questions

These protect you against inheriting a vendor's overclaiming.

Per Goodwin's summary of the SEC's 2026 examination priorities, firms must ensure marketing, Form ADV disclosures and client communications accurately describe AI's extent, nature and limitations. A vendor's marketing claims become your disclosure problem the moment you repeat them.

• What does the system actually do, stated without marketing language?
• What are its known limitations and failure modes?
• What accuracy claims do you make, and what evidence supports them?
• Will you support our Form ADV disclosure language describing your role?

How often

Before adoption, and at least annually after. AI vendors change capability between review cycles more often than traditional software vendors, and a diligence file that describes a product as it was eighteen months ago documents a system the firm is no longer using.