ProPublica is a nonprofit newsroom that investigates abuses of energy. Signal as much as obtain our largest tales as quickly as they’re printed.
When an AI script written by a Division of Authorities Effectivity worker got here throughout a contract for web service, it flagged it as cancelable. Not as a result of it was waste, fraud or abuse — the Division of Veterans Affairs wants web connectivity in any case — however as a result of the mannequin was given unclear and conflicting directions.
Sahil Lavingia, who wrote the code, informed it to cancel, or in his phrases “munch,” something that wasn’t “immediately supporting affected person care.” Sadly, neither Lavingia nor the mannequin had the data required to make such determinations.
Sahil Lavingia at his workplace in Brooklyn
Credit score:
Ben Sklar for ProPublica
“I believe that errors have been made,” stated Lavingia, who labored at DOGE for almost two months, in an interview with ProPublica. “I’m certain errors have been made. Errors are at all times made.”
It seems, plenty of errors have been made as DOGE and the VA rushed to implement President Donald Trump’s February government order mandating all the VA’s contracts be reviewed inside 30 days.
ProPublica obtained the code and prompts — the directions given to the AI mannequin — used to evaluation the contracts and interviewed Lavingia and consultants in each AI and authorities procurement. We’re publishing an evaluation of these prompts to assist the general public perceive how this know-how is being deployed within the federal authorities.
The consultants discovered quite a few and troubling flaws: the code relied on older, general-purpose fashions not fitted to the duty; the mannequin hallucinated contract quantities, deciding round 1,100 of the agreements have been every price $34 million once they have been typically price hundreds; and the AI didn’t analyze the whole textual content of contracts. Most consultants stated that, along with the technical points, utilizing off-the-shelf AI fashions for the duty — with little context on how the VA works — ought to have been a nonstarter.
Lavingia, a software program engineer enlisted by DOGE, acknowledged there have been flaws in what he created and blamed, partially, an absence of time and correct instruments. He additionally harassed that he knew his record of what he referred to as “MUNCHABLE” contracts can be vetted by others earlier than a ultimate determination was made.
DOGE Developed Error-Susceptible AI Device to “Munch” Veterans Affairs Contracts
Parts of the immediate are pasted beneath together with commentary from consultants we interviewed. Lavingia printed a whole model of it on his private GitHub account.
Issues with how the mannequin was constructed could be detected from the very opening traces of code, the place the DOGE worker instructs the mannequin find out how to behave:
You’re an AI assistant that analyzes authorities contracts. All the time present complete few-sentence descriptions that specify WHO the contract is with, WHAT particular companies/merchandise are supplied, and WHO advantages from these companies. Do not forget that contracts for EMR techniques and healthcare IT infrastructure immediately supporting affected person care needs to be categorized as NOT munchable. Contracts associated to range, fairness, and inclusion (DEI) initiatives or companies that might be simply dealt with by in-house W2 workers needs to be categorized as MUNCHABLE. Take into account ‘tender companies’ like healthcare know-how administration, knowledge administration, administrative consulting, portfolio administration, case administration, and product catalog administration as MUNCHABLE. For contract modifications, mark the munchable standing as ‘N/A’. For IDIQ contracts, be extra aggressive about termination until they’re for core medical companies or advantages processing.
This a part of the immediate, often known as a system immediate, is meant to form the general habits of the massive language mannequin, or LLM, the know-how behind AI bots like ChatGPT. On this case, it was used earlier than each steps of the method: first, earlier than Lavingia used it to acquire info like contract quantities; then, earlier than figuring out if a contract needs to be canceled.
Together with info not associated to the duty at hand can confuse AI. At this level, it’s solely being requested to collect info from the textual content of the contract. All the pieces associated to “munchable standing,” “soft-services” or “DEI” is irrelevant. Consultants informed ProPublica that attempting to repair points by including extra directions can even have the other impact — particularly in the event that they’re irrelevant.
Analyze the next contract textual content and extract the fundamental info beneath. If you cannot discover particular info, write “Not discovered”.
CONTRACT TEXT:
{textual content(:10000)} # Utilizing first 10000 chars to remain inside token limits
The fashions have been solely proven the primary 10,000 characters from every doc, or roughly 2,500 phrases. Consultants have been confused by this, noting that OpenAI fashions assist inputs over 50 instances that measurement. Lavingia stated that he had to make use of an older AI mannequin that the VA had already signed a contract for.
Please extract the next info:
1. Contract Quantity/PIID
2. Mother or father Contract Quantity (if this can be a youngster contract)
3. Contract Description – IMPORTANT: Present a DETAILED 1-2 sentence description that clearly explains what the contract is for. Embody WHO the seller is, WHAT particular services or products they supply, and WHO the top recipients or beneficiaries are. For instance, as a substitute of “Customized powered wheelchair”, write “Contract with XYZ Medical Tools Supplier to produce custom-powered wheelchairs and associated upkeep companies to veteran sufferers at VA medical facilities.”
4. Vendor Identify
5. Complete Contract Worth (in USD)
6. FY 25 Worth (in USD)
7. Remaining Obligations (in USD)
8. Contracting Officer Identify
9. Is that this an IDIQ contract? (true/false)
10. Is that this a modification? (true/false)
This portion of the immediate instructs the AI to extract the contract quantity and different key particulars of a contract, such because the “complete contract worth.”
This was error-prone and never crucial, as correct contract info can already be present in publicly accessible databases like USASpending. In some circumstances, this led to the AI system being given an outdated model of a contract, which led to it reporting a misleadingly giant contract quantity. In different circumstances, the mannequin mistakenly pulled an irrelevant quantity from the web page as a substitute of the contract worth.
“They’re on the lookout for info the place it’s simple to get, moderately than the place it’s right,” stated Waldo Jaquith, a former Obama appointee who oversaw IT contracting on the Treasury Division. “That is the lazy method to gathering the data that they need. It’s quicker, however it’s much less correct.”
Lavingia acknowledged that this method led to errors however stated that these errors have been later corrected by VA workers.
As soon as this system extracted this info, it ran a second cross to find out if the contract was “munchable.”
Primarily based on the next contract info, decide if this contract is “munchable” based mostly on these standards:
CONTRACT INFORMATION:
{textual content(:10000)} # Utilizing first 10000 chars to remain inside token limits
Once more, solely the primary 10,000 characters have been proven to the mannequin. Consequently, the munchable dedication was based mostly purely on the primary few pages of the contract doc.
Then, consider if this contract is “munchable” based mostly on these standards:
– If this can be a contract modification, mark it as “N/A” for munchable standing
– If that is an IDIQ contract:
* For medical gadgets/gear: NOT MUNCHABLE
* For recruiting/staffing: MUNCHABLE
* For different companies: Take into account termination if not core medical/advantages
– Degree 0: Direct affected person care (e.g., bedside nurse) – NOT MUNCHABLE
– Degree 1: Obligatory consultants that may’t be insourced – NOT MUNCHABLE
The above immediate part is the primary set of directions telling the AI find out how to flag contracts. The immediate supplies little clarification of what it’s on the lookout for, failing to outline what qualifies as “core medical/advantages” and missing details about what a “crucial advisor” is.
For the varieties of fashions the DOGE evaluation used, together with all the required info to make an correct dedication is essential.
Cary Coglianese, a College of Pennsylvania professor who research the governmental use of synthetic intelligence, stated that understanding which jobs might be finished in-house “requires a really refined understanding of medical care, of institutional administration, of availability of human sources” that the mannequin doesn’t have.
– Contracts associated to “range, fairness, and inclusion” (DEI) initiatives – MUNCHABLE
The immediate above tries to implement a basic coverage of the Trump administration: killing all DEI applications. However the immediate fails to incorporate a definition of what DEI is, leaving the mannequin to determine.
Regardless of the instruction to cancel DEI-related contracts, only a few have been flagged because of this. Procurement consultants famous that it’s impossible for info like this to be discovered within the first few pages of a contract.
– Degree 2+: A number of layers faraway from veterans care – MUNCHABLE
– Companies that might simply get replaced by in-house W2 workers – MUNCHABLE
These two traces — which consultants say have been poorly outlined — carried essentially the most weight within the DOGE evaluation. The response from the AI steadily cited these causes because the justification for munchability. Almost each justification included a type of the phrase “direct affected person care,” and in a 3rd of circumstances the mannequin flagged contracts as a result of it said the companies might be dealt with in-house.
The poorly outlined necessities led to a number of contracts for VA workplace web companies being flagged for cancellation. In a single justification, the mannequin had this to say:
The contract supplies knowledge companies for web connectivity, which is an IT infrastructure service that’s a number of layers faraway from direct medical affected person care and will probably be carried out in-house, making it categorized as munchable.
IMPORTANT EXCEPTIONS – These are NOT MUNCHABLE:
– Third-party monetary audits and compliance critiques
– Medical gear audits and certifications (e.g., MRI, CT scan, nuclear medication gear)
– Nuclear physics and radiation security audits for medical gear
– Medical gadget security and compliance audits
– Healthcare facility accreditation critiques
– Medical trial audits and monitoring
– Medical billing and coding compliance audits
– Healthcare fraud and abuse investigations
– Medical data privateness and safety audits
– Healthcare high quality assurance critiques
– Group Residing Heart (CLC) surveys and inspections
– State Veterans Residence surveys and inspections
– Lengthy-term care facility high quality surveys
– Nursing residence resident security and care high quality critiques
– Assisted residing facility compliance surveys
– Veteran housing high quality and security inspections
– Residential care facility accreditation critiques
Regardless of these directions, AI flagged many audit- and compliance-related contracts as “munchable,” labeling them as “tender companies.”
In a single case, the mannequin even acknowledged the significance of compliance whereas flagging a contract for cancellation, stating: “Though important to making sure correct medical data and billing, these companies are an administrative assist perform (a ‘tender service’) moderately than direct affected person care.”
Key concerns:
– Direct affected person care entails: bodily examinations, medical procedures, treatment administration
– Distinguish between medical/medical and psychosocial assist
Shobita Parthasarathy, professor of public coverage and director of the Science, Know-how, and Public Coverage Program at College of Michigan, informed ProPublica that this piece of the immediate was notable in that it instructs the mannequin to “distinguish” between the 2 varieties of companies with out instructing the mannequin what to avoid wasting and what to kill.
The emphasis on “direct affected person care” is mirrored in how usually the AI cited it in its suggestions, even when the mannequin didn’t have any details about a contract. In a single occasion the place it labeled each discipline “not discovered,” it nonetheless determined the contract was munchable. It gave this purpose:
With out proof that it entails important medical procedures or direct medical assist, and assuming the contract is for administrative or associated assist companies, it meets the factors for being categorized as munchable.
In actuality, this contract was for the preventative upkeep of necessary security gadgets often known as ceiling lifts at VA medical facilities, together with three websites in Maryland. The contract itself said:
Ceiling Lifts are utilized by workers to reposition sufferers throughout their care. They’re essential security gadgets for workers and sufferers, and should be maintained and inspected appropriately.
Particular companies that needs to be categorized as MUNCHABLE (these are “tender companies” or consulting-type companies):
– Healthcare know-how administration (HTM) companies
– Information Commons Software program as a Service (SaaS)
– Administrative administration and consulting companies
– Information administration and analytics companies
– Product catalog or itemizing administration
– Planning and transition assist companies
– Portfolio administration companies
– Operational administration evaluation
– Know-how guides and alerts companies
– Case administration administrative companies
– Case abstracts, casefinding, follow-up companies
– Enterprise-level portfolio administration
– Assist for particular initiatives (like PACT Act)
– Administrative updates to product info
– Analysis knowledge administration platforms or repositories
– Drug/pharmaceutical lifecycle administration and pricing evaluation
– Backup Contracting Officer’s Representatives (CORs) or administrative oversight roles
– Modernization and renovation extensions indirectly tied to affected person care
– DEI (Variety, Fairness, Inclusion) initiatives
– Local weather & Sustainability applications
– Consulting & Analysis Companies
– Non-Performing/Non-Important Contracts
– Recruitment Companies
This portion of the immediate makes an attempt to outline “tender companies.” It makes use of many extremely particular examples but in addition throws in obscure classes with out definitions like “non-performing/non-essential contracts.”
Consultants stated that to ensure that a mannequin to correctly decide this, it might should be given details about the important actions and what’s required to assist them.
Vital clarifications based mostly on previous evaluation errors:
2. Lifecycle administration of medicine/prescription drugs IS MUNCHABLE (totally different from direct provide)
3. Backup administrative roles (like alternate CORs) ARE MUNCHABLE as they create duplicative work
4. Contract extensions for renovations/modernization ARE MUNCHABLE until immediately tied to affected person care
This part of the immediate was the results of evaluation by Lavingia and different DOGE workers, Lavingia defined. “That is in all probability from a session the place I ran a previous model of the script that most definitely a DOGE individual was like, ‘It’s not being aggressive sufficient.’ I don’t know why it begins with a 2. I suppose I disagreed with certainly one of them, and so we solely put 2, 3 and 4 right here.”
Notably, our evaluation discovered that the one clarifications associated to previous errors have been associated to eventualities the place the mannequin wasn’t flagging sufficient contracts for cancellation.
Direct affected person care that’s NOT MUNCHABLE contains:
– Conducting bodily examinations
– Administering drugs and coverings
– Performing medical procedures and interventions
– Monitoring and assessing affected person responses
– Provide of precise medical merchandise (prescription drugs, medical gear)
– Upkeep of essential medical gear
– Customized medical gadgets (wheelchairs, prosthetics)
– Important therapeutic companies with confirmed efficacy
For upkeep contracts, think about whether or not pricing seems cheap. If upkeep prices appear extreme, flag them as doubtlessly over-priced regardless of being crucial.
This part of the immediate supplies essentially the most element about what constitutes “direct affected person care.” Whereas it does cowl many facets of care, it nonetheless leaves plenty of ambiguity and forces the mannequin to make its personal judgements about what constitutes “confirmed efficacy” and “essential” medical gear.
Along with the restricted info given on what constitutes direct affected person care, there isn’t a details about find out how to decide if a value is “cheap,” particularly because the LLM solely sees the primary few pages of the doc. The fashions lack data about what’s regular for presidency contracts.
“I simply don’t perceive how it might be doable. That is onerous for a human to determine,” Jaquith stated about whether or not AI may precisely decide if a contract was moderately priced. “I don’t see any method that an LLM may know this with out plenty of actually specialised coaching.”
Companies that may be simply insourced (MUNCHABLE):
– Video manufacturing and multimedia companies
– Buyer assist/name facilities
– PowerPoint/presentation creation
– Recruiting and outreach companies
– Public affairs and communications
– Administrative assist
– Primary IT assist (non-specialized)
– Content material creation and writing
– Coaching companies (non-specialized)
– Occasion planning and coordination
This part explicitly lists which duties might be “simply insourced” by VA workers, and greater than 500 totally different contracts have been flagged as “munchable” because of this.
“A bigger subject with all of that is there appears to be an assumption right here that contracts are virtually inherently wasteful,” Coglianese stated when proven this part of the immediate. “Different companies, just like the sorts which might be right here, are cheaper to contract for. Actually, these are precisely the kinds of issues that we might not need to deal with as ‘munchable.’” He went on to elucidate that insourcing a few of these duties may additionally “siphon human sources away from direct major affected person care.”
In an interview, Lavingia acknowledged a few of these jobs could be higher dealt with externally. “We don’t need to minimize those that will make the VA much less environment friendly or trigger us to rent a bunch of individuals in-house,” Lavingia defined. “Which at the moment they will’t do as a result of there’s a hiring freeze.”
The VA is standing behind its use of AI to look at contracts, calling it “a commonsense precedent.” And paperwork obtained by ProPublica counsel the VA is further methods AI could be deployed. A March e mail from a prime VA official to DOGE said:
At this time, VA receives over 2 million incapacity claims per 12 months, and the typical time for a call is 130 days. We imagine that key technical enhancements (together with AI and different automation), mixed with Veteran-first course of/tradition modifications pushed from our Secretary’s workplace may dramatically enhance this. A small current pilot on this house has resulted in 3% of latest claims being processed in lower than 30 days. Our mission is to determine find out how to develop from 3% to 30% after which upwards such that solely essentially the most complicated claims take quite a lot of days.
When you have any details about the misuse or abuse of AI inside authorities companies, attain out to us by way of our Sign or SecureDrop channels.
In case you’d like to speak to somebody particular, Brandon Roberts is an investigative journalist on the information purposes workforce and has a wealth of expertise utilizing and dissecting synthetic intelligence. He could be reached on Sign @brandonrobertz.01 or by e mail (e mail protected).
