BFCS.ai
EngineeringCheaper

Can AI Cut Prototyping Costs?

Every physical prototype has to be built, instrumented and tested, and some tests destroy it. Simulation and machine-learning surrogate models promise to replace some of those builds with computed ones. The record shows real reductions in physical testing, but the reductions we found are reported by the companies involved, and in our reading the saving only holds where the model has been validated against tests it was not trained on.

On this page
The BFCS verdictEvery article scored on the same four dimensions
Better
PartlyModerate confidence
Accurate inside the validated range; unreliable beyond itPhysics-informed methods can sharpen predictions from sparse data, but a peer-reviewed review warns that data-driven models can give physically inconsistent answers when they extrapolate.
Better: Note and confidenceModerate confidenceAccurate inside the validated range; unreliable beyond itPhysics-informed methods can sharpen predictions from sparse data, but a peer-reviewed review warns that data-driven models can give physically inconsistent answers when they extrapolate.
Faster
PartlyLow confidence
Fewer build-and-test loops; validation still takes timeComputed variants replace some physical iterations. We found no independent study measuring programme duration before and after surrogate models were introduced.
Faster: Note and confidenceLow confidenceFewer build-and-test loops; validation still takes timeComputed variants replace some physical iterations. We found no independent study measuring programme duration before and after surrogate models were introduced.
Cheaper
ConditionalLow confidence
Fewer prototypes where the model is validatedBoeing reports a fall from 77 to 11 wings tested in wind tunnels as its use of CFD grew, and Ansys (a simulation vendor) reports a race car programme cut to two full-scale crash tests. Both are self-reported, and in our reading neither nets off the cost of validation.
Cheaper: Note and confidenceLow confidenceFewer prototypes where the model is validatedBoeing reports a fall from 77 to 11 wings tested in wind tunnels as its use of CFD grew, and Ansys (a simulation vendor) reports a race car programme cut to two full-scale crash tests. Both are self-reported, and in our reading neither nets off the cost of validation.
Safer
ConditionalGovernance-gated
Only when model credibility matches the consequenceASME V&V 40 and FDA medical device guidance scale the evidence a model needs to its influence on a decision and the consequence of a wrong one. That FDA guidance does not cover standalone machine-learning models; in our reading, the burden then falls on the engineer who relies on one.
Safer: Note and confidenceGovernance-gatedOnly when model credibility matches the consequenceASME V&V 40 and FDA medical device guidance scale the evidence a model needs to its influence on a decision and the consequence of a wrong one. That FDA guidance does not cover standalone machine-learning models; in our reading, the burden then falls on the engineer who relies on one.

The problem, in numbers

Physical prototypes can be a slow and costly way to find out whether a design works. Each one has to be built, instrumented and tested, and destructive tests use it up. The simulation vendor Ansys, in a 2022 press release, put the cost of a single full-scale race car crash test at an estimated $500,000, and said simulation let NASCAR limit development of its Next Gen car to two full-scale physical crash tests, saving $1 million in physical testing material costs.Source 3 These are self-reported vendor figures, not audited.

One long public record of the trade-off comes from aerospace. Boeing reports that it tested 77 wings in wind tunnels for its 1980-era aircraft, the 767 and 757, then 18 for the 777 and 11 for the 787, while its computational fluid dynamics (CFD) runs grew to 60 times the 777-era level.Source 2 These are Boeing's self-reported figures from a 2008 conference presentation.

Researchers from the Defence Science and Technology Group (DST Group) wrote in a 2019 conference submission that a virtual fatigue test is the end goal for its Aerospace Division, giving Defence an agile and affordable way to certify aircraft structures for fatigue, and listed lower certification costs among the benefits of more accurate predictions.Source 1

The evidence, in four numbers
77 to 11
wings Boeing tested in wind tunnels, 1980-era aircraft to the 787
company self-reported (Boeing conference presentation, 2008); physics-based CFD, not machine learning
$500,000
estimated cost of one full-scale race car crash test
vendor estimate (Ansys press release, 2022, self-reported); currency not stated
>100% to <5%
error in inferring an alloy's yield stress, before and after multi-fidelity learning
laboratory study quoted second-hand in a peer-reviewed review (Karniadakis et al., 2021)
4 of 6
US hypersonic weapon efforts that had not adopted leading practices for digital engineering tools
audit finding at July 2024 (US Government Accountability Office); three of its four digital engineering recommendations later closed as implemented

What improves, and why

Two kinds of model do the work. Physics-based simulation, such as finite element analysis and CFD, solves the governing equations directly. Machine-learning surrogates learn to approximate a simulation or an experiment, so many variants can be screened without running the full model each time. Physics-informed machine learning sits between the two, building physical laws into training so the model needs less data.Source 4

The cost case is the same for each: replace some physical builds with computed ones, and use the builds that remain to check the model rather than to search the design space. The US Food and Drug Administration gives a plain example from medical devices: a validated finite element model picks the worst-case sizes in an implant family, so fewer articles need bench durability testing.Source 5 In our reading, that is the realistic pattern. The model narrows the field, and the physical test confirms the answer.

Physics-informed methods earn their place where test data is scarce. A 2021 peer-reviewed review reports a laboratory study in which combining many finite element simulations with sparse experimental data cut the error in inferring the yield stress of 3D-printed alloys from over 100% to below 5%.Source 4 That is a measured result quoted second-hand, in a materials-testing setting, not a saving on a product programme.

A surrogate model is cheap to run and expensive to trust. The saving is real only after someone has paid to prove it right.

What the evidence actually says

The evidence is thinner than the promise. We found no independent, peer-reviewed study that measured the total cost of a development programme before and after machine-learning surrogates were introduced. What we found falls into three kinds.

Company-reported reductions. Boeing's self-reported wing record is the longest-running example we found, and it concerns physics-based CFD, not machine learning. Boeing reports testing 11 wings for both the 737NG and the 787, while its CFD runs rose from four times to 60 times the 777 baseline.Source 2 In our reading, extra computing stopped buying fewer wind-tunnel wings at that point. The same presentation still describes an "extensive wind-tunnel test program" for the 787, and treats certification by simulation as an open question.Source 2

Vendor case studies. The Ansys figures are self-reported by the software vendor. Its release also says physical crash tests were later used to verify the simulation models.Source 3 In our reading, the physical test did not disappear, it moved from search to confirmation.

Research on the methods. The peer-reviewed literature is candid about limits. The 2021 review warns that purely data-driven models can fit observations well yet give physically inconsistent predictions through extrapolation or biased data, and that for well-posed forward problems needing no data assimilation, conventional grid-based solvers still outperformed physics-informed neural networks when it was written.Source 4 It also says meaningful benchmarks are still needed.Source 4 DST Group's Australian programme compares blind predictions with fatigue test data, aiming to close the gaps between predictions and test results.Source 1

Where it falls short

  • Outside the envelope, the model guesses. A surrogate trained on one range of loads, materials or shapes can extrapolate badly beyond it.Source 4

  • Validation is a cost, not a formality. The FDA treats calibration against the data used to build a model as weak evidence compared with validation against independent data.Source 5 It notes that most ASME V&V 40 validation activities assume well-controlled bench testing.Source 5

  • The standards are written for physics-based models. FDA medical device guidance on simulation credibility does not apply to standalone machine-learning models, and NAFEMS's new validation guidelines, described in a conference presentation, focus on physics-based models.Source 5,Source 6

  • Adoption has lagged the promise. In July 2024, US auditors found four of six hypersonic weapon efforts had not adopted leading practices for digital engineering tools. Three have since met the related recommendations; one program office concluded a complete digital twin was not practical or cost effective.Source 7

Where governance decides the outcome

The method for deciding how far to trust a model already exists. ASME V&V 40, a standard the FDA recognises, scales the evidence a model needs to its risk: how much the decision leans on the model, and how serious the outcome would be if that decision were wrong.Source 5 In our reading of that logic, a surrogate that only ranks concepts for a later physical test has low influence. One that replaces a qualification test has high influence and needs far more evidence. A review presented at the NAFEMS World Congress (a conference presentation, not peer-reviewed) points to the weak spot in practice: simulation quality assurance in industry depends largely on individual analysts, and designers and managers rarely question the results.Source 6 In our reading, every surrogate needs a written context of use, a named owner, validation data it was not trained on, and a rule for when a physical test is still required. The US auditor found Defense policy did not describe how to apply leading testing practices under digital engineering, and GAO listed all 13 of its recommendations as open as of March 2026.Source 7

How to think about adoption

Start where the physics is well understood and you already hold test data: a component family, a known material, a familiar load case. Use simulation and surrogates to cut the search, not the sign-off. Record which designs the model ranked and which physical tests confirmed it, so the saving can be measured rather than assumed. Budget for validation tests from the start, because they are what turns a cheap prediction into a trustworthy one. Treat vendor reductions in prototype counts as claims to test against your own programme. And keep a physical test wherever a wrong answer could put people at risk, until the model's credibility evidence matches the consequence.

How we read this evidence

BFCS.ai does not fabricate figures. Reductions in physical testing here are self-reported by a manufacturer or a software vendor, and the error figure is a laboratory result quoted in a review. We found no independent study measuring the total cost of a development programme before and after machine-learning surrogates were introduced, and we say so where it matters. Evidence on generative design is covered separately in our review of design cycles.

Sources

  1. 1

    Dixon, B., Burchill, M., Main, B., Stehlin, T., Rigoli, R. & Jackson, P., "Progress on the Pathway to a Virtual Fatigue Test," extended abstract submitted to ICAF 2019 (Defence Science and Technology Group, Australia, with RUAG Aviation, Switzerland; conference submission, not peer-reviewed). icaf2019.syskonf.pl

    What we took from it

    Responds to DST Group's 2017 call for a collaborative effort "to contribute to progressively closing the gaps between fatigue test results and model predictions"; lists lower certification costs among the benefits of more accurate analytical predictions; states that for DST Group's Aerospace Division "a virtual fatigue test is the end goal", providing an agile and affordable means of aircraft structural fatigue certification. The programme sets challenges in which researchers make blind predictions that are then compared with fatigue test data.

    Back to text
  2. 2

    Ball, D. N. (Chief Engineer, Enabling Technology and Research, Boeing Commercial Airplanes), "Contributions of CFD to the 787 and Future Needs," presentation to the IDC HPC User Forum, Stuttgart and London, October 2008 (company presentation, self-reported, not peer-reviewed). hpcuserforum.com

    What we took from it

    Slide "CFD Has Significantly Improved the Wing Development Process": wings tested 77 (1980-era 767 and 757), 38 (737-300), 18 (777), 11 (737NG) and 11 (787); CFD runs shown as a base at the 777, 4x at the 737NG and 60x at the 787. The 787 aerodynamic design still used an "extensive wind-tunnel test program"; a closing slide lists "Today: Certification by demonstration", "Tomorrow: Certification by simulation" and asks when tomorrow is. The reading that wing counts plateaued while CFD runs rose is ours.

    Back to text
  3. 3

    Ansys, "Ansys Validates Safety of NASCAR's Next Gen Race Car with Simulated Crash Tests that Enable Cost Savings," press release, 14 July 2022. ansys.synopsys.com

    What we took from it

    Vendor case study (self-reported, not peer-reviewed). Physical crash tests are "estimated at $500,000 each" (currency not stated); virtual testing reduced the programme to "only two full scale vehicle physical crash tests"; "When physical crash tests were later performed", the simulation models were verified. The release also says simulation "reduced material costs for physical testing by $1 million" and "resulted in cost savings of $1 million", without a method; it does not state how many simulations were run or what they cost.

    Back to text
  4. 4

    Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S. & Yang, L., "Physics-informed machine learning," Nature Reviews Physics 3, 2021, pp. 422-440 (peer-reviewed review), read via the US Department of Energy OSTI copy. osti.gov

    What we took from it

    Purely data-driven models "may fit observations very well, but predictions may be physically inconsistent or implausible, owing to extrapolation or observational biases"; for forward, well-posed problems without data assimilation, existing grid-based numerical solvers "currently outperform" physics-informed neural networks; the authors say "a concerted effort" is needed to design meaningful benchmarks. Reports, citing Lu et al. (Proc. Natl Acad. Sci. USA 117, 2020), that a multi-fidelity network combining many finite element simulations with sparse experimental indentation data reduced the inference error for the yield stress of 3D-printed alloys from over 100% to lower than 5%; we have not read the original study.

    Back to text
  5. 5

    US Food and Drug Administration, "Assessing the Credibility of Computational Modeling and Simulation in Medical Device Submissions," final guidance, 17 November 2023. fda.gov

    What we took from it

    Applies to first-principles (physics-based or mechanistic) models and "is not intended to apply to standalone statistical or data-driven models" such as machine learning or AI models; for hybrid models it covers only the first-principles part. Builds on the FDA-recognised standard ASME V&V 40-2018, under which model risk combines model influence and decision consequence and the credibility required is commensurate with that risk. Calibration against the same data used to set model parameters "is not validation evidence" and is weak compared with validation against independent data. Gives as an example a validated finite element model used to select worst-case device sizes "to minimize the number of bench test articles needed".

    Back to text
  6. 6

    NAFEMS World Congress presentations (conference presentations, not peer-reviewed). nafems.org

    "Guidelines for Validation of Engineering Simulations: A new NAFEMS Publication," NWC25, Salzburg, 19 May 2025: nafems.org

    What we took from it

    Santandrea, F. (RISE), "Quality Assurance of Engineering Simulations: a Review of Concepts, Methods, and Standards," NWC21, 26 October 2021: current practice for quality assurance of numerical simulations in industrial applications "is largely dependent on the individual responsibility of the analysts", and "the analysis results are rarely questioned by designers and managers".

    the guidelines "specifically focus on the validation of physics-based simulation models" and build on NAFEMS' Engineering Simulation Quality Management Standard (ESQMS:01, 2020).

    Back to text
  7. 7

    US Government Accountability Office. gao.gov

    "Weapon Systems Testing: DOD Needs to Update Policies to Better Support Modernization Efforts," GAO-26-107009, 11 December 2025: gao.gov

    What we took from it

    "Hypersonic Weapons: DOD Could Reduce Cost and Schedule Risks by Following Leading Practices," GAO-24-106792, 29 July 2024: of six offensive hypersonic weapon efforts reviewed, "four efforts have not adopted leading practices for using digital engineering tools"; the four related digital engineering recommendations were later closed as implemented for three efforts (Army LRHW, Navy SM-6 Block IB, Air Force HACM), and one remains open (Air Force ARRW); in GAO's status note on the Army recommendation (July 2025), the program office concluded that "a complete digital twin was not practical or cost effective" given limited planned production.

    DOD's digital engineering policy and the test and evaluation section of its systems engineering policy "do not describe specific processes to ensure application of leading practices to testing"; 13 recommendations, all listed as open on GAO's page with status as of March 2026 (checked 31 October 2026).

    Back to text
Continue the thread
More in EngineeringENGINEERING · FASTERCan AI Accelerate Design Cycles?7 min · Read →More in EngineeringENGINEERING · BETTERCan AI Reduce Engineering Errors?3 min · Read →Also CheaperMANUFACTURING · CHEAPERCan AI Cut Scrap and Waste?7 min · Read →