The shift in perspective
For years, the debate on enterprise artificial intelligence focused on model capabilities: larger sizes, better reasoning, wider context windows, and higher benchmark scores. According to the analysis published by MIT Technology Review, produced by EXL, with the transition from experimentation to production the question for executives has changed: it is no longer what the model can do, but whether that intelligence can be delivered reliably, cost-effectively, and at scale.
The answer increasingly depends on the so-called serving harness, that is, the set of engineering systems surrounding the model: architecture, memory management, scheduling, and optimization choices that determine how efficiently the model's intelligence becomes a production capability. For technical leaders, this also shifts where to direct investments.
The workload before the model
According to the source, there is no universally better model or architecture, only the solution suited to the business problem. A customer-facing assistant optimized for rapid responses has different requirements from a document-processing pipeline that can work overnight; an internal knowledge application and a high-volume customer service platform can use the same base model but require very different serving architectures.
The workload defines the constraints, and the constraints should determine the architecture. It is a simple principle, but contrary to how many organizations approached their early implementations: during the proof of concept, model accuracy dominates, while production introduces latency, concurrency, memory pressure, GPU utilization, infrastructure costs, and unpredictable traffic. A model that provides marginally better responses is not necessarily the right choice if it doubles inference costs or fails to meet latency requirements.
Optimize after measuring
In production, seemingly isolated engineering decisions begin to influence one another: architecture affects memory utilization, memory management affects concurrency, concurrency affects batching and scheduling, and this affects GPU utilization and latency. Only within this framework do techniques such as quantization or speculative decoding express their value. The source cites GPU memory: during inference, model weights remain fixed, but the cache grows with active requests and context length, and a system can exhaust usable memory while compute capacity remains available.
The guidance for technology leaders is to measure before optimizing: identify the dominant constraint, address it, measure the result, and verify whether a new bottleneck has emerged. Optimization techniques should not be collected, but applied to the one that removes the highest-impact constraint. Governance changes accordingly: instead of asking what capabilities can be implemented, the question is what constraints separate a demonstrated capability from sustainable business value.
Adaptability and caveats
A further argument in favor of investing in the serving harness is that today's leading model probably will not be tomorrow's: models, hardware, and techniques will evolve. An architecture built on workload requirements, clear service objectives, and measurable production economics makes it possible to evaluate new models as configuration decisions rather than architectural resets. Because businesses have access to largely similar models, according to the source sustainable differentiation arises from how effectively intelligence is transformed into results.
It should be noted that the text is content produced by EXL and not written by MIT Technology Review's editorial staff: the arguments reflect the company's perspective and are not accompanied by quantitative data or verifiable cases. It therefore remains to be verified in the field how much the guidance on architecture, memory, and costs translates into measurable benefits across different organizations.
