AI for IoT Latency: the Closer the Better
In most IoT systems, latency is an important issue. In critical control systems such as pipelines, electrical grids, factory automation, and autonomous vehicles, reaction time is critical.
Latency determines reaction time. Keeping sensor, actuator, and processing very close to each other produces the lowest latency. Replacing simple hardwired logic/microcontroller processing with
AI in IoT provides a significant increase in capability. But it brings latency challenges with it.
There are two ways of dealing with this challenge: finding ways to reduce the size of the LLM, and placing the LLM in the network as close to the sensors and actuators as possible. As AI hardware and software continue their rapid evolution, placing sensors, actuators, and LLMs in a single package will become possible. Therefore, IoT systems should be designed to fit today’s technology capabilities and to be able to migrate easily to new emerging capabilities as they emerge.
There are two ways of dealing with this challenge: finding ways to reduce the size of the LLM, and placing the LLM in the network as close to the sensors and actuators as possible. As AI hardware and software continue their rapid evolution, placing sensors, actuators, and LLMs in a single package will become possible. Therefore, IoT systems should be designed to fit today’s technology capabilities and to be able to migrate easily to new emerging capabilities as they emerge.
IoT AI Challenges
Latency can be thought of as a function of distance. Distance can be thought of as a combination of propagation delay and processing delay. The lower the distance between sensors and actuators,
the lower the latency. That distance is a function of the length of wires/fibers/wireless links between the sensor and the logic that interprets the sensor data and issues actuator
instructions; the processing time; and the length of wires/fibers/wireless links to the actuators. Putting this whole system into a single small package tends to minimize latency.
Etiya
Etiya is providing CX-focused, AI-driven Digital Transformation with its award-winning product portfolio, and disruptive BSS solutions for Digital Brands from launch to
evolution.
The company has been recognized in the 2025 Gartner® Magic Quadrant™ and Critical Capabilities Report for AI in CSP Customer and Business Operations.
3
With simple processing systems using hardwired logic or a small microcontroller, it was not so challenging to put the processing in the package with the sensors and actuators. With the advent of
GenAI and the desire to have more capable processing, able to make more sophisticated data analysis and decision making, the processing hardware requirements go beyond a simple
microcontroller.
AI processing requires at least a full processor and significant memory. This increases the power requirements. The increased power means increased heat that must be dissipated. Providing the increased power may also be a system challenge.
There are two ways to respond to this challenge: LLM size reduction and placing the LLM outside the sensor/actuator package, but as close as possible to it in the network.
AI processing requires at least a full processor and significant memory. This increases the power requirements. The increased power means increased heat that must be dissipated. Providing the increased power may also be a system challenge.
There are two ways to respond to this challenge: LLM size reduction and placing the LLM outside the sensor/actuator package, but as close as possible to it in the network.
LLM Size
Frontier LLM models are getting quite large. They run in large data centers with lots of space, power, cooling, etc. They are designed to meet a very broad range of uses and respond to a very
wide range of situations. Although the ability to run these large models on Edge devices is increasing, it is still challenging to run them in the same packages as sensors and actuators.
IoT systems tend to have a very narrow set of stimulus-response requirements and operate in relatively narrow, well-defined environments. Therefore, it is possible to use smaller, more specialized LLMs.
