Artificial Intelligence (AI) and the Internet of Things (IoT) are two of today’s most impactful technology developments. Inevitably, an increasing range of both enterprise and consumer applications and solutions leverage both technologies, so that they are enabled by both AI and IoT. A growing subset of these applications and solutions incorporate AI capabilities directly onboard an IoT device, as AIoT, unlocking benefits ranging from faster response times to more efficient use of connectivity bandwidth.
This article discusses some of the key capabilities that a software platform specified to support AIoT devices should have. It focusses on the specific requirements for supporting AIoT devices, rather than more generic requirements that are well-known in either IoT or AI contexts.
1.1 Software platforms to support AIoT
AIoT is a meeting between two different worlds with different rhythms. IoT device estates may contain multiple generations of hardware, in support of use cases that may vary between end-user deployments, by geography, and depending on connectivity technology. From an AI perspective, however, the frequently updated software models and user experience associated with these diverse endpoints should be as homogenous as possible.
Accordingly, the key capabilities that will be required to support AIoT device estates are a consequence of the interplay between two very different technology domains that are themselves relatively complex. They are summarised in the graphic below and discussed in the following subsections.
1.1.1 Compressing AI models for AIoT
AIoT environments are more constrained than cloud systems, requiring AI models to be compressed for deployment. Techniques like quantisation, pruning, knowledge distillation, and training smaller models help reduce size, although models must be retrained and tested after compression. Optimal compression varies by deployment context, connectivity type (and the cost of connectivity), and available hardware, creating trade-offs between performance, autonomy, and consistent user experience. AIoT platforms must reflect device and network differences, and fragmentation may grow as clients demand varying features. Rigorous ongoing testing remains essential for all compressed models as they evolve.
1.1.2 Updating AIoT software models
AI models in AIoT systems require frequent updates, retraining, and re-optimisation. These updates must be distributed efficiently, which is easier with minimal device fragmentation. Uniform software environments across devices is very much preferred, but not always possible in AIoT environments and over-the-air (OTA) updates are critical for deploying new software securely. AIoT platforms should support phased rollouts, rollback options, and A/B testing to manage disruptions and refine models in real-world conditions. Robust device management, already vital in IoT, becomes even more crucial in AIoT environments to ensure consistent, secure software deployment.
1.1.3 Managing AIoT in the field
Managing AI models in AIoT environments requires the management of any input, output, and concept drift. Unlike established AI settings, AIoT introduces unique challenges due to varying device conditions and contexts. Platforms must monitor performance, power use, and connectivity, and support drift detection, root-cause analysis, and contextual comparisons across device estates. Features like pre-emptive hardware maintenance, security monitoring (including physical interference), and fallback options to cloud processing will be key. AIoT platforms should also support adaptive communications strategies to minimise costly data transmission, especially for devices connected to cellular or satellite networks.
1.1.4 Supporting a feedback loop
AI model accuracy can degrade over time due to evolving input data, requiring retraining. In AIoT, decay may vary across subsets of a device estate based on deployment context, hardware variations, or environmental conditions. Platforms must detect instances of faster-than-average decay and provide insights for model maintenance. Such performance monitoring is harder with battery-powered or wirelessly connected devices due to cost and energy limitations. One solution is using always-connected “probe” devices to report performance, though this assumes their performance is representative of the broader device estate, and so this approach has inherent limitations.
1.1.5 Distributed learning
Distributed self-learning AIoT poses challenges as identical devices in different locations may evolve differently based on local conditions and experience of local events. This divergence makes it hard to generalise and share useful learnings, requiring expert insight to identify which rules can be applied elsewhere. For example, machine failure indicators may vary by environment, making direct comparisons difficult. AIoT platforms should detect these evolving differences and support engineers by highlighting potentially valuable new patterns and suggesting ways to adapt and distribute them across the wider device estate.
1.1.6 Hygiene factors
AIoT platforms must also prioritise a range of hygiene factors. Keeping the software bill of materials (SBOM) current ensures AI models run on consistent, compatible systems, helping to avoid suboptimal outcomes. AIoT platforms should track all model and SBOM changes to aid in performance audits and decay detection. They must also comply with evolving AI, data privacy, and sovereignty regulations by adapting software based on device location. Support for explainable AI and audit-ready configuration information will be essential to meet regulatory and operational standards in diverse jurisdictions.
1.1.7 Future requirements
Future AI deployments may involve splitting AI functions between IoT devices (AIoT) and local edge gateways (Edge AI), or across nearby AIoT devices. This creates added complexity, with AI components running on varied hardware and locations based on local context. Managing such distributed AI systems requires platforms that understand topography, connectivity quality, and device capabilities. While some vendors already offer solutions for heterogeneous edge environments, these must be enhanced to meet the specific constraints and requirements of AIoT scenarios.
1.2 Conclusions
AIoT platforms must merge capabilities from both AI and IoT domains, balancing AI’s fast-paced software evolution with IoT’s long-lived, resource-constrained devices. While many required functions exist in AI or IoT domains, they aren’t fully optimised for AIoT. IoT platforms often lack sufficient support for distributed AI, and AI platforms rarely consider IoT constraints. To support AIoT effectively, new capabilities are needed such as model optimisation for connectivity costs and power usage, especially for battery-powered devices. Additionally, performance and condition reporting must also account for similar limitations. A more cohesive and adaptable platform environment will be essential to fully realise the potential of AIoT technologies.