Three trends driving artificial intelligence architectures

Three trends driving artificial intelligence architectures

There is no doubt that business adoption of artificial intelligence (AI) is accelerating. Challenges remain, however, as companies look to deploy AI prototypes at scale. Whereas a proof-of-concept in a lab or public cloud may only use a few data sources, a typical AI/Analytics application in production will use many external data sources. With these data sources generating increasingly more extensive data sets at the digital edge, important decisions need to be made around operationalizing where compute power is placed, how data will be curated and governed and how AI models will be trained and improved, including who to partner with for data sharing.

The rise of data sharing

According to Kaladhar Voruganti, VP, technology innovation at the Equinix CTO Office to solve these challenges, it is helpful to look at three key trends driving AI architectures. The first is data sharing which he believes is essential for AI model accuracy. “AI algorithms are only as good as the data used to build them and usually need additional external data sources for more precision and contextual awareness,” he says. “For example, an AI model built to predict the spread of COVID-19 in a densely populated city like Singapore will not work well for a large, rural area in the US.

“Additional local data such as climate, demographics, testing status, healthcare system and more must be applied to the AI model for it to provide more accurate predictions. Data sharing between organizations is essential for this to work well, but it can be challenging due to data governance and privacy concerns. Confidentiality requirements vary depending on what data is shared, and this is leading to different types of data sharing models.”

Data sharing options

Voruganti highlights three crucial types of data sharing: data to compute, compute to data, and to bring data and compute to a neutral location. The bring data to compute sharing model, currently the most common form of data sharing, is typically used for non-sensitive data. “In this model, data providers send their data to a public data marketplace in the cloud for sharing with data consumers,” Voruganti explains. “Some enterprises are employing a hybrid architecture where they store their data in a cloud-neutral location like Platform Equinix and move it into the appropriate cloud on-demand for relevant AI processing or data sharing with partners.”

Bring compute to data is for more sensitive data such as patient or transaction information where enterprises are hesitant to let the raw data ever leave their premises. For example, hospitals want to share information with each other to build more accurate AI models, but, for confidentiality reasons, they do not wish to share raw data with individual patient records. “In these cases, AI processing is done where the raw data resides,” Voruganti says. “After the raw data is processed, only the resulting insights, anonymized meta-data or AI models are shared.

“This type of data sharing paradigm is also driving new federated AI learning techniques and frameworks. In a federated learning framework, analytics and model inference are moved to the edge, and only the local AI models are transferred to the data centres and clouds for global AI model building and training. This means only the models are moved upstream rather than the raw data.

“Federated learning also helps to ensure that the raw data is kept in the same location it was generated in for data compliance. Many countries have enacted or are in the process of enacting data residency laws that require data to be kept in a particular geographic location. In these cases, enterprises must do their AI processing within a particular country and geography.”

In some cases, the data providers do not want to share their raw data and data consumers do not want to share their AI algorithms. Consortium based data marketplaces within a vendor-neutral global interconnection platform like Platform Equinix make it easy for enterprises to buy and sell data/algorithms securely and compliantly, as well as build their AI models. “In many instances data is already being exchanged at the network level between different providers and enterprises at a neutral interconnection hub like Platform Equinix,” Voruganti adds. “Thus, this is the optimal place to also perform data exchange at a higher level for AI model training.”

Innovative public cloud models

The second trend that Voruganti highlights are that innovative public cloud AI models and services are driving hybrid multicloud architecture. Today organizations across many sectors are not in a position to build AI models from scratch. Instead, they want to augment existing AI models with their contextual data to create new models.

“Because these pre-built AI models require a vast amount of data and compute to train, they are generally only offered by major cloud service providers (CSPs),” Voruganti continues. “Enterprises want to leverage these sophisticated AI algorithms/models in the clouds for processing tasks such as image/video recognition, and natural language translation while maintaining control over their data. Most enterprises will also want to use AI models and services from different clouds for maximum innovation and to avoid vendor lock-in. This is driving a need for distributed, hybrid multicloud infrastructure for AI data processing.”

Growing volumes of data

The third and final trend is that the growing data volumes, latency, cost, and regulatory considerations are shifting AI data management and processing to a cloud-out and edge-in architecture. “Data is growing exponentially everywhere, including the edge,” Voruganti says. “For example, a connected car can generate up to three terabytes plus data a day, while a smart factory can generate 250x that much data a day. AI processing is moving to the edge for cost, latency and compliance reasons. This means that there are different types of edges which impact where AI processing is placed.”

Cloud-Out and Edge-In are two critical processing phenomena concerning how AI is moving from a centralized model to a distributed model. Cloud-out means some AI processing is moving out to the edge: Both AI training and inference operations are moving from the centralized cloud to the edge.

AI inferencing is moving to the edge:  

“Many real-time applications such as video surveillance, augmented/virtual reality (AR/VR) or multiplayer gaming cannot tolerate the latency of sending requests to an AI model in the core clouds for a response,” Voruganti explains. “For these use cases, AI inference needs to happen at the device edge or the micro-edge. In many markets, existing Equinix data centres can provide a round trip latency of < 5ms, and thus, can host these AI inference use cases. Many video surveillance and smart store shopping use cases need round trip network latency between 15-20ms.

“As more data is generated at the edge and IoT datasets are becoming larger, companies do not want to backhaul this data over costly, slow, high-latency networks to a core cloud for AI model training. Also, certain types of data must be kept on-premises for data privacy or residency. Over 132 countries have already enacted or are in the process of adopting data privacy/residency laws. This is ideally suited for federated AI model training techniques to train AI models at the edge (bring compute to data) and then aggregate these local (potentially suboptimal) AI models at a core data centre to build better global AI models. Federated learning also helps to ensure that the raw data is kept in the same location it was generated in for data compliance. And moving analytics closer to the edge improves performance and cost-efficiency.”

Edge-in means deep learning, and model training is moving in from the far edge to the metro edge. Latency sensitive AI inference cannot take place in a public cloud. In many cases, inference operations can take place at either of device, micro or metro edges. “However, for cost reasons and data fusion reasons, it is beneficial to move up the edge hierarchy (edge-in),” Voruganti adds. “For example, smart cameras can do AI inference at the device level, but these devices can be costly.

“Alternatively, regular cameras could be used if the AI inference processing is moved higher up in the edge hierarchy to micro or a metro edge (depending upon the latency requirements). And, except for life-critical operations that require less than five milliseconds (5ms) round trip latency, this should satisfy real-time latency requirements at more optimal cost points.

“Furthermore, in many use cases, data from additional external sources and databases need to be fused to improve model accuracy. Due to their compute and storage resource requirements, these additional data sources often cannot be easily hosted at the device or micro edges. Furthermore, with the emergence of 5G networking technology, more processing can be moved from the devices to the micro and metro edges due to lower latencies and better bandwidth.”

Hardware that does the AI model training has high power requirements (30-40KW for fully loaded rack), so it cannot be hosted at the micro-edge. “Furthermore, most private data centres are also not equipped to handle beyond 10-15 KW per rack, so AI training hardware typically needs to be hosted at a colocation data centre,” Voruganti concludes. “It is also beneficial to colocate AI training hardware at an interconnection rich data centre due to high-speed connectivity to multiple clouds and networks, global footprint so that you can adhere to data residency requirements and a dynamic global ecosystem of nearly 10,000 companies.”