The 4 step process to achieve AI data locality while serving AI globally
By Reda Oulbacha and Nathan Silberman
Artera is improving cancer outcomes around the world
As part of Artera’s mission to improve patient outcomes globally, Artera launched a highly successful product for patients with localized prostate cancer, the ArteraAI Prostate Test. This AI-driven test helps patients and their clinicians make optimal and personalized treatment decisions. The accuracy and clinical utility of the ArteraAI Prostate Test has resulted in its inclusion into the National Comprehensive Cancer Network (NCCN) Clinical Practice Guidelines in Oncology (NCCN Guidelines®). Naturally, Artera is focused on making this product available to patients globally.
To support this goal, Artera’s engineering teams needed a robust and scalable solution to serve our clinical AI models in a variety of countries. Every country has their own regulatory and compliance requirements. A common requirement is that of data locality: sensitive patient data must not leave the country of origin. With that in mind — what is the best scalable architecture that features data locality while ensuring that observability and maintenance are easy for software engineers?
In this blog, we share our 4-step “recipe” for deploying clinical AI models on a global scale, using our recent expansion into Australia as a prime example. By focusing on our journey to a localized deployment in Australia, and through our partnership with GenesisCare, we illustrate how each step in our process meets stringent regulatory requirements in the context of Data Locality.
Background — Workflow Orchestration
Artera’s AI products are composed of many individual steps in a complicated workflow, often involving many individual AI models that perform different tasks. We’ve partnered with Union.ai to manage our workflows, helping us move faster and abstract away complexity as we build our compound AI system.
For the purpose of this article, we’ll assume that a single pipeline is composed of the following components and steps:
- The Data Store houses the input data. Think of this as a bucket store such as AWS S3 or GCP’s Cloud Bucket which houses images.
- An image is pulled from the data store into Compute Node 1 in order to execute Task A.
- Task A breaks up the input image into many small patches.
- Those small image patches are transmitted to Compute Node 2 in order to execute Task B.
- Task B interprets each patch in turn, aggregates the result and outputs a numeric prediction.
Artera’s workflows are much more complex, but even this super simple workflow is surprisingly non-trivial to realize in a manner that respects data locality.
The 4-Step Challenge of Data Locality
Overview
When expanding into new regions, such as our recent partnership with GenesisCare in Australia, ensuring data locality isn’t just a technical preference, it’s a regulatory requirement. There are four critical components where any compliant architecture must implement Locality:
- Input Data Storage: The data used by our AI models must be stored within the country of origin. For Australia, this means all patient images and related information must reside locally.
- Compute Servers: It’s not enough to simply store data locally; the computation must also occur on local servers. This ensures that, when AI models process the data, all operations are contained within the jurisdiction.
- Input Data Transit: Even with local storage and compute, data can inadvertently leave the region during transit. The transit pathway must be confined to secure, private networks within the region rather than exposing data to the public Internet which poses risk of out-of-region transit.
- Artifacts: Finally, any intermediate or final outputs generated by the AI models must also remain within the designated region (e.g. Australia) at all times. This is achieved by ensuring that the Artifact storage and transit is local and secure.
Each of these steps is critical. Missing any one component could lead to inadvertent regulatory non-compliance. Our architectural design rigorously ensures that every step of the data journey, from ingestion to final output, we keep our operations within the designated geographic boundary.
Ground 0: Highly non-compliant
As an initial step, we will illustrate a non-compliant example that we will use as a starting point for the rest of this article. In this non-compliant scenario, the data is stored in a central location that is not necessarily within the geographic region of interest (e.g. Australia). This approach transfers sensitive data across borders during every step of processing, from the initial input data storage to intermediate transfers and final outputs, which is a clear no-starter.
With the above in mind, let’s dive right into the 4-step process to achieve Data Locality.
Step 1 — Localizing Input Data Storage
The journey to full compliance begins with ensuring that patient data is stored locally. In our Australian expansion, this involved migrating all input data into region-specific storage buckets. This critical move not only aligns with local regulatory requirements but also sets the stage for the subsequent steps — where compute resources, data transit, and artifacts are similarly localized. With the input data securely stored in Australia, we have taken the essential first step in building a fully compliant architecture that respects regional data boundaries.
Step 2 — Hosting Compute Resources Locally
With data stored securely within the region, our next move is to deploy compute clusters locally. Although this guarantees that processing happens on in-region servers, it isn’t sufficient on its own. There must also be strict controls to prevent any data from being inadvertently transferred out of the region during computation.
Step 3 — Securing Data Transit
Data movement is the most vulnerable part of any workflow but among the easiest to overlook! Even when storage and compute are local, if data transit occurs over public networks, the risk of breaching local residency requirements becomes real. To mitigate this, we employ regional S3 Gateway Endpoints and private networking protocols that ensure data remains within trusted boundaries as it moves between components.
Union.ai provides this architectural improvement out-of-the-box, not only ensuring Raw Data transfer locality, but also enhancing security and reducing S3 transfer costs in addition to ensuring locality compliance.
Step 4 — Keeping Artifacts In-Region
The final piece of the puzzle is ensuring that every artifact produced, whether a diagnostic result or a a processed input, remains stored and transmitted within the region. By hosting these artifacts in a region-specific S3 bucket and using secured transit channels, we close the loop on data locality compliance.
Putting it all together: How to scale the 4-Step process to Data Locality to multiple regions, all at once.
In modern distributed systems, the architecture is often divided into two distinct layers: the data plane and the control plane.
Data Plane: This layer is responsible for the actual processing (compute) and movement of data. It includes the components that store data, perform computations, and manage data transit. In our architecture, every region that we operate in essentially has its own data plane, a dedicated set of components ensuring that data stays local and compliant with regional regulations.
Control Plane: This is the central nervous system of a system that handles workflow orchestration, monitoring, logging, and access management. It makes decisions regarding task scheduling, error handling, and overall system configuration. Think of it as the command center that ensures all parts of the system work in harmony. Note that a control plane can be responsible for multiple Data Planes at the same time (and this is important!) .
When we introduced the 4-step Data Locality process, we focused primarily on the data plane components: local storage, compute resources, and data transit mechanisms that must strictly remain in-region. For instance, in Australia, the entire data pipeline is confined to local infrastructure to maintain data residency.
Traditionally, to ensure both data locality and effective operations across multiple regions, organizations might deploy a separate control plane for each region. However, this approach can quickly become unmanageable and expensive as the number of regions grows!
A Unified Control Plane for seamless region expansion
Union.ai’s platform has solved this problem for Artera, by enabling us to create a single global control plane, responsible for multiple region-localized Data Planes.
This centralized control system manages multiple regional data planes, one per region (e.g., USA, Australia, etc.). Each regional data plane is responsible for its local data operations, while the global control plane provides a unified interface for monitoring, orchestrating workflows, and managing access across all regions. This decoupling of the control and data planes not only simplifies system management, but also enhances scalability and compliance.
By maintaining one global control plane and segregating regional data operations into distinct data planes, we achieve a streamlined architecture that meets both operational efficiency and regulatory demands.
Key Benefits of the Global Control Plane for Artera
- Simplified Management: Our engineers gain a centralized dashboard to oversee all operations, regardless of where the data is processed.
- Effortless Scalability: Adding new regions no longer requires a full-scale deployment of a new control system; only the data plane needs to be localized, while the control plane remains global, managing existing and new Data Planes.
- Enhanced Reliability: A Control Plane typically needs a plethora of AWS services, some of which are not offered in all AWS regions. This created unpredictability for us when we initially operated with regional Control Planes, before a Unified Control Plane solution became available. With the new Global Control plane solution however, this has become a non-issue as Data Planes can be deployed nearly everywhere, and only a single Control Plane is ever needed. Note that all a Data Plane needs is EKS and S3, which are widely available services in nearly every AWS region.
- Streamlined Maintenance: Updates and configurations are managed from a single control point, simplifying maintenance for our team.
The Unified Control Plane solution allows Artera to focus on what we do best: developing and deploying AI solutions that improve cancer care globally, while ensuring compliance with data locality regulations.
Conclusion
Distribution of AI products globally while maintaining data locality isn’t trivial, but the four steps we’ve outlined will enable you to do it. Artera’s collaboration with Union.ai has realized a scalable approach to deploying clinical AI models on a global level. By designing an architecture that meticulously addresses every component of data locality, from input data storage to artifact management, we have achieved a solution that meets regulatory demands without sacrificing scalability or operational efficiency. This blueprint not only enhances our own clinical offerings but also serves as a valuable guide for any organization facing similar challenges in global AI deployment.
Acknowledgements: Thanks to Akinori Mitani and Hariharan Ananthakrishnan for providing thoughtful reviews and feedback.
