Amazon HealthLake — Does Generally Available mean Enterprise Ready?
Overview
I previously wrote a comparison about Google Cloud’s Healthcare API offering and AWS’s HealthLake offering here. When I did the initial comparison AWS had not released their product for general availability so I did not include the documented limitations for the product. Now that it has been GA-ed, I wanted to add onto the previous publication with usability and scale considerations with AWS HealthLake. The homepage claims to support use cases such as, “manage population health, improve care quality, optimize hospital efficiency.” Upon further evaluation these use cases are not feasible using the out of the box product, as documented in this post.
NOTE: Most of the issues discussed in this blog are based on AWS’ limitations documentation. AWS states that users “can request increases for some quotas, and other quotas cannot be increased.” I attempted to understand which of the HealthLake quotas can be raised by viewing the Service Quotas console, however, none of the HealthLake quotas are listed. I then attempted to file a support ticket, however, HealthLake is not a listed supported product within the service limit dashboard (screenshot below). As far as I can tell, users cannot currently request quota increases to HealthLake. I could not determine what quotas (if any) are able to be raised nor how users could increase quota if available.
Real-World Use Cases with AWS HealthLake
In order to understand the scale issues with HealthLake, let’s start by creating an example health system to base assumptions off of. A typical clinical organization has at least one electronic health record (EHR) system, or often times many, EHRs through out their organization. A common use case systems are implementing is to leverage real-time EHR data to feed machine learning models — shown below.
In total these EHRs can send a few million HL7v2 messages a day. The average number of HL7v2 messages per day is up for debate — OrionHealth quotes 50 million per day here and InterSystems benchmarks at over a billion per day here. In my experience both of those rates are higher than what we see in the wild. Something more practical, for example, is the scale iNTERFACEWARE tests that “closely mimic(s) real world situations” at around 8.3 million inbound messages a day.
Now, let’s assume you’re a small system — your EHR may only produce ~1 million HL7 messages a day (~10/sec). This is a low estimate as it assumes the same rate throughout the day instead of accounting for the ebbs and flows of patient care.
Based on the above, AWS must support ingestion of at least 10 HL7v2 messages per second-for a small hospital system to stream their patient data to HealthLake. Using the defined architecture (above) for streaming EHR data — we need to convert the HL7v2 messages to FHIR resources. A single patient admit message (ADT01) creates 15 FHIR resources based on the open source standards.
This means a small hospital system would create around 150 FHIR resources per second. This is well beyond the AWS HealthLake’s documented limitations for ingestion.
HealthLake’s ResourceCreate operation scales to support 20 resources/sec. Our fictional organization would have to split the streaming pipeline into 8 different FHIR stores to ingest their data — with absolutely no read or deletes allowed.
If our organization chose to use the update operation instead of create, HealthLake’s ResourceUpdate operation scales to support 100 transactions/sec. This would require us to have at least 2 FHIR stores to support our streaming data use case.
You might ask, so what? The issue with multiple FHIR stores is that there is no simple way to aggregate FHIR data from multiple stores. Read: Users cannot easily aggregate patient data if it is stored in multiple FHIR stores. FHIR does not have specs for federated searches and it is difficult to create a single facade over multiple databases as highlighted in this FHIR DevDays talk.
TL;DR — there is no way for an organization to meaningfully use all of their streaming FHIR data in AWS HealthLake today.
So instead, let’s suppose we are only batch loading our data. In Synthea, a realistic synthetic FHIR generator, one patient record is ~546 FHIR resources. AWS’s HealthLake states to only support 500,000 FHIR resources per store. This means each FHIR store only supports 915 patient records.
Therefore, in order to do analytics or searches over more than 900 patients — a user must export the data (for a fee) and send it to a different AWS product. The only supported product integration today is for S3 storage (not an analytics product). Read: analytics requires custom code and pipelines.
HealthLake does not support streaming export, and in fact, the documentation says,
“Only one import or export job can run at time for an active Data Store.” — AWS HealthLake Developer Guide
This means a user cannot do real-time analytics over a HealthLake Data Store. The export capabilities is bottlenecked by a single operation — and that single operation has a limit of one per minute.
Finally — assuming 915 patient records per store, and with a quota of ten stores per account, a single AWS account can only support storage for ~9,150 patients.
A traditional health system that sees around one million patients a year would need a 100x the scale of AWS HealthLake to support their system for a single year.
These product limitations inhibit the very use cases their documentation claims to support. The website states HealthLake can: “manage population health, improve care quality, and optimize hospital efficiency.” Let’s dig.
The first blog on population health states AWS uses the MIMIC-III dataset that has over 40,000 patients. Upon further reading, the actual number of patients ingested into the AWS FHIR Data Store was 1,001 patients (screenshot below) — far less than MIMIC’s 40k patients.
In AWS’ blog for improving quality care using machine learning, the examples also were capped at 1,000 patients (“500 patients with a positive CHF diagnosis…500 patients who didn’t have a CHF diagnosis”).
Summary
I’m hoping AWS provides more support, documentation, or use cases that demonstrate their product in a real world scenario. Based on my research, the current state is not a feasible option for many enterprise customers today.
*This is not discussing the need for reads and searches — most business use cases in FHIR include high-transaction support — such as precision medicine, operational efficiencies, and SMART-on-FHIR applications. Read operations are also throttled to 20 resources per second, even less if users are also running create or delete operations on the store.
Note from the Author
While this is a review of the AWS limits with FHIR, it should not be lost on the reader that there is a bigger theme at play — FHIR at scale. Right now, the community has little solution for supporting use cases that require FHIR data that is stored in different databases. Until we can easily solve this by building a scalable facade or creating a federated search model, users must be considering the scale of their FHIR solution. I hope this article helps users begin to understand the consequences that can come from not appropriately scoping a solution prior to implementation. Just like we do with any other database and server on the market today, I propose we start bringing scalability into the FHIR conversation.
