[ad_1]
Today, we’re excited to announce the overall availability of Azure Managed Lustre, a managed excessive efficiency parallel file system for top efficiency computing (HPC) and AI workloads. Azure Managed Lustre delivers the time-tested Lustre file system as a primary get together managed service on Azure. Long time customers of Lustre on-premises can now leverage the advantages of a whole HPC answer, together with compute and excessive efficiency storage, delivered on Azure. Users profit from the excessive throughput and efficiency capabilities of Lustre as a seamless managed expertise and might now deal with their enterprise imperatives.
Need for Azure Managed Lustre
Lustre, some of the common distributed parallel filesystems within the HPC world has lengthy been deployed on-premises serving scalable and excessive throughput storage wants of HPC workloads. As an open-source answer, it has loved a thriving ecosystem of customers and builders. With the current explosive progress seen in generative AI and the necessity for a excessive throughput storage that retains the costly GPU cores from ready on information, Lustre has discovered a renewed progress in its adoption. Customers in industries leveraging excessive efficiency computing are searching for a seamless end-to-end infrastructure encompassing compute, storage, and networking that’s each quick to deploy and simple to make use of.
Azure Managed Lustre allows:
- A managed PaaS expertise for worry-free deployment and consumption of Lustre file system.
- Reduced TCO with pay-as-you-go pricing.
- Throughput choices at completely different pricing ranges for various workloads.
- Easy deployment of unbiased clusters with predictable efficiency, eliminating the noisy neighbor downside generally skilled in on-premises Lustre deployments.
- Quick deployment in a couple of minutes, assuaging the necessity to plan months or years upfront and assembly compliance and information residency necessities.
- Region of alternative for deployments throughout many Azure areas, supporting compliance and information residency necessities.
- Scalability to tons of of petabytes (PB), with the related sturdiness, availability and value advantages of tiering to Azure Blob storage.
- Seamless integration with current Azure compute companies together with Azure HPC companies, Azure Machine Learning and Azure Kubernetes service.
- Investment safety for patrons’ current automation and platform growth when migrating from on-premises Lustre deployments.
Today, two persistent sturdy situations can be found for patrons to make use of. These are primarily based on strong state drives (SSDs) that are differentiated by their efficiency choice delivered for provisioned Tebibyte of capability.
- AMLFS Standard – 125 MB/s
- AMLFS Premium – 250 MB/s
Users of Azure Managed Lustre can obtain the Lustre shopper packages from packages.microsoft.com for his or her desired Linux distribution and kernel model. Additionally, we may have help for HPC pictures prebuilt with Lustre shopper packages for Ubuntu – 18.04, 20.04, 22.04, and Alma 8.7.
Azure Managed Lustre is out there at the moment from the Azure portal in a number of areas. To maintain updated on what areas you should use, please seek advice from our merchandise obtainable by area web page. To be taught extra about pricing, please go to our pricing web page.
Customers are already seeing outcomes with Azure Managed Lustre
Azure Managed Lustre serves numerous companies and organizations who’ve the necessity for top throughput storage entry inside vitality, life sciences, meteorology, automotive, healthcare, and public sector industries.
Azure Managed Lustre has you coated whether or not you might be operating instruments to simulate 3D product fashions to speed up time-to-market, serving 1000’s of customers querying monetary info on information analytics platforms, or constructing the subsequent AI supercomputer to your generative AI answer.
When Petronas, a world chief in vitality, wanted to speed up their upstream vitality analysis with extremely advanced quantitative interpretation of seismic processing workloads that want giant information units to be fed to the compute VMs with very excessive throughput, they selected Azure Managed Lustre.
“Azure Managed Lustre is an outstanding file system that has been specifically crafted to meet the unique requirements of PETRONAS’ high performance computing (HPC) and AI workloads. It delivers exceptional performance, providing high storage bandwidth and seamlessly integrating with essential Azure services.”—Faizal B Zainol Alam, PETRONAS Digital General Manager
Meteomatics, a frontrunner and disruptor in democratizing climate intelligence wanted a excessive efficiency file system for his or her climate modeling and simulations, they picked Azure Managed Lustre for his or her wants. Azure Managed Lustre’s integration with Azure CycleCloud allows Meteomatics to deploy their HPC clusters rapidly and programmatically permitting them to ship worth to their finish clients searching for quick and constant entry to climate information.
“At Meteomatics, we depend on excessive efficiency computing (HPC) to run advanced numerical climate prediction fashions, delivering correct forecasts to our clients. By leveraging Lustre, a trusted file system within the HPC neighborhood, and the seamless deployment of LustreFS, we acquire the excessive variety of IOPS and throughput essential for dealing with huge information volumes. Azure Managed Lustre accelerates our climate forecast simulations with a excessive spatial and temporal decision, considerably lowering time-to-completion and enabling quicker supply of forecast information to our valued clients.“—Dr. Martin Fengler, CEO Meteomatics
“Duke Clinical Research Institute was searching for a scalable answer for his or her excessive I/O must help their analytics instruments serving medical analysis. Azure Managed Lustre addressed that want with a easy deployment mannequin permitting them to deal with offering worth to their clients rapidly. I see the easy simplicity and energy of AMLFS as the last word alternative for high-speed shared storage, empowering your analytics instruments with unequalled simplicity and flexibility for each rapid and long-term wants.“—Emilio Power, Research Analytics Lead, SAS System Architect, Duke Clinical Research Institute
Using Azure Managed Lustre
Azure Managed Lustre is delivered in a hosted-on-behalf-of subscription, by means of a easy interface inside the buyer digital community. Customers don’t should be involved in regards to the internals of deploying, managing, and working the Lustre file system together with metadata servers/targets (MDS/MDT), administration servers/targets (MGS/MGT) and object storage servers/targets (OSS/OSTs).
The Azure portal presents a easy interface to configure and deploy Azure Managed Lustre, the place clients present some inputs like useful resource group id, subnet ID, cluster dimension, and desired efficiency choices. Users have the flexibility to observe and observe their cluster efficiency with throughput, latency, and IOPS metrics.
Hierarchical storage administration performance in Azure Managed Lustre allows customers to seamlessly switch information between Azure Managed Lustre and Azure Blob. This empowers customers to ascertain information archival, retention, and safety methods. When coping with giant datasets, customers can import solely the info that’s presently lively and essential for his or her information processing duties into Azure Managed Lustre clusters, whereas archiving and retaining the remaining information in Azure Blob. This method successfully minimizes runtime prices. Furthermore, by using information tiering to Azure Blob, customers can make the most of its world presence and availability to create Azure Managed Lustre clusters in a number of areas as wanted. The integration with Azure Blob additionally facilitates the accessibility of knowledge by means of a number of protocols, corresponding to community file programs (NFS), hadoop distributed file system (HDFS), and REST.
We are additionally very excited to announce the overall availability of the CSI driver for Azure Managed Lustre. Now, containerized purposes operating on Azure Kubernetes Service can simply devour Azure Managed Lustre.
Next steps
Learn extra about how you can use Azure Managed Lustre and its numerous supported options from our documentation.