Alluxio Accelerates Deep Learning in Hybrid Cloud using Intel’s Analytics Zoo powered by oneAPI

Author profile picture

@bin-fanBin Fan

VP of Open Supply and Founding Member @Alluxio

This newsletter describes how Alluxio can boost up the educational of deep finding out fashions in a hybrid cloud surroundings when using Intel’s Analytics Zoo open supply platform, powered by oneAPI. Main points at the new structure and workflow, in addition to Alluxio’s efficiency advantages and benchmarks effects might be mentioned. The unique article will also be discovered on Alluxio’s Engineering Weblog.

Structure Evolution to Hybrid Mode

Historically, knowledge processing and analytics programs have been designed, constructed, and operated with compute and garage products and services as one monolithic platform, dwelling in an on-premises knowledge warehouse. Whilst easy to regulate and performant, this structure with deeply coupled garage and compute is ceaselessly difficult to offer programs elasticity and scale extra sources for one sort with out scaling the opposite. 

Extra customers are shifting in opposition to a hybrid type, combining sources from each cloud and on-premises environments. This type practices another structure to go away the information the place it is living, in most cases in the on-premises knowledge warehouse, however release a separate compute layer as wanted. The hybrid type permits compute and garage sources to be scaled independently, resulting in a lot of benefits:

  1. No useful resource rivalry: On-premise machines will also be absolutely applied by garage products and services as a result of there is not any festival for sources from compute products and services
  2. No compute downtime: There are not any idle compute sources as a result of clusters are introduced on call for in the cloud
  3. No knowledge duplication: Lengthy-running batch jobs or ephemeral ad-hoc queries can proportion the similar set of information with out making separate copies

Demanding situations to Ship Rapid I/O for Deep Learning

Even supposing the hybrid structure supplies flexibility and price benefits, there are further demanding situations for deep finding out analytics when coaching on large knowledge. Deep finding out coaching comes to a lot of trials of various neural community fashions and other hyper-parameters using the similar set of information. As well as, the scale of coaching datasets has been steadily rising. There’s a massive overhead price in loading all this information for each and every trial when the educational knowledge is saved in a faraway garage device.

A not unusual apply nowadays to regulate knowledge throughout hybrid environments is to replicate knowledge to a garage provider dwelling in the compute cluster prior to operating deep finding out jobs. Most often, customers use instructions like “distCP” to replicate knowledge from side to side between the on-premise and cloud environments. Whilst this seems to be simple, it in most cases calls for a handbook procedure which is sluggish and error-prone. 

To handle the I/O demanding situations of coaching deep finding out fashions in hybrid environments and leverage Intel’s oneAPI efficiency optimizations, we evolved and examined a brand new structure/workflow integrating Alluxio in the Analytics Zoo platform, powered by oneAPI.

What’s Analytics Zoo

Analytics Zoo is an open supply unified analytics and AI platform evolved by Intel to seamlessly unite a number of deep finding out programs into an built-in pipeline. Customers can transparently scale from operating pattern jobs on a computer to processing manufacturing scale large knowledge on massive clusters.

It helps:

  • Writing TensorFlow or PyTorch inline with Spark code for allotted coaching and inference
  • Local deep finding out (TensorFlow/Keras/PyTorch/BigDL) give a boost to in Spark ML Pipelines
  • At once operating Ray techniques on large knowledge clusters thru RayOnSpark.
  • Simple Java/Python APIs for (TensorFlow/PyTorch/BigDL/OpenVINO) Type Inference

What’s Alluxio

Alluxio is an open-source knowledge orchestration layer for knowledge analytics. It supplies top efficiency to knowledge analytics or gadget finding out programs like Analytics Zoo, serving as a allotted caching layer to forestall studying knowledge many times from faraway knowledge assets. In comparison to different answers, Alluxio supplies the next benefits in a hybrid cloud surroundings with “zero-copy burst” features to burst knowledge processing to the cloud:

  1. Compute-driven knowledge on-demand: When a garage device is fastened onto Alluxio, handiest its metadata is first of all loaded. Alluxio handiest caches knowledge as an application requests for it. This on call for habits permits burst knowledge processing to the cloud, getting rid of the wish to manually reproduction knowledge from an on-premise cluster to the cloud. 
  2. Knowledge Locality: Alluxio intelligently caches knowledge on the subject of programs, replicates scorching knowledge, or evicts stale knowledge in keeping with knowledge get entry to patterns. 
  3. Knowledge Elasticity: Alluxio will also be elastically scaled at the side of the analytics frameworks, together with container orchestrated environments. 
  4. Commonplace APIs for knowledge get entry to: Alluxio supplies knowledge abstraction with other not unusual APIs together with the HDFS API, S3 API, POSIX API and others. Current programs constructed for analytical and AI workloads can run at once in this knowledge with none adjustments to the application itself

Setup and Workflow

The next determine is the structure that integrates Alluxio with Analytics Zoo for quick and environment friendly deep finding out workloads:

On-premise or faraway knowledge shops are fastened onto Alluxio. Analytics Zoo application launches deep finding out coaching jobs by operating Spark jobs, loading knowledge from Alluxio in the course of the allotted report device interface. First of all, Alluxio has no longer cached any knowledge, so it retrieves it from the fastened knowledge retailer and serves it to the Analytics Zoo application whilst preserving a cached reproduction among its staff.

This primary trial will run at roughly the similar velocity as though the application used to be studying at once from the on-premise knowledge supply. In next trials, Alluxio can have a cached reproduction, so knowledge might be served at once from the Alluxio staff, getting rid of the faraway request to the on-premise knowledge retailer. Word that the caching procedure is clear to the person; there is not any handbook intervention had to load the information into Alluxio.

Alternatively, Alluxio does supply instructions like “distributedLoad” to preload the running dataset to heat the cache if desired. There could also be a “loose” command to reclaim the cache space for storing with out purging knowledge from underlying knowledge shops.

This phase summarizes Alluxio’s efficiency checking out and benchmark effects for the built-in workflow. 


We run experiments in a 7-node Spark cluster (1 example because the grasp node and the rest as employee nodes) deployed by AWS EMR. The benchmark workload is inception v1 coaching, using the ImageNet dataset saved in AWS S3 in the similar area. 

Because the baseline, the Spark cluster is at once getting access to the dataset from the S3 bucket. That is in comparison to a setup the place Alluxio is put in at the Spark cluster, with the S3 bucket fastened as its beneath filesystem.

The next desk main points the particular surroundings configurations:

Outcome Comparability

We measured knowledge loading efficiency when operating an inception coaching on ImageNet knowledge by using Analytics Zoo. The measured time comprises coaching knowledge and take a look at knowledge loading time.

The typical load time with and with out Alluxio is 579 and 369 seconds, respectively. That is roughly a 1.5x speedup when Analytics Zoo makes use of Alluxio for loading the ImageNet coaching and checking out knowledge. Word that, the enter knowledge is positioned in S3 in the similar area of the compute.

(Supply: Alluxio’s inside efficiency checking out)

The next determine displays that with Alluxio, variation in efficiency (15.9 2nd) could also be a lot less than the baseline variation (32.3 2nd). This means that Alluxio no longer handiest is helping the typical loading time but in addition makes efficiency extra constant.

By leveraging Alluxio as a knowledge layer on Analytics Zoo, the hybrid cloud resolution supplies acceleration of information loading in Analytics Zoo programs and deep finding out analytics on large knowledge programs. Our Alluxio’s inside efficiency benchmark checking out displays this structure is roughly a 1.5x speedup when Analytics Zoo makes use of Alluxio for loading the ImageNet coaching and checking out knowledge.

Persevered developments in synthetic intelligence programs have introduced deep finding out to the leading edge of a brand new era of information analytics building. There’s an expanding call for from organizations to use deep finding out applied sciences to their large knowledge research pipelines.

On behalf of all of the Alluxio open supply neighborhood, we inspire our readers to offer this resolution a take a look at, and invite you to invite questions in our neighborhood slack channel every time you come upon any problems.

Particular because of Intel’s Jennie Wang and Louie Tsai for his or her treasured Analytics Zoo’s technical session & give a boost to.



The Noonification banner

Subscribe to get your day by day round-up of most sensible tech tales!