Welcome to an introduction to Amazon SageMaker Neo. Hello, my name is Michelle Metzger and I'm a technical curriculum developer in the Training and Certification organization of AWS. In this video, I'll start by explaining some of the challenges that our customers are facing when performing Machine Learning. Then, introduce you to what Amazon SageMaker Neo is, and some of its benefits. I'll walk through a couple of use cases and wrap up with some key takeaways. First, let's begin by covering the concepts and challenges of Machine Learning, and how Amazon SageMaker Neo addresses some of these. Nowadays, Machine Learning is universally used by customers to make informed business decisions. Developers who want to use Machine Learning encounter hurdles at every step. First, they need to choose a framework that's best-suited for the task at hands. Second, they need to build models using their chosen framework. Third, they need to train a model using sample data to make accurate predictions on their data sets. Forth, they need to integrate their model with their application. Finally, they need to deploy the application, the model, and the framework on a platform. Amazon SageMaker Neo makes it easy to build Machine Learning models and get them ready for training by providing everything you need to quickly connect to your training data. It also helps you to select and optimize the best algorithm and framework for your application. However, developers still face hundreds of hurdles when they try to deploy a model with optimal performance on multiple platforms. Let's review a few details about the realities of model deployment. To begin with, developers need to have the specific version of the framework used for building the models installed on all of their chosen platforms. However, platform vendors often limits support to just one or two frameworks because of the cost and complexity of optimizing and installing those frameworks. Not to mention the size of the frameworks themselves, which limit the type of platform on which it can be installed. To work around these limitations, developers either restrict the model deployment to fewer platforms or worse, use a framework that can run on more platforms even if it's not the best-suited framework for the task at hand. In this situation, developers must manually tune the performance of their models on each platform, which requires rare skills and consumes a considerable amount of time. As a result, developers often have to settle for sub-optimal performance in their models. This sub-optimal performance underutilizes the platform resources, which increases the overall cost of the model's deployment. Ultimately, these hurdles restrict the diffusion of innovations in Machine Learning. Developers find themselves in a situation requiring their solutions to run on multiple platforms using multiple frameworks. A machine learning framework provides functions and operations that developers use as building blocks for their models. These functions and operations differ from framework to framework. Also, each framework has its own mechanism to save the models and often has unique file formats. At runtime, the framework reads the model to generate the code and runs that code on each unique platform. This means, that the model can only run on the framework on the platform in which it was built, and only on the hardware configuration on which the framework is supported. Do you see the problem here? It is painful to have all of the platforms, support all of the models from all of the frameworks. This solution is super messy, just like you see here. Amazon SageMaker Neo is designed to solve the messy problem I just showed you. It frees a model from the framework in which it was developed and allows it to make the best use of the hardware on which it was deployed. Now, let's take a look at how Neo works, and review some of its benefits. Amazon SageMaker Neo is a new SageMaker capability that helps developers take models trained in any framework, and port them on to any platform while increasing their performance and reducing their footprints. Neo provides model portability by converting models written in Apache MXNet, PyTorch, TensorFlow, and XGBoost, from a framework specific formats into a portable code. During conversion, Neo automatically optimizes the model to run up to two times faster without a noticeable loss in accuracy. After conversion, the model no longer needs the framework, which reduces its runtime footprint on the development platform by a 100 times. Customers can access Neo from the SageMaker console, and with just a few clicks, generate a model optimized for their specific platform. Neo contains two components; a compiler, and a runtime library. First, the Neo compiler reads models saved in various formats. Those formats used framework specific functions and operations. Neo converts those specific functions and operations, into non-specific functions and operations, often called Framework Agnostic Representations. Next, Neo will perform a series of optimizations. Now here's a high-level explanation of how Neo could work. First, the user submits the compilation job requests to Amazon SageMaker, and the job status is changed to starting. After a few moments, the job status will change to, in-progress. This is when the process of compiling the model really begins, this process can take a while. Now, during this time, the user has the ability to stop the job if they need to for any reason. When the user stops the job, the status will be changed to stopping, Amazon ECS will then begin the process of stopping the compilation. Once that is completed, the job status will then be changed to stop. Now in most cases, the user is not going to want to stop the job right in the middle, Amazon ECS will then be allowed to complete the compilation. Once this is completed, the job status will then be changed to complete. So what are the benefits of SageMaker Neo? Well, earlier I mentioned the burdens that developers carry when it comes to developing Agnostic Machine Learning models. The prior lack of simple-to-use profilers and compilers and Machine Learning had forced developers into a manual trial and error process, that is unreliable and quite frankly unproductive. Enter Amazon SageMaker Neo. Amazon sage Maker Neo provides developers with a simple easy to use tool to perform the conversion of framework specific models, and port them to any platform. Neo automatically optimizes the model to run up to two times faster, without a noticeable loss in accuracy. To optimize the model, Neo detects and exploits patterns in the input data, model computation, and platform hardware. After conversion, the model can run without needing any framework, which dramatically reduces the model's runtime footprint. This allows the model to run in resource constrained, edge devices, and performance-critical Cloud services. Let's take a look at a possible use case for a SageMaker Neo. Neo enables and accelerates Machine Learning models both in the Cloud and at the Edge. For mobile phone and IoT devices, Neo can help developers better deploy image classification, object detection, and anomaly detection applications by relaxing multiple constraints on those devices. In the Cloud, Neo compiles models to handle data stored in Amazon S3 buckets, or coming in as data streams. Neo also enables new use cases, such as the integration of Machine Learning with databases. Developers will be able to take Machine Learning models produced by Neo and run them in databases in the form of user-defined functions. These models will enable complex predictive queries that are currently impossible or unwieldy to express in ANSI standard SQL language. So what are the key takeaways from this video? Well, SageMaker Neo supports the most popular deep learning models, that power computer vision applications and the most popular decision tree models currently used in Amazon SageMaker. Neo optimizes the performance of AlexNet, ResNet, VGG, Inception, MobileNet, SqueezeNet, and DenseNet models, trained in MXNet, TensorFlow, and even PyTorch. It also optimizes the performance of classification and Random Cut Forest models trained in XGBoosts. Neo supports many different SageMaker instance types as well. It supports AWS DeepLens, Raspberry Pi, Jetson TX1 or TX2 devices, Amazon Greengrass devices, based on Intel processors, as well as in video Maxwell and Pascal GPUs. Developers can train models elsewhere, and use Neo to optimize them for SageMaker ML instances, or Greengrass supported devices. Neo provides model inference modules, which can run up to two times faster, while occupying a dramatically reduced memory footprint, versus previous solutions. You can use the Neo API to generate an optimized model for your desired platform, and you can even deploy and run that optimized model on your desired platforms at no additional charge. I hope you learn just a little something about Amazon SageMaker Neo, and we'll continue to explore other AWS videos. I'm Michelle Metzger with AWS Training and Certification. Thank you for watching.
