AWS Transforms Amazon SageMaker into a Single Platform for AI and Data Analytics

Amazon Web Services Inc. popular neural network development platform Amazon SageMaker is getting a major refresh, with a host of new features that will support the integration of faster structured query language analytics, petabyte-scale data processing, and much more.

The new updates, announced at the annual AWS extravaganza re:Inventare designed to transform Amazon SageMaker into a more comprehensive, fully integrated platform for artificial intelligence development.

They include the new SageMaker Unified Studio, which is a portal through which customers can access data from across their organization, as well as various AI and machine learning development tools, and the SageMaker Catalog, which hosts a collection of large, powerful language models and other development tools. artifacts.

Meanwhile, SageMaker is getting its own data platform, called SageMaker Lakehouse, which unifies data from multiple data lakes, warehouses and operational databases and applications, making it easier for developers to access.

The company launched Amazon SageMaker in 2017, well before the current AI development craze inspired by OpenAI’s ChatGPT, and has since become the leading AI application development platform for the technology giant. cloud infrastructure. It accommodates a glut of tools for creating AI applicationsallowing you to create neural networks, train them, deploy and monitor their performance, fine-tune them, and perform a variety of other essential tasks all in one place

All AI development tools in one place

SageMaker Unified Studio is now available in preview and represents a major evolution of the platform, giving users access to a simplified environment through which they can access all their data and use it in AI systems. It unifies all the tools found in Amazon’s previously disparate ecosystem of developer studios, query editors, and visual tools found in platforms such as Amazon Bedrock, Amazon EMR, Amazon Redshift, AWS Glue and the existing SageMaker Studio, the company said. It’s about making everything easier to access, so users can discover and prepare data, create queries and code, and build AI models all in one place.

In addition to having everything in one place, developers will also benefit from Amazon Q Developer, an AI-powered assistant that can make data discovery easier. They can ask questions like what data they should look at to get a better idea of ​​their organization’s product sales and get the answers they need immediately, AWS said.

Amazon Bedrock’s integrated development environment is also integrated with SageMaker Unified Studio, allowing you to create AI applications using a large library of high-performance base models, as well as various AI agents. AI, knowledge bases, guardrails and predefined workflows.

The company said it was integrating everything in response to how customers use SageMaker today. He said he’s seen how most users also leverage its data analysis tools to support the tasks they do with SageMaker. It therefore makes sense to group them all under one cover to allow easier access.

A giant data repository

As for the SageMaker Catalog, it is built on Amazon DataZone and provides access to hundreds of approved AI models as well as protections such as granular access controls and AI guardrails. They can prevent AI applications from exhibiting toxic or biased behavior.

With the arrival of SageMaker Lakehouse, SageMaker users have the ability to centralize access to the underlying data resources that power their AI models, as well as analytics capabilities. One benefit of this setup is that it makes it easier to combine data from multiple sources, such as Amazon S3 data lakes, Redshift data warehouses, or other federated data sources. SageMaker Lakehouse itself is accessible through SageMaker Unified Studio.

It also makes it easier for users to query data because SageMaker Lakehouse is compatible with the Apache Iceberg open data standard, meaning the information it contains can be explored with various SQL analysis tools.

Pharmaceutical giant F. Hoffmann-La Roche AG used SageMaker Lakehouse in early access and claims to have successfully eliminated data silos and made information easier to access, without any complicated data transfer procedures. As a result, data processing times were reduced by 40%.

Easier app integrations

As it strives to make developers’ lives easier, AWS is also announcing what it says are “zero ETL integrations” with various third-party software-as-a-service applications. For example, customers can integrate Amazon Aurora MySQL, PostgreSQL, and other database types directly with Amazon SageMaker, without the need to first create complex data pipelines.

Zero ETL integrations eliminate the complex “extract, transact, and load” process traditionally required to change the format of data found in one application to meet the requirements of another. Essentially, AWS does all of this itself, and that’s a huge advantage because building data pipelines can be a time-consuming and error-prone process that creates major headaches for even the largest organizations.

Swami Sivasubramanian, AWS vice president of data and AI, said the convergence of AI and data analytics means businesses rely on “increasingly interconnected” data sources, hence the need to make all this information more accessible.

“Many customers are already using a combination of our purpose-built analytics and machine learning tools, such as Amazon SageMaker, Amazon EMR, Amazon Redshift, and more. “, he said. “The next generation of SageMaker brings together these capabilities, along with exciting new features, to give customers all the tools they need for data processing, SQL analysis, model development and training, right within of it.

Image: AWS