re:Invent Amazon introduced a new generation of SageMaker at the re:Invent conference in Las Vegas, bringing together analytics and AI, but with some confusion due to the variety of services that bear the SageMaker name.
SageMaker Unified Studio, now in preview, covers model development, data, analysis, and building generative AI applications.
However, there remains the old SageMaker, now renamed SageMaker AI, and which also has a separate studio from the new one – as well as a classic version still available. The difference is that SageMaker AI focuses more narrowly on building and training ML models. That said, SageMaker AI is also considered part of Unified Studio, as is Bedrock, a tool for building generative AI applications. Unified Studio can also be used programmatically, via the DataZone API.
Other Unified Studio features planned include access to streaming data such as Amazon Kinesis, integration with Amazon Quicksight business intelligence, and Open search search analytics (Amazon fork of Elasticsearch and Kibana).
According to G2 Krishnamoorthy, vice president of AWS Database Services, the heart of next-generation SageMaker is Lakehouse, a service featured here at re:Invent. “We have built an open, interoperable database that is very easy for customers to manage,” Krishnamoorthy told us.
SageMaker Lakehouse combines data from S3 and Redshift data lakes (AWS data warehouse) so that it can be queried with SQL as an Apache Iceberg database using tools such as AWS Athena or Apache Spark . Lakehouse also supports connections to DynamoDB, Google BigQuery, MySQL, PostgreSQL and Snowflake. Data can be imported or analyzed on site. Via Lakehouse and Unified Studio, the same data can be used for analytics as well as machine learning and generative AI application development.
Brian Ross, AWS Engineering Lead: Analytics Builder Experience, said in a session attended by The register: “Customers are saying their analytics workloads are increasing, their machine learning workloads are increasing, now their generative AI workloads are increasing and they’re starting to converge as well.”
The same data is used for analysis, training models, and creating knowledge bases for generative AI. “The big challenge with data is trying to find it. It’s somewhere within the organization, but where is it? How can I access it?” Ross said. He believes customers tended to build their own enterprise data platforms to solve this problem, using AWS services and tools, but this was costly while the new SageMaker offers “a unique end-to-end experience.” » supporting all these different uses.
SageMaker includes low-code/no-code tools, but it’s still aimed at what AWS calls “builders” rather than business users. These are geared toward Amazon Q Business apps and Amazon Quicksight dashboards, Krishnamoorthy told us.
SageMaker features demonstrated at re:Invent also include flexible training plans for HyperPod, a service introduced a year ago that manages the infrastructure for training models. Using flexible training plans, the user specifies the required accelerated computing resources as well as start and end deadlines. HyperPod will then offer you a detailed schedule and calculate the cost.
There appears to be a high demand for accelerated computing and re:Invent attendees were advised that using HyperPod is the best way to secure these resources, taking into account periods of lower usage.
Q Developer, Amazon’s AI assistant, is integrated into SageMaker Unified Studio. AWS also added Q Developer to SageMaker Canvas, a SageMaker AI tool for creating ML models, for a chat-based user interface for selecting a model type, uploading data, preparing data, testing, and deploy.
Pricing follows the typical AWS model. There is no charge for using SageMaker Unified Studio itself, but most actions consume other AWS resources which will be billed at their usual rate, although some have a free tier displayed on SageMaker. pricing page. There may be some risk that careless experimentation could result in a large bill.
Amazon SageMaker was introduced for the first time seven years ago as a service for data scientists and developers, part of the AWS Management Console. SageMaker offered a simple user interface to select training data, select a machine learning model, train the model, and deploy it to a cluster of Amazon EC2 instances.
Today’s SageMaker not only has more features, but its scope is expanded. The name can be confusing, as the overall SageMaker platform includes products that are also well known in their own right. Why is this all called SageMaker?
“The world of analytics and AI is coming together, so we thought it was appropriate for us to say that the new expanded SageMaker Platform is the product or suite of products for all data analytics and AI…so it’s the name confusion,” Krishnamoorthy said. “The alternative would have been to come up with a new name, like Microsoft did with Fabric, and then you’d have to teach everyone about all the components that are there.” ®
