|
Today we are announcing the next generation of Amazon SageMakera unified platform for data, analytics and AI. The all-new SageMaker includes virtually every component you need for data exploration, data preparation and integration, big data processing, fast SQL analysis, machine learning (ML) model development and training, and Generative AI application development.
The current Amazon SageMaker has been renamed to Amazon SageMaker AI. SageMaker AI is integrated into the next generation of SageMaker while also being available as a standalone service for those who want to focus specifically on building, training, and deploying large-scale AI and ML models.
Highlights of the new Amazon SageMaker
At the base is SageMaker Unified Studio (preview), a unique data and AI development environment. It brings together features and tools from the range of standalone “studios”, query editors and visual tools we have today in Amazon Athena, Amazon DME, AWS Glue, Amazon Redshift, Amazon Managed Workflows for Apache Airflow (MWAA), and the existing SageMakerStudio. We have also integrated Amazon Bedrock IDE (preview), an updated version of Amazon Bedrock Studio, for creating and customizing generative AI applications. Furthermore, Amazon Q provides AI assistance throughout your workflows in SageMaker.
Here is a list of key features:
- Amazon SageMaker Unified Studio (overview) – Create with all your data and analytics and AI tools in a single environment.
- Amazon SageMaker Lakehouse – Unify data across Amazon Simple Storage Service (Amazon S3) data lakes, Amazon Redshift data warehouses, and third-party and federated data sources with Amazon SageMaker Lakehouse.
- Data and AI governance – Securely discover, manage, and collaborate on data and AI with Amazon SageMaker Catalog, powered by Amazon data area.
- Computer science – Analyze, prepare, and integrate data for analytics and AI using open source frameworks on Amazon Athena, Amazon EMR, and AWS Glue.
- Model development – Create, train and deploy ML and foundation models (FM) with fully managed infrastructure, tools and workflows with Amazon SageMaker AI.
- Development of generative AI applications – Create and scale generative AI applications with Amazonian base.
- SQL Analysis – Get information with Amazon Redshiftthe most efficient SQL engine in terms of price.
In this article, I give you a quick overview of the new SageMaker Unified Studio experience and how to get started with data processing, model development, and generative AI application development.
Working with Amazon SageMaker Unified Studio (preview)
With SageMaker Unified Studio, you can discover and work with your data using familiar AWS tools to achieve end-to-end development workflows, including data analysis, data processing, model training, and creation of generative AI applications, in a single governed environment. .
A built-in SQL editor lets you query data from multiple sources, and a visual extract, transform, and load (ETL) tool simplifies creating data integration and transformation workflows. The new unified Jupyter notebooks enable seamless work across different compute services and clusters. With new built-in data catalog functionality, you can search, access, and query data and AI assets across your organization. Amazon Q is integrated to streamline tasks throughout the development lifecycle.
Let’s explore individual capabilities in more detail.
Computer science
SageMaker integrates with SageMaker Lakehouse and allows you to analyze, prepare, integrate and orchestrate your data into a unified experience. You can integrate and process data from various sources using the connectivity options provided.
Start by creating a project in SageMaker Unified Studio, choosing the SQL analysis Or data analysis and AI-ML model development project profile. Projects are a place to collaborate with your colleagues, share data, and use tools to work with data securely. Project profiles in SageMaker define the preconfigured set of resources and tools provided when you create a new project. In your project, choose Data in the left menu and start adding data sources.
The built-in SQL Query Editor lets you query your data stored in data lakes, data warehouses, databases, and applications directly in SageMaker Unified Studio. From the SageMaker Unified Studio top menu, select Build and choose Query Editor to start. Also try creating natural language SQL queries with Amazon Q while you’re at it.
You should also explore the built-in visual ETL tool to create data integration and transformation workflows using a drag-and-drop visual interface. From the top menu, select Build and choose Visual ETL flow to start.
If Amazon Q is enabled, you can also use generative AI to create feeds. Visual ETL comes with a wide range of data connectors, pre-built transformations, and features like data planning, monitoring, and previewing to streamline your data workflows.
Model development
SageMaker Unified Studio includes the capabilities of SageMaker AI, which provides infrastructure, tools, and workflows for the entire ML lifecycle. From the top menu, select Build to access tools for data preparation, model training, experiment tracking, pipeline creation, and orchestration. You can also use these tools for model deployment and inference, implementing machine learning operations (MLOps), model monitoring and evaluation, and governance and compliance.
To start developing your model, create a project in SageMaker Unified Studio using the data analysis and AI-ML model development project profile and explore the new Jupyter notebooks. From the top menu, select Build and choose Jupyter Lab. You can use the new unified notebooks to work seamlessly across different compute services and clusters. You can use these notebooks to switch between environments without leaving your workspace, streamlining your model development process.
You can also use Amazon Q Developer to help you with tasks like code generation, debugging, and optimization throughout your model development process.
Development of generative AI applications
Use the new Amazon Bedrock IDE to develop generative AI applications in Amazon SageMaker Unified Studio. The Amazon Bedrock IDE includes tools for building and customizing generative AI applications using FM and advanced features such as Amazon Bedrock Knowledge Bases, Amazonian bedrock railing, Agents of Amazonian BedrockAnd Amazonian bedrock flow to create tailor-made solutions aligned with your requirements and responsible AI guidelines.
Choose Discover in the top menu of SageMaker Unified Studio to browse Amazon Bedrock models or experiment with model playgrounds.
Create a project using the GenAI application development profile to start building generative AI applications. Choose Build in the top menu of SageMaker Unified Studio and select Chat Agent.
With the Amazon Bedrock IDE, you can create chat agents and build knowledge bases from your first-party data sources in just a few clicks, enabling Recovery Augmented Generation (RAG). You can add guardrails to promote safe interactions with AI and create features to integrate into any system. With built-in model evaluation capabilities, you can test and optimize the performance of your AI applications while collaborating with your team. Design flows for deterministic genAI-based workflows and, when ready, share your applications or prompts within the domain or export them for deployment anywhere, while maintaining control of your project and assets in your domain.
For a detailed description of all Amazon SageMaker features, see SageMaker Unified Studio User Guide.
To start
To begin using SageMaker Unified Studio, administrators must complete several configuration steps. This includes setting up AWS IAM Identity Centerby configuring the necessary virtual private cloud (VPC) and AWS Identity and Access Management (IAM) roles, creating a SageMaker domain, and enabling Amazon Q Developer Pro. Instead of IAM Identity Center, you can also configure SAML through IAM federation for user management.
Once the environment is configured, users log in via the SageMaker Unified Studio domain URL provided with single sign-on. You can create projects to collaborate with team members, choosing from pre-configured project profiles for different use cases. Each project connects to a Git repository for version control and includes a sample unified Jupyter notebook to help you get started.
For detailed setup instructions, see the SageMaker Unified Studio Administrator’s Guide.
Now available
The next generation of Amazon SageMaker is available today in the AWS US East (N. Virginia, Ohio), US West (Oregon), Asia Pacific (Tokyo), and Europe (Ireland) regions. Amazon SageMaker Unified Studio and Amazon Bedrock IDE are available in preview today in these AWS Regions. Check the full list of regions for future updates.
For pricing information, visit Amazon SageMaker Pricing And Amazon Bedrock Pricing. To learn more, visit Amazon SageMaker, SageMaker Unified StudioAnd Amazon Bedrock IDE.
Existing Amazon Bedrock Studio preview domains will be available until February 28, 2025, but you will not be able to create new workspaces. To experience the advanced features of Bedrock IDE, create a new SageMaker domain by following the instructions in Administrator’s Guide.
Try the new Amazon SageMaker in the console today and let us know what you think! Send your comments to AWS re:Post for Amazon SageMaker or through your usual AWS Support contacts.
— Antje
