- 95% reduction in Data Storage Expenses
- 40% to 68% Cost Savings
Overview of the Client
Our client is a global provider of electronics manufacturing services and solutions. They offer electronics design, production, and product management services to companies operating in different industries and markets such as automotive and transportation, cloud, computing and storage and many others. They operate in over 100 locations across 30 countries to ensure the smooth functioning of their business.
Business Challenges and Objectives
- The client manages operations across multiple AWS Accounts, using S3 Buckets with version control for historical and daily data.
- Current processes involve backend modifications and creating new versions, leading to increased storage costs and multiple copies.
- The client seeks an automated solution to eliminate extra versions without manual intervention, including an expiry date.
- Objectives include reducing storage costs, archiving less-used data, and enabling on-demand data retrieval.
- Certain S3 data becomes obsolete after 30 days, requiring an automated method for cleaning and cost reduction.
- Some S3 data with unpredictable usage needs immediate access; an automated approach is sought for cost-effective storage.
The Solution
-
S3 Lifecycle Rules:
-
Implemented rules to automatically delete older data versions after 30
days, aligning with policy for reduced storage costs.
-
Implemented rules to automatically delete older data versions after 30
-
Utilities with boto3 SDK (AWS SDK for Python):
-
Developed tools to:
- Archive S3 data at a specified prefix to S3 Glacier
-
Retrieve archived data and return it to the frequent access layer
for a specified duration, automatically re-archiving afterward,
simplifying data retrieval. -
Run periodic processes to delete unnecessary data post the specified
expiry period, contributing to storage cost reduction.
-
Developed tools to:
-
S3 Intelligent Tiering Rules:
-
Utilized S3 Intelligent Tiering rules to transition data to the
infrequent/Archive layer automatically.
-
Utilized S3 Intelligent Tiering rules to transition data to the
Business Outcomes and Benefits
- Implemented measures leading to a 95% reduction in data storage expenses.
- Used smart rules to cut costs further while ensuring data is ready when needed (40% to 68%).
- Set up rules and tools for S3 datalake, saving $150 – $200 per AWS account monthly with little impact on the business.
- Made it easy to get back archived data, improving overall business efficiency.
- Streamlined data management, cutting S3 storage costs without complications.
