Reading Notes: "Large-scale Distributed Storage Systems: Principles and Architecture in Action": VIII
This article was last updated on: February 7, 2024 pm
8 Cloud storage
8.1 Concepts
Typical features of cloud storage:
- Hyperscale (up to TB or even on EB capacity)
- High scalability (smooth scale-out possible)
- High reliability and availability (taking public cloud provider Tencent Cloud as an example, the SLA of its cloud storage COS is: data durability 99.99999999999% (12 9s) service availability 99.995%)
- Security (in public cloud, Internet, and multi-tenant scenarios, security is a basic requirement)
- On-demand services (public cloud storage features, on-demand services, pay-as-you-go)
- Transparent service
- Low cost (running on a large number of low-cost x86 servers)
8.2 Product Form
Product forms include several categories:
- Block storage
- Object storage (e.g., Amazon S3)
- File system storage
Among them, Amazon S3 is the first of its kind in the Amazon public cloud and has become the de facto object storage standard, and its application programming interface is as follows:
- List Bucket: Lists buckets
- Put Bucket: Create a bucket
- Delete Bucket: Delete the bucket
- Head Bucket: Determine whether the bucket exists and has access permissions
- Put Object: Create an object and add it to the bucket or modify an existing object. Optionally enable the object multi-version policy.
- Get Object: Read the data and metadata of the object, including the object length, MD5 hash value, creation time, etc
- Delete Object(s): Delete the object
- Head Object: Gets the metadata of the object
- Initial Multipart Upload: Initialize multiple uploads and get the upload ID of multiple uploads
- Upload Part: Upload some data
- Complete Multipart Upload: After multiple uploads, S3 will join part of the previously uploaded data into one large object
- Abort Multipart Upload: Abort multiple upload requests.
8.3 Core Benefits of Cloud Storage
Cost, cost, or TMD cost!
The following table compares the costs of medium data centers and very large data centers, and the network and storage costs of very large data centers are 1/7.1 and 1/5.7 of medium data centers, respectively.
| Category | Midsize Data Center Costs | Extralarge Data Center Costs | Ratio |
|---|---|---|---|
| Network | $95/Mb/Month | $13/Mb/Month | 7.1 |
| Storage | $2.20/GB/Month | $0.40/GB/Month | 5.7 |
| Manage | 140 servers/admin | More than 1000 servers/administrators | 7.1 |
PUE (Power Usage Effectiveness). The PUE > 2 of general domestic medium-sized data centers means that more than half of the energy is wasted, while the PUE value of a solar-powered data center in a very large data center, such as Fackebook, is 1.07, with almost no additional energy consumption.
Significantly reduced IT costs and high resource utilization
Cloud storage core advantages:
- Lower hardware and network costs
- Lower management costs and electricity costs
- Higher resource utilization
Innovate faster
8.4 Cloud Storage Security
Security challenges of cloud storage:
- There has been a huge change in the boundaries of trust
- More stakeholders (cloud consumers, cloud providers, cloud auditors, cloud brokers, cloud operators)
- Cloud storage services are exposed to the internet
- Ingestion of multi-tenant sharing
- Data storage
Cloud Storage Security Classification:
- User security: Authentication, authorization
- Cybersecurity: Secure communication, network firewall, intrusion detection, DDoS attack protection
- Multi-tenant isolation
- Storage security: Data backup and data security