AWS Unveils "Amazon S3 Files": Bridging Object Storage and High-Performance File Systems via NFS 4.1Amazon Web Services (AWS) has officially launched Amazon S3 Files, a transformative service that allows users to treat their S3 object buckets as a full-fledged file system. Supporting NFS 4.1, this new offering enables Linux servers and on-premises applications to mount S3 buckets directly, simplifying data access without complex API integrations.
The Architecture: Leveraging EFS for Millisecond Latency
While many Linux users have historically used tools like s3fs to mount buckets, S3 Files introduces a significant architectural upgrade. The service implements a high-performance caching layer powered by Amazon EFS (Elastic File System). By staging data in EFS, AWS can deliver sub-1ms latency, providing a much snappier and more responsive experience compared to traditional S3 mounting methods.
Seamless Bi-Directional Sync
AWS already offers Amazon FSx for network-attached storage (NAS) needs. However, the unique advantage of S3 Files is its bi-directional synchronization. Files written via NFS are automatically synced back to S3 as objects, allowing users to continue utilizing S3 APIs, lifecycle policies, and integration with other AWS services concurrently.
Pricing Model
The cost structure for S3 Files is split into three components:
Standard S3 Pricing: Based on total data stored in the bucket.
High-Performance Sync Storage: Fees for the EFS-based caching layer.
Data Operations: Throughput costs based on the volume of read and write activities.
A major problem with traditional S3 is the high latency when reading and writing large numbers of "small files." EFS-based caching helps easily migrate tasks like software builds, log processing, and legacy applications that require a traditional (POSIX) file system to the cloud without needing to modify any code.
This service is designed to truly address hybrid cloud needs. Organizations can connect their on-premises data centers to S3 Files via Direct Connect or VPN, allowing employees to manage files from on-premises servers like regular drives, while all data is backed up and processed immediately on the AWS cloud.
Data lakes aren't limited to raw data; S3 Files allows data scientists to directly access large datasets in S3 using Linux-based analytics tools via the filesystem interface, which is faster and more convenient than writing scripts to retrieve files individually via the SDK.
While there's an additional fee for the EFS layer, compared to running a large FSx system continuously, S3 Files is often more cost-effective for demanding tasks. "Elasticity" is key because the system scales high-performance storage based on actual usage (on-demand).
Greece to Ban Social Media for Under-15s
Source: AWS Blog
AWS Unveils "Amazon S3 Files": Bridging Object Storage and High-Performance File Systems via NFS 4.1Amazon Web Services (AWS) has officially launched Amazon S3 Files, a transformative service that allows users to treat their S3 object buckets as a full-fledged file system. Supporting NFS 4.1, this new offering enables Linux servers and on-premises applications to mount S3 buckets directly, simplifying data access without complex API integrations.
The Architecture: Leveraging EFS for Millisecond Latency
While many Linux users have historically used tools like s3fs to mount buckets, S3 Files introduces a significant architectural upgrade. The service implements a high-performance caching layer powered by Amazon EFS (Elastic File System). By staging data in EFS, AWS can deliver sub-1ms latency, providing a much snappier and more responsive experience compared to traditional S3 mounting methods.
Seamless Bi-Directional Sync
AWS already offers Amazon FSx for network-attached storage (NAS) needs. However, the unique advantage of S3 Files is its bi-directional synchronization. Files written via NFS are automatically synced back to S3 as objects, allowing users to continue utilizing S3 APIs, lifecycle policies, and integration with other AWS services concurrently.
Pricing Model
The cost structure for S3 Files is split into three components:
Standard S3 Pricing: Based on total data stored in the bucket.
High-Performance Sync Storage: Fees for the EFS-based caching layer.
Data Operations: Throughput costs based on the volume of read and write activities.
A major problem with traditional S3 is the high latency when reading and writing large numbers of "small files." EFS-based caching helps easily migrate tasks like software builds, log processing, and legacy applications that require a traditional (POSIX) file system to the cloud without needing to modify any code.
This service is designed to truly address hybrid cloud needs. Organizations can connect their on-premises data centers to S3 Files via Direct Connect or VPN, allowing employees to manage files from on-premises servers like regular drives, while all data is backed up and processed immediately on the AWS cloud.
Data lakes aren't limited to raw data; S3 Files allows data scientists to directly access large datasets in S3 using Linux-based analytics tools via the filesystem interface, which is faster and more convenient than writing scripts to retrieve files individually via the SDK.
While there's an additional fee for the EFS layer, compared to running a large FSx system continuously, S3 Files is often more cost-effective for demanding tasks. "Elasticity" is key because the system scales high-performance storage based on actual usage (on-demand).
Greece to Ban Social Media for Under-15s
Source: AWS Blog
Comments
Post a Comment