How to Transfer Petabytes Without Going Broke

The Scale Problem

A petabyte is 1,000 terabytes or roughly one million gigabytes. In concrete terms: about 500,000 hours of HD video, 20 billion document pages, or a single large AI training dataset. As AI workloads grow, petabyte-scale transfers are becoming routine for AI data centers, research institutions, and media companies.

The problem is that most file transfer pricing was designed for gigabyte-scale usage. At petabyte scale, those per-GB fees become absurd. Let us do the math.

Cost Breakdown: Moving 1 PB

Method 1: Cloud Egress ($50K-$90K)

If your data is already in a cloud provider, egress fees are the most straightforward cost. The pricing is per-GB, tiered by volume:

Azure is typically 30-40% cheaper for egress. GCP falls in between. But all three share the same fundamental problem: data gravity. Once your data is in one cloud, the egress fee is effectively a tax on leaving.

When cloud egress makes sense

When data is already in the cloud and needs to move to another cloud region or on-premises location. You are paying the egress whether you use a transfer tool or not. The transfer tool cost is additive.

Method 2: Pay-Per-GB Service ($250,000)

Pay-per-GB services charge $0.25/GB for downloads. The math is simple and painful:

These services also use TCP-based transfer, which means throughput is constrained by the bandwidth-delay product on high-latency links. A 10 Gbps connection between US coasts might achieve 2-5 Gbps effective throughput, extending transfer time from 12 days to 20-30 days. See our TCP limitations analysis for the full breakdown.

When pay-per-GB services make sense

Pay-per-GB services are designed for project-based transfers in the gigabyte to low-terabyte range: video deliverables, design assets, client uploads. At that scale, $0.25/GB is reasonable for the convenience. They are not designed for petabyte-scale data movement.

Method 3: Enterprise UDP Tools ($10K-$30K/year)

Enterprise UDP transfer tools use the FASP protocol for UDP-based acceleration, achieving near line-rate throughput regardless of latency. At 10 Gbps, 1 PB transfers in roughly 12 days.

The cost model is a flat annual license ($10,000-$30,000 depending on features and throughput tiers) plus the infrastructure to run dedicated transfer servers. The infrastructure cost varies widely: a single on-prem server might cost $5,000-$10,000, while a cloud-hosted deployment can run $2,000-$5,000/month.

When enterprise UDP tools make sense

When you are an enterprise with dedicated IT staff, need formal compliance certifications (SOC 2, HIPAA), and move petabytes regularly. The fixed cost amortizes well if you are transferring 10+ PB/year. For a one-time petabyte move, the setup overhead is hard to justify.

Method 4: AWS Snowball ($9K-$15K)

AWS Snowball Edge devices hold 80 TB of usable capacity. To move 1 PB, you need 13 devices. Each device costs $300 for a 10-day rental plus shipping.

The catch: Snowball is designed for getting data into AWS, not for arbitrary point-to-point transfer. It also takes 5-10 business days for the full cycle (ship, load, ship back, ingest). For data that needs to reach a non-AWS destination, Snowball adds complexity.

When Snowball makes sense

One-time bulk migration into AWS with limited network bandwidth. If your internet connection is under 1 Gbps, Snowball is faster than network transfer for anything over about 50 TB.

Method 5: Shipping Hard Drives ($3K-$6K)

The original sneakernet. Buy 20 TB drives, load them, ship them.

This is genuinely the cheapest option for a one-time bulk transfer when you do not have high-bandwidth connectivity. A box of drives shipped next-day is faster than a 1 Gbps connection for anything over about 100 TB (100 TB over 1 Gbps takes ~9.3 days).

When drive shipping makes sense

Initial bulk data migration with limited bandwidth. This is an honest recommendation: if you need to move 1 PB once and your connection is under 5 Gbps, shipping drives is likely your best option for the first transfer. Use network-based tools for ongoing incremental updates afterward.

Method 6: Handrive ($0)

Handrive is free at any scale. No per-GB fees, no license costs. The transfer uses a UDP-based protocol that achieves full bandwidth utilization regardless of latency, similar to the FASP protocol but without the license fee.

Transfer time depends entirely on your network bandwidth:

The trade-off: both endpoints must be online simultaneously (no intermediate storage server). For a multi-day transfer, this means running a headless server instance on a NAS or always-on machine. The data is end-to-end encrypted and never touches a third-party server, which matters for proprietary AI training datasets and model checkpoints.

When Handrive makes sense

Ongoing transfers between fixed endpoints with reasonable bandwidth (5+ Gbps). The zero cost means you can run continuous data pipelines without worrying about the bill. For AI data centers with 10-100 Gbps interconnects, Handrive delivers enterprise-class performance at zero cost.

The Honest Recommendation

There is no single best method. The right choice depends on your specific situation:

• One-time bulk migration, limited bandwidth: Ship drives. It is unglamorous but it works and it is cheap.
• One-time migration into AWS specifically: AWS Snowball. Purpose-built for this.
• Ongoing transfers, enterprise compliance required: Enterprise UDP tools. The license pays for itself if you are moving 10+ PB/year.
• Ongoing transfers, cost-sensitive: Handrive. Zero cost at any volume with full-speed UDP protocol.
• Ad-hoc project transfers under 5 TB: Pay-per-GB services. Simple, no setup, per-GB cost is manageable at this scale.

Many teams combine methods: ship drives for the initial bulk load, then use Handrive for ongoing incremental transfers. This hybrid approach minimizes both cost and time to first data availability.

For a detailed comparison of the network-based options, see our AI data transfer tools comparison.

Related Posts

• The Data Transfer Crisis in AI Infrastructure
• File Transfer for AI Training Data
• Why TCP Fails for AI Data Transfer