A Solution for Slow Transfers to a Local Server

Preface: A Hopeless 50KB/s

Recently I was deploying a project to a remote Linux server and ran into an utterly absurd network environment:

1. The files were huge: code plus data packages came to a full 6.7GB.
2. The network was terrible: Ping between my local Mac and the server was as high as 700ms+, accompanied by severe packet loss.
3. The speed was something else: I tried transferring directly with scp or rsync, and the speed held steady at 40KB/s… at that rate the transfer would take an estimated 40 hours, and it could disconnect at any moment.

But while troubleshooting, I found that the server’s own downlink bandwidth was actually not bad (Speedtest measured 70Mbps). The bottleneck was the congested link “between me and the server”, which caused the TCP sliding window to collapse and kept the speed from rising.

Approach: Since a direct connection doesn’t work, find a “relay station.” Use Baidu Netdisk (together with SVIP) as a high-speed bridge, and on the Linux server use command-line tools to download at full bandwidth.

This article introduces the ultimate solution: multi-threaded downloading with bypy + aria2.

Advantages of This Approach

• Fast: uses Baidu Netdisk’s CDN and the server’s own bandwidth, avoiding point-to-point congestion. But you need a Baidu Netdisk membership to enjoy high-speed downloads.

• Stable: aria2 supports multi-threaded concurrency and resuming interrupted downloads, so it can handle large files.

The Full Workflow

Step 1: Compress and Upload the File Locally

Compression: First compress the project folder into a zip archive. You can run this in the terminal:

zip -r project.zip project_folder

Upload: Open the Baidu Netdisk client, upload project.zip to the /apps/bypy/ directory in the netdisk, so that later steps are easier.

Step 2: Prepare the Server Environment

Log in to your remote Linux server. We need to install two tools:

1. bypy: a Python client for Baidu Netdisk.
2. aria2: the multi-threaded download workhorse (critical! without it, large files will stall).

If you have a conda environment (the terminal shows (base)), installation is very simple:

# 1. 安装 bypy
pip install bypy

# 2. 安装 aria2
conda install -c conda-forge aria2
# 或者
apt-get install aria2  # 如果你有 root 权限

Verify the installation: Enter aria2c -v. If you can see a version number, the installation succeeded.

Step 3: Link Your Baidu Netdisk Account

In the server terminal, enter:

bypy info

1. The terminal will display a long link.
2. Copy this link and open it in your local browser.
3. Log in to your Baidu account and copy the authorization code shown on the page.
4. Go back to the server terminal, paste the authorization code, and press Enter.

Once capacity information is displayed, the account is linked successfully.

Step 4: Start Multi-Threaded High-Speed Downloading (the Core Step!)

bypy uses Python single-threaded downloading by default, and with large files of several GB it often “starts fast, then quickly loses speed until it stalls (stuck at 0%)”.

You must hook in aria2 for multi-threaded fetching.

Run the following command:

# --downloader aria2: 指定使用 aria2 下载器
# --processes 4: 开启 4 线程并发（跑满带宽）
# --timeout 600: 增加超时容忍度，防止网络波动中断

bypy --downloader aria2 --processes 4 --timeout 600 download project.enc

Comparison of results:

• Regular download: 200KB/s -> 0KB/s (stalled).
• Aria2 download: [#20a9c1 1.2GiB/6.2GiB(19%) CN:4 DL:5.2MiB] -> a steady 5.2MB/s, running at full speed!

Step 5: Decrypt and Extract

When the download finishes, you have project.zip. Now use the same password to turn it back into a zip archive.

# 解压
unzip project.zip

Conclusion

When the SSH link quality is terrible, don’t stubbornly keep trying SCP or Rsync.

“Compress locally -> relay through Netdisk -> fetch with Aria2 multi-threaded” is a path that, although it has a few more steps, is absolutely workable, safe, and fast. Especially for several GB of code or model weight files, it is the most time-saving choice.

Tip: If your file is not in the /apps/bypy/ directory, remember to move it there on the Netdisk web page, otherwise bypy may not find the file (the apps folder name may be in Chinese, so pay attention to identifying it).