this post was submitted on 29 Nov 2023
2 points (75.0% liked)

Self-Hosted Main

511 readers
1 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

For Example

We welcome posts that include suggestions for good self-hosted alternatives to popular online services, how they are better, or how they give back control of your data. Also include hints and tips for less technical readers.

Useful Lists

founded 1 year ago
MODERATORS
 

I'm using a SFF Lenovo M700 (i5-6400T, 8GB RAM, 256GB SSD, 16TB USB HDD, Debian 11) for my server.

At the moment, I'm just copying all the files from the drives on my Windows PC to the server using WinSCP to make sure that I have a backup. Speeds are around 40MB/s with large files, which is probably as good as I can expect transferring from/to spinning disk, but sshd on the M700 is using 35-50% CPU, and sftp-server is using 15-20%, so about 70% in total. That only seems to happen with large files, when transferring lots of small files they're using about 10% and 5% respectively, although it varies and can be double or half that.

If it's going to use this much CPU whenever someone (or my sync or backup software) is transferring large files, I'm concerned that it won't have the capacity to run the other services that I need (Adguard, Home Assistant (probably as a VM in Proxmox), Jellyfin, Tailscale, Crowdsec, etc.). The 16TB USB HDD is encrypted with Veracrypt, but I don't think that's the issue as I see separate processes in top for kcryptd, and they generally add up to less than 10%.

Is there anything I can do to reduce the CPU usage when transferring files to/from other PC's on the LAN to the server? Once it is deployed, the users won't be using WinSCP to transfer files, they'll probably use Filebrowser or STFPGo, and I'll set up automated syncs and pull backups, so will it use less CPU to transfer files using those methods rather than WinSCP?

top 2 comments
sorted by: hot top controversial new old
[โ€“] [email protected] 3 points 11 months ago

Is SFTP compression at play?

[โ€“] [email protected] 1 points 11 months ago

As other commenter mentioned, check compression.

Transferring over ssh will use more CPU than something simpler/not encrypted. If you want fast, I would try NFS or even Samba. If you want fastest possible, netcat will be hard to beat but that's getting silly.