2·
6 months agoi can imagine some kind of LRU cache being reasonably useful for this situation, assuming you have some latency hierarchy. For example if the desktop has an SSD, HDD, and some USB HDDs attached I can imagine you having a smaller cache that keeps more frequently accessed files on the SSD, followed by a bigger one on the internal HDD, and followed again by USB HDDs as the ultimate origin of the data. Or even just have the SSD as cache and everything else is origin. I don’t know if there’s software that would do this kind of thing already though.
You may want to consider zipping files for transfer though, especially if the transfer protocol is creating new tcp connections for every file.
gucci gucci