It all started as a simple task to download through 100 megabit network large amounts of data using rsync. The question arose whether it is possible to accelerate this process. Top utility showed that the encryption takes no more than 10 percent of the CPU in the source server, so it was decided to try to compress the data. Then it was unclear to me if there will be enough CPU performance to package data with the necessary speed, therefore there was set the smallest level of compression, namely it was used the flag - compress-level = 1 for rsync. It turned out that the CPU loading does not exceed 65%, namely there was enough CPU performance at that downloading speed of data has increased slightly.

After that, the question arose about the applicability of the analysis of accepted compression software for transferring large amounts of data over the network.
Siera 13 april 2012, 18:27