Moderating cloud upload speed and/or times

  • 2
  • Idea
  • Updated 3 months ago
  • (Edited)
Testing the trial version of the new Lightroom CC 1.5 today, I hit an unexpected issue.  I dropped about 5GB of images on the application and it furiously started uploading them to the cloud.  This is good, of course, except that my uplink is only about 15Mbps .. not too shabby, but less than LR's upload helper wanted.

The unfortunate result is called "buffer-bloat" (https://en.wikipedia.org/wiki/Bufferbloat) and the paradoxical effect is to choke the downlink .. my 50Mbs down was cut to ~1Mbs, effectively shutting down any download in the house, sometimes for long enough to timeout some services.

Many applications with the potential do this (especially cloud-based backup services) provide throughput controls (eg: max upload rate: 2Mbps) or time windows (eg: only upload between: 1am and 7am), or the combination of both (slow the upload during working hours, go as fast a possible overnight).

I'd like LR's helper to do this too.

In passing, I'll note that this is not a new problem on the Internet and standards have been developed to allow TCP links to detect and self-correct buffer-bloat.  One is Explicit Congestion Notification (https://en.wikipedia.org/wiki/Explicit_Congestion_Notification).  It's implemented in Apple's TCP stack, but to work right, it needs to be implemented in both endpoint and all the routers along the path.  Hopefully Adobe's servers implement ECN.
Photo of Gavin Eadie

Gavin Eadie

  • 2 Posts
  • 0 Reply Likes
  • concerned

Posted 3 months ago

  • 2
Photo of Dave Taht

Dave Taht

  • 1 Post
  • 0 Reply Likes
The other answer is to get a router that supports smart queue management (fq_codel, rfc8290). There are now a lot of these available commercially (evenroute, edgerouters, most gaming routers), and nearly universal support in the openwrt/dd-wrt/tomato/pfsense/netgate universe if you are willing to reflash and reconfigured. ECN (although supported by the aqm in all these cases) is really not needed for bulk uploads, just a little bit of constructive packet drop and fair queuing and you won't even notice you doing the big upload while doing other tasks. 

Rate limiting as you suggest is also useful so long as you are the only one using the link and that's the only application you have bulking things up.

There are lots of examples of the results of applying sqm all over the internet, google for bufferbloat sqm . see http://burntchrome.blogspot.com/2014/05/fixing-bufferbloat-on-comcasts-blast.html