Tuesday, May 20, 2014

Dropbox API

Dropbox has a high quality and well documented API that provides Oath2 RESTful read/write access to their online file store.

Their free account provides 2GB of space.

Their pay account has the following options:

total    billed     billed
space    monthly    yearly

100 GB   $10.99/mo   $9.08/mo
200 GB   $21.99/mo  $18.25/mo
500 GB   $54.99/mo  $45.75/mo

Dropbox for business is insane. It's $15/user/mo with a minimum 5 users, thus minimum $75/mo. It provides unlimited storage.

This compares to AWS RRS S3 rates as follows, although that is only storage costs. Upload to S3 is free. Download of 5 TB/mo costs an additional $614.4/mo.

total    billed 
space    monthly 

100 GB    $3/mo
200 GB    $6/mo
500 GB   $15/mo
  • Files uploaded via the desktop or mobile apps have no file size limit.
  • Files uploaded through the website have a 10 GB limit.
  • The bandwidth limit for public links is 20 GB/day (free account) or 200 GB/day (pay account). It seems that once the limit is reached then all public links return error 404 for the rest of the day. There is no limit on private and shared folder (i.e. not web links) file bandwidth.
  • Dropbox keeps one redundant copy in addition to the redundant copy on your local drive.
  • Their packrat feature seems to be a giant loophole in that "deleted files do not count against your storage quota", thus when you expect to not need a file for a long time you can "delete" it to free-up your storage quota, and when you need it again, just "un-delete" it.
  • The business agreement states that "Customer will not (i) sell, resell, or lease the Services".
  • The acceptable use policy states that you will not "abuse the Services in a manner that circumvents their storage space limits" or "sell the Services or Services accounts via unauthorized channels".
  • The size of a shared folder will be counted against the quota of every member of that folder. For Business accounts, the size of the shared folder will be counted against the team's shared quota only once.
  • When you share a folder with someone they get an email asking if they want to participate in the share. It only shows the sender and the folder name, not its size. If you only have a 2GB account that the shared folder has 3GB it's not clear what happens.
  • The files and folders in your Dropbox for Business account will not be accessible to your teammates unless you deliberately share them.
  • I couldn't find anywhere that explicitly states it's against the rules for two humans to use the same dropbox username/password on two machines simultaneously but this seems obviously against the rules and they confirmed that (see below) when I asked.
  • You can have both a dropbox for business and personal dropbox on the same device.

Let's say you signed up for a business account and produced 1TB/mo and via public links shared 200 GB/day. This costs 75*12=$900/yr plus your local internet account must support the upload of 1TB/mo plus your local disk must support all 12TB produced.

Comparatively, this would cost 0.0236*1024*(1+2+3+4+5+6+7+8+9+10+11+12) = $1884 for RRS S3 AWS storage and 0.120*200*365/12 = $730 for public download.

To work around the 200 GB/day limit you could share your data to free dropbox accounts which would only be able to access 2GB per session or to pay dropbox accounts which would have a higher limit.

To work around the local disk limit you could delete data that you expect to not need in the near future (thus freeing local space) and thanks to their packrat feature, undelete it at any time in the future.

Questions and Answers

I asked these yes/no questions about drop-box for business and got the following answers:

1. Can I choose to be billed yearly?
Yes

2. According to this and this, dropbox stores at least two online copies of my data (in addition to the local copy on my computer) and uses storage designed for 99.999999999% durability. Does the same apply for deleted files? Said another way: dropbox for business can "restore deleted ... versions of any file", is the recovery of deleted files just as reliable as access to regular files except that I don't have a local copy of deleted files?
Yes

3. If I delete a folder that contains 1TB of files today, can I expect to be able to restore the entire folder two years from today?
Yes

4. Is there a significant delay when restoring deleted files (other than the regular download time)?
No

5. According to this, dropbox for business costs $15/user/month. Can a workstation be considered equal to a user? For example if I have two geographically separate offices each containing a PC that is always logged in as the same OS user and dropbox user, can this count as two users regardless of which human operator(s) sits at the PC?
We really don't recommend that usage. And, sharing one Dropbox for Business account with multiple users is against the Terms of Service. It also can render the event histories null.

6. According to this, the bandwith limit on public links is 200GB/day. Is there a bandwidth limit for desktop synchronization of shared folders?
No

7. According to this, there is no storage limit. Is there any storage threshold that requires intervention or confirmation to surpass?
No

Backup

What about using dropbox purely as a mass-data backup solution? Dropbox will only sync the files in your dropbox folder. The unofficial dropbox wiki suggests that synobox implements a full dropbox client in a NAS appliance using the dropbox REST API. The offering from QNAP is more promising. Their cheapest unit is a hot-swapable 12-bay NAS drive with integratoin to AWS-S3, ElephantDrive, Google Drive, Dropbox, and Symform.

Custom Desktop Client

How can you write a desktop client without a webserver for the oath callback? First create an app with full dropbox access from dropbox.com

Create App
Dropbox API app
Files and datastores
No - My app needs access to files already on Dropbox
All file types - My app needs access to a user's full Dropbox
App name = HoltstromFileManager
I agree to Dropbox API Terms and Conditions

That means that you agreed to the following:

(b) We may limit the number of calls accepted by the API if we believe that the number of calls to the Dropbox API may negatively impact the Dropbox API or Dropbox service.

Also note the following from the developer guide:

Don't build file sharing apps

Dropbox doesn't support building publicly searchable file sharing networks on top of Dropbox.

Once you start implementing OAuth2, you'll see that it supports a code flow where there is no redirect url. Thus your desktop app can launch a web browser provide web view and that navigates to

https://www.dropbox.com/1/oauth2/authorize?response_type=code&client_id=YOUR_APP_KEY

which will prompt the user to login or create an account then ask if they're willing to let your app access their data. If they say yes, they are taken to a page that displays a code which you can screen-scrape or they can copy/paste into your app. With the code you can get a token which shockingly never expires (until manually revoked by the user via the dropbox website). Thereafter with this token you can read/write that user's files, so guard it carefully. See thier python example for a command-line version of the above.

Why bother writing your own desktop client? Well when you share a folder there is no download priority. Maybe you only want part of that shared folder delivered to you on demand. Is an app of that description a violation of their terms? That's not clear. Can you join a shared folder and not sync the whole thing to your desktop? I'm not sure about that either.

Web Links

It appears that web links are no good for structured sharing through the browser. This is because when you share a dropbox folder via a web link that link is actually to an html page that shows the contents of the folder. The user can then individually download the files. They can view images in the browser, but if you shared a folder that contained index.html and styles.css and goat.jpg, the prefix urls of the individual files would differ so there's no easy way to make that a browse-able website. I think that via the API you could get the web-link of each file, but now you're jumping through too many hoops and should probably use a different service.

web
{ "loggedin": false, "owner": false, "avatar": "", "render": "nothing", "trackingID": "UA-36983794-1", "description": "", "page": { "blogIds": [ 499 ] }, "domain": "holtstrom.com", "base": "\/michael", "url": "https:\/\/holtstrom.com\/michael\/", "frameworkFiles": "https:\/\/holtstrom.com\/michael\/_framework\/_files.4\/", "commonFiles": "https:\/\/holtstrom.com\/michael\/_common\/_files.3\/", "mediaFiles": "https:\/\/holtstrom.com\/michael\/media\/_files.3\/", "tmdbUrl": "http:\/\/www.themoviedb.org\/", "tmdbPoster": "http:\/\/image.tmdb.org\/t\/p\/w342" }