Joho the BlogOctober 2015 - Page 3 of 3 - Joho the Blog

October 5, 2015

Enabling JavaScript to read files on your drive via Dropbox: A “Where I went wrong” puzzle.

Ermahgerd, this was so much harder than I thought it would be. In fact, what follows is best approached as a puzzler in which your task is to find the earliest place where I’ve gone horribly wrong. The winning comment will be of the form, “You’re such an idiot! All you had to do was____!” Second place, because less satisfying but no less humiliating, will be comments of the form, “OMFG, why are you writing this? How can you get the simplest thing wrong???”

I know. Forgive me.


So, let’s say you’re writing, oh, an outliner for your own personal use because the one you’ve been using for seven years or so no longer supports Dropbox: If you save an OmniOutliner file to a Dropbox folder, it only gets one of the sub-files. You poke around with alternatives but none of have exactly the set of minimal features you want. (Dave Winer’s Fargo gets damn close to my peculiarities, and it saves outlines in Dropbox…but in only one special folder. I’m picky. And I was looking for a programming project.) So, you decide to write your own. Sure, it’ll be under-featured, and it’ll break. But it’ll be yours.


It’s going to run in a browser because you can’t find any other way to write an app for your Mac except Objective C and Swift, both of which require three days of tutorials and a hundred pushups to get to “Hello, world.” So, you’re using JavaScript and jQuery, JavaScript’s smarter older brother. (Hi, Andy.) And PHP.


Now, you can try as hard as you want, but “The browser is going to insist on protecting you from accessing files on anyone’s hard drive, even your own”the browser is going to insist on protecting you from being able to access files on anyone’s hard drive, even your own, because otherwise Malefactors are going to install apps that will suck your brains out through your smallest aperture and take your credit card numbers with it. For real.

I tried many of the things the Internet recommends to circumvent this well-meaning rule. I wouldn’t have even tried, but I’m running my outliner on my local hard drive, using the Apache2 web server that comes with MAMP. So, I understand why there’s a wall around the files that are not part of what the web server considers to be its home, but those files are mind. So close, yet so far.

I tried figuring out how to set up a virtual directory, but the initial efforts failed and monkeying with apache files scares me. Likewise for changing the server’s document root.

I put a symbolic link to my Dropbox folder into the JavaScript project’s folder (actually in the “php” sub-folder), and was able to write a file into it via PHP. But I couldn’t figure out a way to read the Dropbox folder, which means that if I wanted to switch from loading an outline from

/Dropbox/blogposts/2015/October

to

/Dropbox/Bad_Ideas/Recipes_For_Disaster


I’d have to type in the entire pathname. No directory browsing for you!

(To create a symbolic link, in your Mac terminal type: “ln -s /Users/YOUR_NAME/Dropbox”. )


So, I had a brainstorm. I use outlines in almost everything I do, but virtually everything I do is in Dropbox. “Dropbox.com has a perfect mirror of my files and folder structure. ”Dropbox.com therefore has a perfect mirror of my files and folder structure. Perhaps Dropbox has an API that would let me browse its mirror of my local disk.


It does! With lots of documentation, almost none of which I understand! I’m sure it’s terrific for people who know what they’re doing, but won’t someone please think of the people who need a For Dummies book to read a For Dummies book?


What I’d like to do is to browse the file structure at Dropbox.com so I can identify the file I want to open, and have Dropbox.com tell me via its API what that file’s path is. Then I could use PHP or even JavaScript (I think) to directly open that file on my own disk via the Dropbox symbolic link in my PHP folder. Right?


Guess what the Dropbox.com API doesn’t tell you. Which is too bad because I want to use the same info later to save a file to a pathname inside that symbolic link.

But Dropbox does make it easy for you to drop a magic button into your HTML that will launch a Dropbox mini-file-browser. The version called the “Chooser” downloads files. The version called “Saver” uploads them. Just what I need.


Sort of. What I’d really like to do is:


  • Browse my Dropbox folders using the Chooser.

  • Click to download my chosen file.


  • Read the content of that file into my app, so I can display the outline contained within.



“As a matter of principle, I want to be able to have a user choose it, and read the contents programmatically. Thus did I lose, oh, two days of my life.”As a matter of principle, I want to be able to have a user choose it, and read the contents programmatically. Thus did I lose, oh, two days of my life.


I will not bore you with the many ways I tried to do this basic thing. I am sure that someone is going to read this and give me the one line of code I need. Instead, here is the complex way I managed to accomplish this basic task.

Register your app

First, you have to register your app with Dropbox in order to get a key that will let you access their API. This is relatively simple to do. Go to their App Console, and click on the “Create App” button. When asked, say you want to use the Dropbox API, not the Dropox for Business API, unless you have a business account with Dropbox. It will ask if you want to access a particular folder or all of the user’s folders. It will ask you to give your app a name; it has to be unique among all apps registered there.

On the next page, the fourth line is your app key. Copy it. Click on the “app secret” and copy it too. For OAuth redirect, use “localhost” if you’re hosting your app locally. Or put in the URL of where you’re hosting if it’s going to be out on the Web. Likewise for “Chooser/Saver domains.”

Now, into your HTML file place the line:

<script type=”text/javascript” src=”https://www.dropbox.com/static/api/2/dropins.js” id=”dropboxjs” data-app-key=”YOUR_KEY”></script>

Obviously, insert your Dropbox key (see above) appropriately.

Ok, let’s create the app.

The app


Into your HTML document create an empty div where the Dropbox button will go:

1

<div id=”DBbutton”></div>


In the header of your HTML document make sure you’ve included jQuery:

1

<script src=”https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js”></script>


Of course, if you prefer to download your own copy of jQuery instead of using Google’s, go ahead. But at this point so much of what I do goes through Google that avoiding using it for jQuery seems foolish.

Also in your header, after the jQuery line, place the following:

1

<script type=”text/javascript” src=”./js/Your_File_Name.js”></script>

Create a subfolder in the directory where your HTML file is and name it “js”. Using your favorite text editor create a file called whatever you want to call it, with a “js” extension. Obviously make sure that the file’s name is exactly the one in the line above. That .js file is where you’ll put your JavaScript…although in this example I’m including it all in the HTML file itself because all I’m going to do is going to occur in the script that loads immediately after the file loads. So never mind.

Here’s the rest of what should go into the head section of your HTML file.

1

<script type=”text/javascript”>

 

2

$(document).ready(function(){

3

var opts= {

4

success: function(files) {

5

var filename = files[0].link;

6

filename = filename.replace(“dl=0″,”dl=1”);

7

//alert(filename);

8

$.ajax({

9

url: “./php/downloadDropboxContents.php”,

10

data: “src=” + filename,

11

success: function(cont){

12

//alert(cont);

13

},

14

error: function(e){

15

alert(e.responseText);

16

}

17

});

18

},

19

extensions: [“.txt”,”.opml”],

20

multiselect: false,

21

linkType: “download”

22

};

23

var button = Dropbox.createChooseButton(opts);

24

document.getElementById(“DBbutton”).appendChild(button);

25

});

26

</script>


Line 2 is a very handy jQuery function that gets executed after the entire page has been downloaded into the browser. That means all the bits and pieces are there before the code in the function is executed.


In this case, the code is going to create a Dropbox button for the user to press. The options for that button are expressed in lines 2-22. Let’s start with the last lines.


Line 19 lists the extensions I want to let users (= me) select for download. There are only two: files that end with .txt and ones that end with .opml. OPML is the standard format for outlines. (Thank you, Dave Winer.)

Line 20 says that I don’t want users to be able to open more than one file at a time.


On line 21 we specify that we want Dropbox to give us back the downloaded file. The alternative is “preview,” which will provide a preview.


By the way, note that each option line ends with a comma, except for the last one. This whole option statement is actually a JSON set of key:value pairs, each delimited by a comma. In some cases, as in Dreaded Line 4, the values are multi-line and complex. Nevertheless, they’re still just values of the keyword to the left of the colon.


But I’m just putting off talking about the “success” option, lines 4-18, that set what happens if the download operation is successful.


Line 4 creates a function that will get passed an array of downloaded files, which unimaginatively I am capturing in the variable “files.”


Line 5 gets the link to the first file in the array. The array is files[]. The appended “.link” gets the URL to the Dropbox file, but it’s a funky link that, alas, doesn’t express the pathname, but some seemingly arbitrary set of characters. For example:

https://www.dropbox.com/s/smwhasdgztdiw5gbn/A%20History%20of%20the%20Philosophy%20of%20Time.%20txt?dl=0



If you were instead to say “files[0].name”, you’d get the file’s name (“A History of the Philosophy of Time.txt”). And if you say “.path” you — damn their eyes — get an error. Aargh. This could have been so much easier! Anyway.


“Line 6 is something I discovered on my own, i.e., I didn’t read the documentation carefully enough.”Line 6 is something I discovered on my own, i.e., I didn’t read the documentation carefully enough. Notice the “dl=0” at the end of the file link above. I’m going to guess the “dl” stands for “download.” If you leave it at 0, you get the user interface. But — aha! — if you replace it with 1, it downloads the actual damn file into your normal download folder, which defaults on the Mac to the Download folder. So, line 6 does the search and replace. (If line 7 weren’t commented out, it’d pop up the file link.)


So now we have a link that will download the file. Excellent!


Lines 8-17 use that URL to actually download it and read it. This requires (i.e., it’s the only way I know how to do it) executing a PHP script. For that we use AJAX, which JavaScript makes hard but jQuery makes easy.


Line 9 points to the PHP file. It lives in a folder called “php.” The “./” is redundant — it says “that folder is in the current directory” but I’m superstitious. We’ll write the PHP file soon.


Line 10 is the dumb way of saying what data we’re going to pass into the PHP script. We’re using the variable “src” and we’re passing the path to the downloadable Dropbox file. The better way to express this data would be to use JSON, but I never remember whether you put the key in quotes or not, so I’d rather do it this way (which in essence simply writes out the appendage to the basic PHP’s script URL) than look it up. But, I just did look it up, and, no, you don’t quote the keys. So line 10 should really be:

data: {src : filename},

but I’m too lazy to do that.

Now in line 11 we get to what we do with the results of the PHP script’s processing of the content it’s going to receive. The commented-out line would post the content into a dialogue box so you can confirm you got it, but what I really want to do is turn the content of that file into a outline displayed by my app. So, my real line 12 will be something like “displayOutline(cont)”, a function that I’ll stick elsewhere in my JavaScript. But that’s not what we’re here to talk about.


Lines 14-6 get invoked if the PHP fails. It displays a human-readable version of the error code. You’ll also want to be looking at your console’s error log. If you’re using MAMP, look at php_error.log, which you’ll find in MAMP/logs.


At line 23, we’re outside of the options declaration. Line 23 uses Dropbox to create a Chooser button that when pressed will pop up the Chooser with the right options set. “With luck, when you load it, you’ll see a Dropbox button sitting there.”


The button exists but not on your page. For that to happen, you need line 24 to tell your page to find the div with the id of “DBbutton” and to insert the button into it as a new last element. (Since there are no elements in that div, the button becomes its only element.)


All this happens before your page becomes visible. With luck, when you load it, you’ll see a Dropbox button sitting there.


Now onto the PHP.

The PHP

Create a folder named “php” in the same directory as your HTML file. In it create a file called “downloadDropboxContents.php”.


Here it is:

1

<?php

 

2

$src = $_REQUEST[‘src’]; // url

3

$filename = basename($src); // get the file’s name

4

error_log(“SRC; $src – FILENAME: $filename”);

5

$dir=”Downloads”; // set the folder to download into

6

// create the pathname for the downloaded file

7

$downloads = $dir . “/” . $filename; // md5($src);

8

// get the contents of the download — YAY!

9

$out = file_get_contents($src);

10

error_log($out);

11

// put the downloaded file there

12

file_put_contents($downloads, $out);

13

// repeat the contents out loud

14

echo $out;

 

15

?>


The comments should tell the story. But just in case:


Line 2 picks up the data we’ve passed into it. $src now should have the URL to the Dropbox file.


Line 3 gets the file name from the pathname. We’re going to need that when we save the file into our designated folder (which is “Downloads,” which you may recall, we created a symbolic link to in our php folder.)


Line 4 optionally writes the Dropbox URL and the filename into the console (see above), just to see where we’ve gone wrong this time.


Line 5 specifies what folder to put the downloaded file into. Remember that it has to be within the realm your web server counts as document root. Hence the symbolic link to Downloads in the php folder.


Line 7 creates the path name to that download folder by appending the file name to the path, with a “/” in between them.


Line 9 copies the actual damn contents of the downloaded file into a variable I’ve called “$out”. Line 10 checks the content. You probably want to comment that line out.“Line 14 reports the contents back to the “success” function…”


Line 12 writes the content into the download directory.


Line 14 reports those contents back to the “success” function in the JavaScript. It will there be captured by the variable “cont” in line 11.


That’s it. I know this is sub-optimal at best, and probably far more fragile than I think. But it works for now, at least with simple text files. And I couldn’t find anything at this level of beginnerness online.


I’m sorry.

1 Comment »

October 4, 2015

This blog has gone spamtacular

In August, the comment section of this blog was hit with 13,000 spam messages, which was at the low end of its normal 25k-35k range. At least this is what Akismet tells me. The number of actual comments is usually in 30-50/month range, I think.

In September, my comment sectionss got 186,998 spams. This has driven up my hosting costs rather spectacularly.

My host, MediaTemple.net — very reasonably priced, a little geeky to use, which is not a bad thing — pointed this out to me. I started checking my WordPress plugins and only then found out that my Akismet API key was no longer valid. I have no idea why it stopped being valid, or when that happened, but I’m hoping it was at the beginning of September. I have reenlisted in Akismet.

Being a dolt, I don’t know if using a comment spam filter like Akismet will reduce the hits on my site, or whether it will simply lower the number of bogus comments I have to manually wade through. I will check tomorrow.

I am also willing to accept ideas today.

(I have temporarily closed comments on posts older than 14 days. Sorry. But it’s not like I get a lot of those.)

UPDATE, that afternoon: The support person at MediaTemple suggested replacing Akismet with WP-Spamshield, and adding WordPress Spam Cleaner to get rid of existing spam. He also suggested this helpful article: Hardening WordPress. Thanks, Media Temple support person!

UPDATE, two days later: My comments traffic has dropped down to its usual trickle. Whew! I’m now going to turn back on the ability to comment on posts older than two weeks, and will see what happens.

2 Comments »

October 2, 2015

Reason #2,645 to love the Web

Back in the early 1980s—yes, children, it’s time for an anecdote from the Dark Ages—WordPerfect was my writing tool. I was a power user and was quite attached to it. But there were some things I thought they could do better. So, I wrote a four page letter that was (as I recall) very appreciative of the program overall — not a set of gripes, but a fan’s notes. I sent it to the WordPerfect corporation.


I never heard anything back. Not even the form letter I expected.

That was back then.

On my Mac I frequently use Sync2Folders “its techie rawness is one of the reasons I like it”to, well, sync two folders. It does exactly what I want, and it’s free, although donations are suggested. (I’ve donated the suggested €6 more than once.)


In terms of the look and feel, Sync2Folders isn’t slick, and in its functionality it tends towards the techie. But it’s simple enough that I can do the basic things that I want to do. In fact, its techie rawness is one of the reasons I like it: It does a job that’s not trendy, and it does it without gussying itself up.


Also, and perhaps more important, it looks like something that a developer created and put out in the world for free. Which is exactly what it is.


A couple of days ago I got an automated email from the developer, Thomas Robisson when I donated for the third time. I’d like to pretend that I’m just that generous, but the truth is that I’m just that forgetful. So, I appreciated that the developer noted the duplication, told me how to avoid the app’s request for fiscal aid, and reminded me that a single license can be used on multiple computers.


I responded by email to thank Thomas, and also to point out a feature that I’d like and that I’d thought was in an earlier version. I was confident that this was going to turn out to be a DUM— a dumb user mistake — and at least I was right about that.“ The Net occasions the generosity of people like Thomas” Over the course of a couple of emails in which Thomas asked for some basic debugging info, it turned out that, yes, I had simply missed the button that did what I was asking for. D’oh.


I know that the Internet is the defiler of youth and the death of civilization. But it also occasions the generosity and creativity of people like Thomas.


Further, before the Net, there was only the slightest chance that a user and a product creator could engage. And if they did it was likely to be in the stilted, inhuman voice of the Marketing department.


So, thank you, Thomas. And thank you, Internet.

2 Comments »

« Previous Page