Joho the Blogbeginner2beginner Archives - Joho the Blog

March 16, 2016

Advanced advanced Google search

Suppose you want to find pages that use the word “Disneyland” as a link, as in: Disneyland?

I looked and looked but could not find a way to do that at Google. The first return at Google for the query “google advanced search” takes you to a form where you can construct a more finicky search than normal. The Google help page for advanced search has some interesting operators, but not what I need. Limiting my exploration to Google.com (advanced search site:google.com) I could find no explanation of how to do this.

Fortunately, Gary Stock knows more about this than I do. He told me about “inanchor.” Once you know what to look for, you can find some well-hidden Google pages that mention it, such as this aging Google Sites page. Inanchor gets close to what I’m looking for. In fact, for my particular purposes, it’s better.

The following query will return all the pages linked to by the word “Disneyland”:

inanchor:disneyland

For example, if you click on the link in the first paragraph of this post, you’ll go not to Disneyland.com but to a page about Disney’s role in tightening copyright restrictions. If you run the inanchor query, that page about copyright will show up in the results, because it is a page linked to by the word “Disneyland.” In other words, you don’t get back a list of the pages that contain a link that uses the word “Disneyland”; you get back the pages that those pages link to.

A better example might help. If you search for:

inanchor:”most likeable person ever”

you’ll find the pages people have pointed to with a Most Likeable Person Ever link. (“allinanchor” searches for links containing the words that follow in any order.)

I’m not 100% sure inanchor actually works because I don’t see a way to get the pages that contain the links. Maybe I’ll ask Gary.

Comments Off on Advanced advanced Google search

October 5, 2015

Enabling JavaScript to read files on your drive via Dropbox: A “Where I went wrong” puzzle.

Ermahgerd, this was so much harder than I thought it would be. In fact, what follows is best approached as a puzzler in which your task is to find the earliest place where I’ve gone horribly wrong. The winning comment will be of the form, “You’re such an idiot! All you had to do was____!” Second place, because less satisfying but no less humiliating, will be comments of the form, “OMFG, why are you writing this? How can you get the simplest thing wrong???”

I know. Forgive me.


So, let’s say you’re writing, oh, an outliner for your own personal use because the one you’ve been using for seven years or so no longer supports Dropbox: If you save an OmniOutliner file to a Dropbox folder, it only gets one of the sub-files. You poke around with alternatives but none of have exactly the set of minimal features you want. (Dave Winer’s Fargo gets damn close to my peculiarities, and it saves outlines in Dropbox…but in only one special folder. I’m picky. And I was looking for a programming project.) So, you decide to write your own. Sure, it’ll be under-featured, and it’ll break. But it’ll be yours.


It’s going to run in a browser because you can’t find any other way to write an app for your Mac except Objective C and Swift, both of which require three days of tutorials and a hundred pushups to get to “Hello, world.” So, you’re using JavaScript and jQuery, JavaScript’s smarter older brother. (Hi, Andy.) And PHP.


Now, you can try as hard as you want, but “The browser is going to insist on protecting you from accessing files on anyone’s hard drive, even your own”the browser is going to insist on protecting you from being able to access files on anyone’s hard drive, even your own, because otherwise Malefactors are going to install apps that will suck your brains out through your smallest aperture and take your credit card numbers with it. For real.

I tried many of the things the Internet recommends to circumvent this well-meaning rule. I wouldn’t have even tried, but I’m running my outliner on my local hard drive, using the Apache2 web server that comes with MAMP. So, I understand why there’s a wall around the files that are not part of what the web server considers to be its home, but those files are mind. So close, yet so far.

I tried figuring out how to set up a virtual directory, but the initial efforts failed and monkeying with apache files scares me. Likewise for changing the server’s document root.

I put a symbolic link to my Dropbox folder into the JavaScript project’s folder (actually in the “php” sub-folder), and was able to write a file into it via PHP. But I couldn’t figure out a way to read the Dropbox folder, which means that if I wanted to switch from loading an outline from

/Dropbox/blogposts/2015/October

to

/Dropbox/Bad_Ideas/Recipes_For_Disaster


I’d have to type in the entire pathname. No directory browsing for you!

(To create a symbolic link, in your Mac terminal type: “ln -s /Users/YOUR_NAME/Dropbox”. )


So, I had a brainstorm. I use outlines in almost everything I do, but virtually everything I do is in Dropbox. “Dropbox.com has a perfect mirror of my files and folder structure. ”Dropbox.com therefore has a perfect mirror of my files and folder structure. Perhaps Dropbox has an API that would let me browse its mirror of my local disk.


It does! With lots of documentation, almost none of which I understand! I’m sure it’s terrific for people who know what they’re doing, but won’t someone please think of the people who need a For Dummies book to read a For Dummies book?


What I’d like to do is to browse the file structure at Dropbox.com so I can identify the file I want to open, and have Dropbox.com tell me via its API what that file’s path is. Then I could use PHP or even JavaScript (I think) to directly open that file on my own disk via the Dropbox symbolic link in my PHP folder. Right?


Guess what the Dropbox.com API doesn’t tell you. Which is too bad because I want to use the same info later to save a file to a pathname inside that symbolic link.

But Dropbox does make it easy for you to drop a magic button into your HTML that will launch a Dropbox mini-file-browser. The version called the “Chooser” downloads files. The version called “Saver” uploads them. Just what I need.


Sort of. What I’d really like to do is:


  • Browse my Dropbox folders using the Chooser.

  • Click to download my chosen file.


  • Read the content of that file into my app, so I can display the outline contained within.



“As a matter of principle, I want to be able to have a user choose it, and read the contents programmatically. Thus did I lose, oh, two days of my life.”As a matter of principle, I want to be able to have a user choose it, and read the contents programmatically. Thus did I lose, oh, two days of my life.


I will not bore you with the many ways I tried to do this basic thing. I am sure that someone is going to read this and give me the one line of code I need. Instead, here is the complex way I managed to accomplish this basic task.

Register your app

First, you have to register your app with Dropbox in order to get a key that will let you access their API. This is relatively simple to do. Go to their App Console, and click on the “Create App” button. When asked, say you want to use the Dropbox API, not the Dropox for Business API, unless you have a business account with Dropbox. It will ask if you want to access a particular folder or all of the user’s folders. It will ask you to give your app a name; it has to be unique among all apps registered there.

On the next page, the fourth line is your app key. Copy it. Click on the “app secret” and copy it too. For OAuth redirect, use “localhost” if you’re hosting your app locally. Or put in the URL of where you’re hosting if it’s going to be out on the Web. Likewise for “Chooser/Saver domains.”

Now, into your HTML file place the line:

<script type=”text/javascript” src=”https://www.dropbox.com/static/api/2/dropins.js” id=”dropboxjs” data-app-key=”YOUR_KEY”></script>

Obviously, insert your Dropbox key (see above) appropriately.

Ok, let’s create the app.

The app


Into your HTML document create an empty div where the Dropbox button will go:

1

<div id=”DBbutton”></div>


In the header of your HTML document make sure you’ve included jQuery:

1

<script src=”https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js”></script>


Of course, if you prefer to download your own copy of jQuery instead of using Google’s, go ahead. But at this point so much of what I do goes through Google that avoiding using it for jQuery seems foolish.

Also in your header, after the jQuery line, place the following:

1

<script type=”text/javascript” src=”./js/Your_File_Name.js”></script>

Create a subfolder in the directory where your HTML file is and name it “js”. Using your favorite text editor create a file called whatever you want to call it, with a “js” extension. Obviously make sure that the file’s name is exactly the one in the line above. That .js file is where you’ll put your JavaScript…although in this example I’m including it all in the HTML file itself because all I’m going to do is going to occur in the script that loads immediately after the file loads. So never mind.

Here’s the rest of what should go into the head section of your HTML file.

1

<script type=”text/javascript”>

 

2

$(document).ready(function(){

3

var opts= {

4

success: function(files) {

5

var filename = files[0].link;

6

filename = filename.replace(“dl=0″,”dl=1”);

7

//alert(filename);

8

$.ajax({

9

url: “./php/downloadDropboxContents.php”,

10

data: “src=” + filename,

11

success: function(cont){

12

//alert(cont);

13

},

14

error: function(e){

15

alert(e.responseText);

16

}

17

});

18

},

19

extensions: [“.txt”,”.opml”],

20

multiselect: false,

21

linkType: “download”

22

};

23

var button = Dropbox.createChooseButton(opts);

24

document.getElementById(“DBbutton”).appendChild(button);

25

});

26

</script>


Line 2 is a very handy jQuery function that gets executed after the entire page has been downloaded into the browser. That means all the bits and pieces are there before the code in the function is executed.


In this case, the code is going to create a Dropbox button for the user to press. The options for that button are expressed in lines 2-22. Let’s start with the last lines.


Line 19 lists the extensions I want to let users (= me) select for download. There are only two: files that end with .txt and ones that end with .opml. OPML is the standard format for outlines. (Thank you, Dave Winer.)

Line 20 says that I don’t want users to be able to open more than one file at a time.


On line 21 we specify that we want Dropbox to give us back the downloaded file. The alternative is “preview,” which will provide a preview.


By the way, note that each option line ends with a comma, except for the last one. This whole option statement is actually a JSON set of key:value pairs, each delimited by a comma. In some cases, as in Dreaded Line 4, the values are multi-line and complex. Nevertheless, they’re still just values of the keyword to the left of the colon.


But I’m just putting off talking about the “success” option, lines 4-18, that set what happens if the download operation is successful.


Line 4 creates a function that will get passed an array of downloaded files, which unimaginatively I am capturing in the variable “files.”


Line 5 gets the link to the first file in the array. The array is files[]. The appended “.link” gets the URL to the Dropbox file, but it’s a funky link that, alas, doesn’t express the pathname, but some seemingly arbitrary set of characters. For example:

https://www.dropbox.com/s/smwhasdgztdiw5gbn/A%20History%20of%20the%20Philosophy%20of%20Time.%20txt?dl=0



If you were instead to say “files[0].name”, you’d get the file’s name (“A History of the Philosophy of Time.txt”). And if you say “.path” you — damn their eyes — get an error. Aargh. This could have been so much easier! Anyway.


“Line 6 is something I discovered on my own, i.e., I didn’t read the documentation carefully enough.”Line 6 is something I discovered on my own, i.e., I didn’t read the documentation carefully enough. Notice the “dl=0” at the end of the file link above. I’m going to guess the “dl” stands for “download.” If you leave it at 0, you get the user interface. But — aha! — if you replace it with 1, it downloads the actual damn file into your normal download folder, which defaults on the Mac to the Download folder. So, line 6 does the search and replace. (If line 7 weren’t commented out, it’d pop up the file link.)


So now we have a link that will download the file. Excellent!


Lines 8-17 use that URL to actually download it and read it. This requires (i.e., it’s the only way I know how to do it) executing a PHP script. For that we use AJAX, which JavaScript makes hard but jQuery makes easy.


Line 9 points to the PHP file. It lives in a folder called “php.” The “./” is redundant — it says “that folder is in the current directory” but I’m superstitious. We’ll write the PHP file soon.


Line 10 is the dumb way of saying what data we’re going to pass into the PHP script. We’re using the variable “src” and we’re passing the path to the downloadable Dropbox file. The better way to express this data would be to use JSON, but I never remember whether you put the key in quotes or not, so I’d rather do it this way (which in essence simply writes out the appendage to the basic PHP’s script URL) than look it up. But, I just did look it up, and, no, you don’t quote the keys. So line 10 should really be:

data: {src : filename},

but I’m too lazy to do that.

Now in line 11 we get to what we do with the results of the PHP script’s processing of the content it’s going to receive. The commented-out line would post the content into a dialogue box so you can confirm you got it, but what I really want to do is turn the content of that file into a outline displayed by my app. So, my real line 12 will be something like “displayOutline(cont)”, a function that I’ll stick elsewhere in my JavaScript. But that’s not what we’re here to talk about.


Lines 14-6 get invoked if the PHP fails. It displays a human-readable version of the error code. You’ll also want to be looking at your console’s error log. If you’re using MAMP, look at php_error.log, which you’ll find in MAMP/logs.


At line 23, we’re outside of the options declaration. Line 23 uses Dropbox to create a Chooser button that when pressed will pop up the Chooser with the right options set. “With luck, when you load it, you’ll see a Dropbox button sitting there.”


The button exists but not on your page. For that to happen, you need line 24 to tell your page to find the div with the id of “DBbutton” and to insert the button into it as a new last element. (Since there are no elements in that div, the button becomes its only element.)


All this happens before your page becomes visible. With luck, when you load it, you’ll see a Dropbox button sitting there.


Now onto the PHP.

The PHP

Create a folder named “php” in the same directory as your HTML file. In it create a file called “downloadDropboxContents.php”.


Here it is:

1

<?php

 

2

$src = $_REQUEST[‘src’]; // url

3

$filename = basename($src); // get the file’s name

4

error_log(“SRC; $src – FILENAME: $filename”);

5

$dir=”Downloads”; // set the folder to download into

6

// create the pathname for the downloaded file

7

$downloads = $dir . “/” . $filename; // md5($src);

8

// get the contents of the download — YAY!

9

$out = file_get_contents($src);

10

error_log($out);

11

// put the downloaded file there

12

file_put_contents($downloads, $out);

13

// repeat the contents out loud

14

echo $out;

 

15

?>


The comments should tell the story. But just in case:


Line 2 picks up the data we’ve passed into it. $src now should have the URL to the Dropbox file.


Line 3 gets the file name from the pathname. We’re going to need that when we save the file into our designated folder (which is “Downloads,” which you may recall, we created a symbolic link to in our php folder.)


Line 4 optionally writes the Dropbox URL and the filename into the console (see above), just to see where we’ve gone wrong this time.


Line 5 specifies what folder to put the downloaded file into. Remember that it has to be within the realm your web server counts as document root. Hence the symbolic link to Downloads in the php folder.


Line 7 creates the path name to that download folder by appending the file name to the path, with a “/” in between them.


Line 9 copies the actual damn contents of the downloaded file into a variable I’ve called “$out”. Line 10 checks the content. You probably want to comment that line out.“Line 14 reports the contents back to the “success” function…”


Line 12 writes the content into the download directory.


Line 14 reports those contents back to the “success” function in the JavaScript. It will there be captured by the variable “cont” in line 11.


That’s it. I know this is sub-optimal at best, and probably far more fragile than I think. But it works for now, at least with simple text files. And I couldn’t find anything at this level of beginnerness online.


I’m sorry.

1 Comment »

August 1, 2010

Beginner to Beginner: When a Javascript program only works in a debugger

[LATER the same day:] Thanks to expert advice from some generous expert folks in the comments, I found the mistake I was making; I explain it in the comment #5. But, the mystery of why it worked only when run in debug mode remains. (Tom in comment #4 has a good hypothesis.) Nevertheless, I’m sticking with my overall point that for a home-made, one-user project like this, a brute-force, dumb-ass solution sometimes serves. I should maybe mention also that one reason I post about “programming” is precisely because I’m an idiot about it, and I think it’s good to be a beginner in public.

Aaarrrggghhh. I spent way too much time yesterday and today trying to fix a problem with a javascript program the right way. When I finally gave up and did it the wrong way, it worked fine…except for being embarrassingly inelegant.

It’s a mystery. The program doesn’t work ever except if you run it in a debugger and set a breakpoint anywhere in either of the two affected functions. It doesn’t even work if you put an “alert()” into it. It really really wants a breakpoint. This is consistent in both Firefox and Chrome, with Firebug and the integrated Chrome debugger. It only works if you set a breakpoint. It does not work if you do not set a breakpoint. Did I mention Aaaarrrrrrggggghhhhhh?

Some details: I added a function to my little all-crap blog editor. The function (call it FunctionA) gets called when the user (= me and only me) presses a button. FunctionA calls FunctionB to see if the selected text is listed in an array of terms. If so, then FunctionA inserts some text using associated text. (To be more concrete, although it doesn’t matter: If the selected text is the name of someone for whom I have a twitter name, FunctionA builds and inserts a link to the person’s Twitter address.) FunctionB calls one other function (that trims spaces and other cruft) and one method (to temporarily uppercase both the selected text and the name in the database, so that the search won’t be case sensitive, and yes, I’m sure there’s a better way to do that). Then it simply walks through the array, looking for a match.

I googled around and didn’t find much help with the mysterious fact that it only works if I set a breakpoint. People with similar problems can sometimes solve them by using the setTimeout() function, to let the script catch up with itself. But, that’s not really what setTimeout does, apparently. It doesn’t pause the script. Rather, it delays the execution of a particular function while the rest of the script continues merrily on. I also found that a common cause of a similar problem is that your script is trying to access a part of the document that hasn’t been loaded or created yet. But: (a) My script isn’t trying to do that, and (b) If that were the problem, then putting in an alert() would have worked for me, no? The cause might be some multithreaded mystery for which there is no known solution, but I really don’t know. (You might try this.)

So, after looking at some clever recursive setTimeout strategies that I couldn’t figure out how to adapt to my particular script, I had a stupid idea. I put the code from FunctionB — the one that looks the selected text up in an array — straight into FunctionA. So, FunctionA doesn’t call FunctionB. It has within it a copy of the code in FunctionB.

And it works.

It works even though it is just about the definition of “inelegant.” The whole point of every programming language since Turing first ran marked paper off a mental spool is to write functions for any process your script might use more than once. Not only does that save time writing, it makes debugging a billion times easier because you only have to fix it in one place. So my solution to my problem is like telling a chef to beat each egg separately or to suggest reloading a supersoaker after each and every squeeze.

But, dammit, it worked. I don’t know why, and I ‘d say that I don’t care, but, I do. Well, at least I can now get back to wasting my time in marginally less frustrating ways.

9 Comments »