Joho the Blog » tech

December 3, 2012

Tab Rocker – a Chrome utility to return to previous tab

I get lost in my browser tabs all the time. The place I most often want to go is back to the tab I was just in. On Firefox, there are a few utilities that let me do that. My nephew Joel Weinberger has written one that does that and nothing but that for Chrome. You can grab it (free, of course) here. (The source code is on github.)

Joel wrote this, as the result of my whining, during our annual post-Thanksgiving-dinner viewing of Jurassic Park, although he did some clean up of the code afterwards. I should add that, among other things, Joel is a certified computer genius working in deep areas of computer security.

Be the first to comment »

October 30, 2012

[beginner2beginner] How to add a row to MySQL unless it already exists

This is really basic, but it drove me crazy. Suppose you want to add a row of data to a MySQL database, but not if there’s already an entry in it with the essence of that data. For example, suppose you have a table of notes and the titles they’re clustered under, and don’t want to allow the same note to appear more than once for any particular title. Assume also that that table is also recording an identifier of which book the note is about.

Here’s a SQL statement that works (line breaks don’t matter):

INSERT IGNORE into 
playlistentries(title,noteid,bookid)
VALUES('fiction', '1234','5678');

With the exception of the mysterious “IGNORE” (see below), this is a straightforward command that inserts a row into the table “playlistentries” with a series of values (‘fiction’, ‘1234’,’5678′) mapped to the series of fields (title, noteid, bookid). If there’s already a row with those title and noteid values, the table will be left unchanged. Otherwise, a new row will be added.

But, this will not work unless you set up your MySQL table so that it has a unique key based on the fields you’re testing for (title and noteid in this example). That way when you go to insert a row that has an already-existing title and noteid, it will be automatically rejected. The “IGNORE” in the SQL statement means it will be rejected without creating an error message that just gets in the way.

To set up your table so that it has the right unique key, use a SQL statement like this:

ALTER TABLE playlistentries 
ADD UNIQUE KEY noteidtitle (noteid,title(100))

This tells MySQL to create a unique key (named “noteidtitle”) based on the fields noteid and title. The “(100)” is there to tell MySQL that it should only look at the first 100 characters of the title field; if you’ve set up your table with that field as “text,” you’ll get an error unless you put a limit on it. A hundred characters is probably 75 more than I need.

Note also that you only run the “alter table” command once in the lifetime of your database.

Finally, please note that there is a high probability that what I’m telling you is wildly inefficient, non-robust, and suboptimal. On a scale of 1-100, I am about 3 points past raw beginner. But these commands work for me, and I am assuming as always that if you’re reading this, you are an amateur like me engaged in some small project designed primarily for your own use. Improvements and do-overs will be gladly accepted.

3 Comments »

September 9, 2012

Beginner2Beginner: Javascript multi-file upload + PHP to process it

Much as I love being a hobbyist programmer, it can sometimes be frustrating as all get-out. Sometimes it’s just a bug because I made a dumb error (getting a variable’s scope wrong) or because I made an assumption about how something works (BBedit‘s hex dump does not show you the content of the file on the disk, but of the file in memory, including the line endings it’s transformed). But then there are the frustrations that come from not having the slightest idea of the basics. The worst are the basics that are so basic that the explanations of them assume you already know more than you do.

Welcome to the world of pain known as uploading multiple files using Javascript. For example, suppose you are writing an app that lets users take notes on an article using a plain old text processor. They can then upload those note files to some code that processes them, perhaps turning them into a database. Rather than having users upload one file at a time, you want to let them upload a bunch.

There are plenty of slick, free examples on the Web that provide beautiful front ends for doing this. Many of them I could get to work, but not exactly the way that I wanted. Besides, I was really really really confused about what happened after the front end.

So, after a lot of skullpounding and forehead slapping, here’s a guide. But please take seriously this warning: I barely know what I’m doing, and I’m undoubtedly doing this in the clunkiest fashion possible. I am very likely getting important things wrong. Some of them may be fatal, at least in the programmatic sense. (If you have a correction, please let me know. Note I may fix the code that follows, rather than doing the usual — and proper — Web thing of striking through the errors, etc.) Here goes….

To enable uploads, you’ll be writing code that will live in two places. The user interface code is in the HTML running in the user’s browser. The files are going to be uploaded — copied — to a web server. The code in the browser is going to be Javascript. The code on the server is going to be PHP, because Javascript is for browsers. (Oversimplification noted.) Those two places can be physically the same machine. If you’re using a Mac, a web server comes standard; if you don’t have a web server handy, there are bunches of free ones; that’s beyond the scope of this post.

After experimenting with many of the beautiful packages that are available, I gave up and went with the HTML5 solution. Modern browsers will have no problem with this. If you need to design for the world of old-fashioned browsers, then you need to find a competent explanation. In short: I love you, but go away! (Apparently, if you use the HTML5 solution, it will still be usable in lesser browsers, although it will allow users to select only one file at a time.)

HTML5 makes it ridiculously easy to do the user interface bit: you just tell it you want to allow multiple selections within a file-chooser dialogue. This comes from a post by Tiffany Brown:

<form action="processThem.php" method="post" enctype="multipart/form-data">
<input type="file" value="" name="upload[]" multiple>
<button type="submit">Upload!</button>
</form>

This will create an input field that when clicked will launch a file-browsing dialogue box in which the user can select multiple files. It then sends it to the PHP script called “processThem.php” on your server. The two key points to note are that the first line does the work of allowing multiple choices, and the “[]” in the name turns that variable into an array that will pass the entire list of user choices to the PHP script waiting on your server.

Here’s what it looks like, although this particular example won’t actually upload anything:



Now you have to create the “processThem.php” script (or whatever name you’ve specified) on your server. The uploaded files get placed in an array called $_FILES. But, they don’t get stored on the server for long: they are stored in a temporary folder from which they are automatically deleted after a little while. So, you need to process them, and quite possibly move them for permanent storage to a folder of your choosing. Here’s some sample PHP code from an anonymous commenter (“Me”) on the Tiffany Brown post:


<?php
$error_message[0] = "Unknown problem with upload.";
$error_message[1] = "Uploaded file too large (load_max_filesize).";
$error_message[2] = "Uploaded file too large (MAX_FILE_SIZE).";
$error_message[3] = "File was only partially uploaded.";
$error_message[4] = "Choose a file to upload.";

$upload_dir  = './tmp/';
$num_files = count($_FILES['upload']['name']);

for ($i=0; $i < $num_files; $i++) {
    $upload_file = $upload_dir . urlencode(basename($_FILES['upload']['name'][$i]));

    if (!preg_match("/(gif|jpg|jpeg|png)$/",$_FILES['upload']['name'][$i])) {
        print "I asked for an image...";
    } else {
        if (@is_uploaded_file($_FILES['upload']['tmp_name'][$i])) {
            if (@move_uploaded_file($_FILES['upload']['tmp_name'][$i], 
                $upload_file)) {
                /* Great success... */
                echo "hooray";
                //$content = file_get_contents($upload_file);
                //print $content;
            } else {
                print $error_message[$_FILES['upload']['error'][$i]];
            }
        } else {
            print $error_message[$_FILES['upload']['error'][$i]];
        }    
    }
}
?>

Let’s walk through this.


$error_message[0] = "Unknown problem with upload.";
$error_message[1] = "Uploaded file too large (load_max_filesize).";
$error_message[2] = "Uploaded file too large (MAX_FILE_SIZE).";
$error_message[3] = "File was only partially uploaded.";
$error_message[4] = "Choose a file to upload.";

In the first few lines, Me does us the favor of providing non-technical explanations of possible errors, so that if something goes wrong, it will be easier to know exactly what it was.


$upload_dir  = './tmp/';

Then Me designates a particular folder as the container for the uploaded files. Me chooses one called “tmp” in the same directory as the PHP script. (Make sure you have such a folder and the permissions are set, or, of course, create one with whatever name you’d like.)


$num_files = count($_FILES['upload']['name']);

Then Me gets a count of how many files were uploaded, and stores it in the variable $num_files. You tell that variable that you want the number of files that were included in “upload[]” in the form on your HTML. (You can use whatever name you want on that form, so long as you use the same one in your PHP.)


for ($i=0; $i < $num_files; $i++) {
    $upload_file = $upload_dir . urlencode(basename($_FILES['upload']['name'][$i]));

Then Me loops through all the files, assigning each one in turn to the variable $upload_file. But notice the weirdness of this part of the line:

$upload_file = $upload_dir . urlencode(basename($_FILES[‘upload’][‘name’][$i]));

First the easy parts. The baseline function returns just a file’s name without any path information; we want that because the point of this line is build a path to where the file will be saved in the folder you’ve set up for it. Also, I added the urlencode function in case the name of the file your user uploaded contains spaces or other characters that makes your server barf.

Now consider $_FILES[‘upload’][‘name’][$i]. It’s got those weird bracketed terms because $_FILES is an associative array. You can think of the usual sort of arrays as pairing a value with a number; give the array the number and it returns the value. In an associative array, values are paired with arbitrary keys (i.e., a word); give it the key and it returns the value. Here are the pre-defined keys for the associative array that gets sent to the PHP script when a user uploads files:

  • name: The file name of the uploaded file
  • type: Is it an image? A music file? etc.
  • size: The size in bytes
  • tmp_name: The crazy-ass name of the copy being stored on the server
  • error: Any error codes resulting from the upload

So, suppose you’re cycling through the array of uploaded files as in our example, and you want to get the name of the current file (i.e., file $i in the sequence):


$fname = $_FILES['upload']['name'][$i];

The [‘upload’] designates the array of all uploaded files. The [$i] pulls out of that array all of the information about one particular uploaded file, just like with ordinary array. The [‘name’] gets the value associated with that key for that particular file. As $i is incremented, you get the name of each file. If you wanted the crazy-ass temporary name, you would put in tmp_name instead of name, and likewise for the other parameters.


if (!preg_match("/(gif|jpg|jpeg|png)$/",$_FILES['upload']['name'][$i])) {
        print "I asked for an image...";
    }

Next Me optionally checks the name of the uploaded file for a particular set of extensions in case you want to limit the uploads to images (as in Me’s example) or whatever. Me is using regex to do the work, a function for which I am doomed to a copy-and-paste understanding.


  if (@is_uploaded_file($_FILES['upload']['tmp_name'][$i])) {

Next Me does a check to make sure that the file was actually uploaded and is not the result of malicious code. is_uploaded_file is a built-in PHP function. The preceding “@” suppresses error messages; I don’t know why Me did that, but I’m confident it was the right thing to do. While you are debugging, you might want to take the @ out.


move_uploaded_file($_FILES['upload']['tmp_name'][$i], $upload_file)

If the file passes that test, then Me moves it to the folder s/he designated. Note that we’ve already put the pathname to the storage folder into $upload_file.

Obviously, where the script has the comment “Great Success” you would put the code that does what you want to do to the uploaded file. I have added two lines — commented out — that get and print the content of the file; that gets ugly if the file isn’t some type of text file.

So, please let me know what I’ve gotten wrong here. (And maybe sometime I’ll give you some truly embarrassing code for returning the results of the PHP on the page that has the upload form on it.)

7 Comments »

August 22, 2012

When php will only read itself

Some little apps I’ve written for myself over the years refused to work on a Mac with Lion that I’ve started using at work. The problem was that any php invoked via Ajax would return only itself — that is, the return value would be the lines of the php script itself. This is not helpful.

I poked around a lot on the Net and couldn’t find anything relevant. The closest was the observation that this can happen if you begin your PHP scripts with just a <? instead of <?php, which seems awfully persnickety. Unfortunately, my scripts didn’t begin with the abbreviated form.

After getting some help by posting at Stackoverflow.com, it seemed that the machines that my scripts have worked on have PHP v.5.3.8, whereas my Lion computer for some reason was at PHP 5.3.1. So, I upgraded the php to 5.4. This page has easy instructions for doing so: http://php-osx.liip.ch. (If you want to know what version of PHP you have, in a terminal window type “php -v”.)

I’m not 100% sure that it was the PHP upgrade that did it and not some random other change or a change in the phase of the moon. But it’s working. And if you’ve been having the same weird problem with PHP, maybe it will work for you.

Be the first to comment »

July 27, 2012

Importing HTML into Google Docs spreadsheets

Rick Klau points [g+] to a feature of Google Docs spreadsheets I didn’t know about (although I’m far from a spreadsheet maven): It can automatically include a table from any HTML document accessible on the Web. It turns out it can also include the contents of lists.

It’s not the most intuitive feature. Into a cell you type:

=ImportHTML(“[URL]”,”[query]”,”[index]”)

Except you put in the HTML page’s url instead of [URL], “table” or “list” instead of [query], and which the number of the tables or list you have in mind instead of [index]. For example:

=ImportHTML(“http://www.hyperorg.com/blogger/index.html”,”list”,1)

gets the first list (ul or ol) on Joho The Blog (this page you’re reading), which turns out to be the one on the left called “Other Stuff.” If you ask for 2 instead of 1, you’ll get my blogroll.

Or, to use Rick’s more useful example:

=ImportHtml(“http://www.accuweather.com/en/us/anchorage-ak/99501/august-weather/346835?view=table”,”table”,1)

That imports AccuWeather’s table of weather for Anchorage (where Rick is headed for vacation.)

The data updates every time you open the spreadsheet.

ImportCVS does the same for CVS data. And Kingsley Idehen explains how you can update your spreadsheet with Linked Open Data by going through SPARQL. (SPARQL lets you query a database for linked data.) (Yes, it’s over my head.)

Wouldn’t it be useful to be able to import a single element into a Google spreadsheet, even if it’s not in a list or a table? For example, suppose I want to get the headline of the first posting at, say, DailyKos.com. That element has an id of “article-1″. (I know this because I looked at the source.) So, why not let me specify the url and the id, and plop the contents into a cell in the spreadsheet? Or suppose I want the content of one particular cell of a table?

No, we’re never satisfied.

 


Two seconds after I pressed the “Publish” button, Rick Klau responded to my questions on the Google Plus thread where he talks about this feature. He suggests importXML for grabbing an item by its id. And to get a frozen copy of the data, copy and paste it. He also points to a post from 2007 about these features. (Oh, yeah, you can trust Joho to stay on top of the news!) In fact, that post gives an example of how to obtain the latest headline from the NYT:

=GoogleReader (“http://graphics8.nytimes.com/services/xml/rss/nyt/HomePage.xml”, “items title”, “false”, 1)

It still works. Cool!!

8 Comments »

July 22, 2012

Capturing control keys in Chrome et al.

Hallelujah! For years — literally years — I’ve been limping along with a blender full of spaghetti to do something that should be really simple: capture control key combos (like CTRL-S or CTRL-I) via Javascript in all the major browsers. I finally found some simple code that seems to work beautifully.

The problem is that the browsers don’t agree about what’s going on when a user presses a control key and another key simultaneously, which is, after all, the usual thing people do with the control key. Some of the browsers think that it’s two events, so you have to record the control keypress, remember it, and treat the next keypress differently. Other browsers think of it as a single keypress that you can just process as a if CTRL-S were a unique key. Then, depending, you may or may not have to nullify the S press. The way I was doing it (cribbed from multiple sources, of course) involved first checking on which browser the Javascript was running in, and then process keystrokes, looking for an initial control press. Pain in the butt, and it was fragile.

I am certain that this is not a problem for actual developers. For example, jquery handles keystrokes, although I had trouble getting it to work (becaues, if it’s not clear, I am a ham-fisted hobbyist who mainly just copies in other people’s code. Thank you, other people!)

Today I had a a few minutes, so I went back to Google and found some simple code from Ganesh. Thank you!

Here’s what you do:

First, include jQuery. Place the following into your Javascript the following block. Put it toward the top, and don’t put inside a function. You want it to run whenever your Javascript loads. (Well, you could put it into an initialization function if you want.)

$.ctrl = function(key, callback, args) {
    $(document).keydown(function(e) {
        if(!args) args=[]; // IE barks when args is null 
        if(e.keyCode == key.charCodeAt(0) && e.ctrlKey) {
            callback.apply(this, args);
            return false;
        }
    });        
};

After that, you can bind a control combo, such as Control-S, to the function you want (e.g., “SaveMe()”) this way:

$.ctrl('S', function() {
    SaveMe();
});

By the way, Ganesh thanks an anonymous commenter for improving the code. God bless the iterative Web!

1 Comment »

May 20, 2012

Lion fixed my SuperDrive. (Alternative title: Snow Leopard broke my SuperDrive)

Yesterday Disk Utility told me to restart my Mac from a boot disk and run the disk repair function (= Disk Utility). Fine. Except I was unable to boot from any of my three Mac boot disks (including the original) whether they were in my laptop’s SuperDrive (= Apple’s plain old DVD drive) or in a USB-connected DVD drive. The system would notice the DVD when asked to look for boot devices (= hold down the Option key when starting up), but froze after I clicked on the DVD (= no change in the screen after 30 mins).

So, what the hell, I installed Lion, which I had been hoping to avoid (= my pathetic resistance to Apple’s creeping Big Brotherism). Thanks to the generosity of Guillaume Gète, I downloaded Lion DiskMaker, followed the simple instructions (= re-downloaded Lion, all part of Apple’s makings things hard by making them easy program), and now have a Lion boot disk. I was able to boot from it and fix my hard drive.

The whole episode was so reminiscent of why I left Windows (= Windows 7 looks pretty good these days).

3 Comments »

January 15, 2012

So you think you can scrape?

If you’re thinking about scraping a web page to extract the delicious data bits from it, ScraperWiki looks like a great place to start. It’s got tools, examples, and a community. Right now the tools are in Ruby, Python and PHP, but they’re thinking about adding Javascript.

If I have time this weekend, I’m going to give it a try scraping the weekly Berkman Buzz post. Until a couple of weeks ago, I was fairly routinely posting the Buzz on this blog, because I had written a little scraper and formatter that let me go from the email version to the blog markup I prefer. But then those bahstahds at Berkman went all HTML on the weekly email, which completely broke my scraper. But the Berkman page that lists the Buzz looks like it’s ripe for trying out the ScraperWiki tools. Looking forward to it…

3 Comments »

January 2, 2012

Fixing the quirky noQuirks blog template

Thanks to Mirek Sopek, the folks at Mako Lab diagnosed why my new WordPress blog template was going all wonky in Internet Explorer 9. Even after I’d discovered that the problem was that I was declaring the HTML page with Quirks, I’d put the type declaration in the wrong spot. I put it at the top of header.php, thinking that would put it at the top of the HTML page that WordPress assembles out of various files. Nope. You have to put it at the top of the index.php file. D’oh!

We still don’t know why it worked on my copy of IE 9, at the same version level and both 64-bit.

Thank you Mirek and Mako Lab! I would never have figured this out without you.

2 Comments »

December 30, 2011

Quirky html

In the recent — and probably unabated — unpleasantness attending the launch of the update of this blog’s look, I have learned a little about Quirks mode. I learned this because Internet Explorer 9 was not displaying rounded corners or laying out divs (blocks of content) the way Firefox and Chrome were. Once I switched off Quirks mode in my blog pages, it worked much better.

There’s a good explanation and some very detailed info here. But as I “understand” the story, quirks mode was introduced to handle the problem that different browsers were expecting different sorts of markup (particular for CSS style information). Then, once the browsers realized it would be helpful to everyone if they agreed to support truly standardized standards, they had to decide what to do with the old code written in the particularities of each browser. So, they agreed to allow the HTML developer to specify whether the page she’s created should be interpreted according to the modern standardized standards, or if it’s quirky and ought to be interpreted according to the idiosyncratic expectations of the various browsers.

You, the HTML developer, specify your intentions in the DTD declaration at the very beginning of your HTML pages. This page will help you figure out exactly how that line should read.

Meanwhile, the shame and humiliation the launch of the new look of this blog has brought upon me only deepens, for I have given up on controlling the placement of divs by getting my floats in order. Screw it. I’ve plunked them into a table. Yeah, I’m CSS-ing like it’s 1995.

2 Comments »

« Previous Page | Next Page »