Joho the Blogapi Archives - Joho the Blog

January 16, 2016

Getting the path from the Dropbox API

Suppose you’re using the Dropbox API to let a user choose a file from her Dropbox folder and open it in your application. Dropbox provides a convenient widget — the Chooser — you can more or less just drop into your Web page. But…suppose you want to find out the path of an item that a user opens. For example, you want to know not only that the user has opened “testfile.txt” but that it’s “Dropbox/testfolder/TestA/testfile.txt”. The chooser only tells you the link is something like:

https://www.dropbox.com/s/lry43krhdskl0bxeiv/testfile.txt?dl=1

Figuring out how to get that path information took me /for/ev/er. I know it shouldn’t have, but it did. So, here’s how I’m doing it. (As always, please try not to laugh at my efforts at coding. I am an amateur. I suck. Ok?) (I owe thanks to Andrew Twyman at Dropbox who went out of his way to help me. Thanks, Andrew! And none of this is his fault.)

The way to get the path is explained in Dropbox’s API documentation, but that documentation assumes I know more than I do. Dropbox also provides an API Explorer that lets you try out queries and shows you the code behind them. Very helpful, but not quite helpful enough for the likes of me, because I need to know what the actual PHP or JavaScript code is. (It’d be easier if I knew Python. Someday.)

So, here’s roughly how I got it working. I’m going to skip some of the preliminaries because I went through them in a prior post: how to register an app with Dropbox so you can embed the Dropbox Chooser that lets users browse their Dropbox folders and download a file.

That prior post included code that initializes the Chooser. I want to add a single line to it so we can get the pathname of the downloaded document:

1

var opts= {

2

success: function(files) {

 

3

var filename = files[0].link;

4

filename = filename.replace(“dl=0″,”dl=1”);

5

$(“#filenamediv”).text(files[0].name);

6

alert(filename);

7

$.ajax({

8

url: “./php/downloadDropboxContents2.php”,

9

data: {src : filename},

10

success: function(cont){

11

$(“#busy”).show();

12

openOpmlFile_File(filename);

13

$(“#busy”).hide();

14

setCookie(“lastfile”,”/php/currentFile/” + filename);

15

getDropboxPath(filename);

 

16

},

17

error: function(e){

18

alert(e.responseText);

19

}

20

});

 

21

},

22

multiselect: false,

23

extensions: [‘.opml’],

24

linkType: “download”

25

};

26

var button = Dropbox.createChooseButton(opts);

27

document.getElementById(“uploadDB”).appendChild(button);

When a user chooses a file from Chooser, the “success” function that starts on line 10 is invoked. That function is passed information about the files that have been opened by the user in an array, but since I’m only allowing users to open one file at a time, the information is always going to be in the first and only element of that array. That information includes something called “link,” which is a link to the file that does not include the path information. So, in line 15 — the only new line — we’re going to pass that link to a function that will get that elusive path.

1

function getDropboxPath(link){

2

$.ajax({

3

type: “POST”,

4

beforeSend: function (request)

5

{

6

request.setRequestHeader(“Content-Type”, “application/json”);

7

},

8

url: “https://api.dropboxapi.com/2/sharing/get_shared_link_metadata?authorization=Bearer [INSERT YOUR AUTHORIZATION CODE]”,

9

data: JSON.stringify({url : link}),

10

success: function(cont){

11

alert(cont.path_lower);

12

},

13

error: function(e){

14

alert(e.responseText);

15

}

16

});

17

}

This is another AJAX call; it too assumes that you’ve included jQuery. (See the prior post.)

Now, how does this work. Well, I’m not entirely sure. But it’s sending a request to the Dropbox API. It’s doing this as a standard http web call, which means (I think) that you have to include metadata that web servers expect when you’re using http. (I could be wrong about this.) So, in line 6 you tell it that you are expecting to get JSON back, not a standard Web page. (JSON is a standard way of encoding human-readable, multipart information.)

In line 8 you’re constructing the URL you’re going to send your request to. Everything up to the question mark is simply the URL of the Dropbox API for getting metadata about a link. After the question mark you’re telling it that you’re authorized to make this request, which requites getting an authorization code from Dropbox. I’m probably cheating by using the one that the API Explorer gives you, but it works for now so I’ll worry about that when it breaks, which will probably be the next time I use it. Anyway, you need to insert your authorization code where it says “insert your authorization code” in all caps.

Line 9: The data is the internal link that the Chooser gave you as the URL of the file the user downloaded. I use JSON.stringify because it didn’t work until I did.

Line 10 is what happens when your query works. You’ll get an object from Dropbox that contains several different pieces of info. You want the one called “path_lower,” presumably because it gives you the path that is lower on the great Tree of Files that is a Dropbox folder. [LATER THAT DAY: Andrew tells me it’s actually called path_lower because it’s the path in all lower case, which is useful because the Dropbox file system is case insensitive. Frankly, I prefer my explanation on poetic grounds, so we’ll have to agree to disagree :)] Line 11 gets that path (cont.path_lower) and pops it into an alert box, which is almost certainly not what you actually want to do with it. But this is a demo.

That’s it. If you have questions, try to find someone who understands this stuff because I got here through many trials and even more errors.

Good luck.

3 Comments »

October 26, 2015

[liveblog][act-tiac] A federal portal in the age of search?

Sarah Crane, Dir., Federal Citizen Information Center, GSA., is going to talk about USA.gov. “In a world where everyone can search and has apps, is a web portal relevant?,” she asks.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.


When the US web portal (first.gov [archival copy]) was launched in 2000, it had an important role in aggregating and centralizing content. Now people arrive through search.


USA.gov is a platform that offers a full suite of bilingual products, built around a single structured API: information, contacts, social media, etc. All is built around a single API. The presentation layer is independent of the content. Thanks to the API, all the different outputs use the same consistent content.


It’s designed to integrate with other agency content. In fact, they don’t want to be writing any of the content; it should come from other agencies. Also, it’s built to so its modules and content can be reused. And it’s built to scale. It can support expansion or consolidation. E.g., if an initiative loses steam, its content can be pulled in and be kept available.


How people use govt services: They look online, they ask their friends, and they expect it to be easy. People are surprised when it’s complex. Some people prefer in-person help.


So, how does the portal remain relevant?


Customer experience is a core tenant. They recently launched a Customer Experience division. Constant measurement of performance. Fixing what doesn’t work. Clear lines of reporting up to the senior management. The lines of reporting also reach all the way to the devs.


Last year they re-did their personas, based on four different behaviors: 1. Someone who knows exactly what s/he’s looking for. 2. Someone has a general idea, but not informed enough to search. 3. Someone wants to complete a transaction. 4. Someone who wants to contact an elected official. They analyzed the experiences, and did “journey maps”: how someone gets to what she wants. These journeys often include travels into other agencies, which they also mapped.


What’s next for them now that info is cheap and easy to find? Sarah likes Mint.com‘s model:


  • Aggregated, personalized content collected from multiple agencies.

  • Pre-emptive service – alert, etc.

  • Relevant updates as you are in the task.

For further info, see Blog.USA.gov, and USA.gov/Explore


Q&A

Q: [me] Are people building on top of your API?


A: Some aspects, yes. Heavily used: the A-Z agency index – the only complete listing of every agency and their contact info. There’s a submission to build a machine-readable org chart of the govt that will build on top of our platform. [OMG! That would be incredible! And what is happening to me that I’m excited about a machine-readable org chart?]


Also if you use bit.ly to shorten a gov’t url, it creates one.usa.gov which you can use to track twitter activity, etc.


Certain aspects of the API are being used heavily, primarily the ones that show a larger perspective.


Q: Won’t people find personal notifications from the govt creepy, even though they like it when it’s Mint or Amazon?


A: The band-aid solution is to make it opt-in. Also being transparent about the data, where it’s stored, etc. This can never be mandatory. The UK’s e-verify effort aims at making the top 20 services digital through a single ID. We’d have to study that carefully We’d have to engage with the privacy groups (eg., EPIC) early on.


Q: Suppose it was a hybrid of automated and manual? E.g., I tell the site I’m turning 62 and then it gives me the relevant info, as opposed to it noting from its data that I’m turning 62.


Q: We’re losing some of the personal contact. And who are you leaving behind?


A: Yes, some people want to talk in person. Our agency actually started in 1972 supplying human-staffed kiosks where people could ask questions. Zappos is a model: You can shop fully online, but people call their customer service because it’s so much fun. We’re thinking about prompting people if they want to chat with a live person.


The earliest adopters are likely to be the millennials, and they’re not the ones who need the services generally. But they talk with their parents.

 


 

I briefly interviewed Sarah afterwards. Among other things, I learned:



  • The platform was launched in July


  • They are finding awesome benefits to the API approach as an internal architecture: consistent and efficiently-created content deployed across multiple sites and devices; freedom to innovate at both the front and back end; a far more resilient system that will allow them to swap in a new CMS with barely a hiccup.


  • I mentioned NPR’s experience with moving to an API architecture, and she jumped in with  COPE (create once, publish everywhere) and has been talking with Dan Jacobson, among others. (I wrote about that here.)


  • She’s certainly aware of the “government as platform” approach, but says that that phrase and model is more direclty influential over at 18F


  • Sarah is awesome.

Comments Off on [liveblog][act-tiac] A federal portal in the age of search?

[liveblog][act-tiac] The nation's front door

Sarah Crane, Dir., Federal Citizen Information Center, GSA., is going to talk about USA.gov. “In a world where everyone can search and has apps, is a web portal relevant?,” she asks.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

When the US web portal (first.gov [archival copy]) was launched in 2000, it had an important role in aggregating and centralizing content. Now people arrive through search.

USA.gov is a platform that offers a full suite of bilingual products, built around a single structured API: information, contacts, social media, etc. All is built around a single API. The presentation layer is independent of the content. Thanks to the API, all the different outputs use the same consistent content.

It’s designed to integrate with other agency content. In fact, they don’t want to be writing any of the content; it should come from other agencies. Also, it’s built to so its modules and content can be reused. And it’s built to scale. It can support expansion or consolidation. E.g., if an initiative loses steam, its content can be pulled in and be kept available.

How people use govt services: They look online, they ask their friends, and they expect it to be easy. People are surprised when it’s complex. Some people prefer in-person help.

So, how does the portal remain relevant?

Customer experience is a core tenant. They recently launched a Customer Experience division. Constant measurement of performance. Fixing what doesn’t work. Clear lines of reporting up to the senior management. The lines of reporting also reach all the way to the devs.

Last year they re-did their personas, based on four different behaviors: 1. Someone who knows exactly what s/he’s looking for. 2. Someone has a general idea, but not informed enough to search. 3. Someone wants to complete a transaction. 4. Someone who wants to contact an elected official. They analyzed the experiences, and did “journey maps”: how someone gets to what she wants. These journeys often include travels into other agencies, which they also mapped.

What’s next for them now that info is cheap and easy to find? Sarah likes Mint.com‘s model:

  • Aggregated, personalized content collected from multiple agencies.

  • Pre-emptive service – alert, etc.

  • Relevant updates as you are in the task.

See Blog.USA.gov, and USA.gov/Explore

Q&A

Q: [me] Are people building on top of your API?

A: Some aspects, yes. Heavily used: the A-Z agency index – the only complete listing of every agency and their contact info. There’s a submission to build a machine-readable org chart of the govt that will build on top of our platform. [OMG! That would be incredible! And what is happening to me that I’m excited about a machine-readable org chart?]

Also if you use bit.ly to shorten a gov’t url, it creates one.usa.gov which you can use to track twitter activity, etc.

Certain aspects of the API are being used heavily, primarily the ones that show a larger perspective.

Q: Won’t people find personal notifications from the govt creepy, even though they like it when it’s Mint or Amazon?

A: The band-aid solution is to make it opt-in. Also being transparent about the data, where it’s stored, etc. This can never be mandatory. The UK’s e-verify effort aims at making the top 20 services digital through a single ID. We’d have to study that carefully We’d have to engage with the privacy groups (eg., EPIC) early on.

Q: Suppose it was a hybrid of automated and manual? E.g., I tell the site I’m turning 62 and then it gives me the relevant info, as opposed to it noting from its data that I’m turning 62.

Q: We’re losing some of the personal contact. And who are you leaving behind?

A: Yes, some people want to talk in person. Our agency actually started in 1972 supplying human-staffed kiosks where people could ask questions. Zappos is a model: You can shop fully online, but people call their customer service because it’s so much fun. We’re thinking about prompting people if they want to chat with a live person.

The earliest adopters are likely to be the millennials, and they’re not the ones who need the services generally. But they talk with their parents.

 


 

I briefly interviewed Sarah afterwards. Among other things, I learned:

  • The platform was launched in July.

  • The platform software is open source.

  • They are finding awesome benefits to the API approach as an internal architecture: consistent and efficiently-created content deployed across multiple sites and devices; freedom to innovate at both the front and back end; a far more resilient system that will allow them to swap in a new CMS with barely a hiccup.

  • I mentioned NPR’s experience with moving to an API architecture, and she jumped in with COPE (create once, publish everywhere) and has been talking with Dan Jacobson, among others. (I wrote about that here.)

  • She’s certainly aware of the “government as platform” approach, but says that that phrase and model is more directly influential over at 18F

  • Sarah is awesome. The people in government service these days!

Comments Off on [liveblog][act-tiac] The nation's front door

April 14, 2015

[shorenstein] Managing digital disruption in the newsroom

David Skok [twitter:dskok] is giving a Shorenstein Center lunchtime talk on managing digital disruption in the newsroom. He was the digital advisor to the editor of the Boston Globe. Today he was announced as the new managing editor of digital at the Globe. [Congrats!]

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

As a Nieman fellow David audited a class at the Harvard Business School taught by Clay Christensen, of “creative destruction” fame. This gave him the sense that whether or not newspapers will survive, journalism will. Companies can be disrupted, but for journalism it means that for every legacy publisher that’s disrupted, there are new entrants that enter at the low end and move up market. E.g., Toyota started off at the low end and ended up making Lexuses. David wrote an article with Christensen [this one?] that said that you may start with aggregation and cute kittens, but as you move up market you need higher quality journalism that brings in higher-value advertising. “So I came out of the project doubly motivated as a journalist,” but also wanting to hold off the narrative that there is an inevitability to the demise of newspapers.


He helped started GlobalNews.ca and got recruited for the Globe. There he held to the RPP model: the Resources, Process, and Priorities you put in place to help frame an organizational culture. It’s important for legacy publishers to see that it isn’t just tech that’s bringing down newspapers; the culture and foundational structure of those organizations are also to blame.

Priorities:
If you take away the Internet, a traditional news organization is a print factory line. The Internet tasks were typically taken up by the equivalent groups with in the org. Ultimately, the publisher’s job is how to generate profit, so s/he picks the paths that lead most directly to short-term returns. But that means user experience gets shuffled down, as does the ability of the creators to do “frictionless journalism.” On the Internet, I can write the best lead but if you can’t read it on your phone in 0.1 seconds, it doesn’t exist. The human experience has to be the most important thing. The consumer is the most important person in this whole transaction. How are we making sure that person is pleased?


In the past 18 months David has done a restructuring of the Globe online. He’s been the general mgr of Boston.com. Every Monday he meets with all the group leads, including the sales team (which he does not manage for ethical journalism reasons). This lets them set priorities not at the publisher level where they are driven by profit, but by user and producer experience. The conceit is that if they produce good user and producer experiences, the journalism will be better, and that will ultimately drive more revenue in advertising and subscriptions.


The Globe had a free site (Boston.com) and a paywall site (bostonglobe.com). This was set up before his time. Boston.com relative to its size as a website business has a remarkable amount of revenue via advertising. BostonGlobe.com is a really healthy digital subscription business. It has more subscriptions in North America outside of the NYT and WSJ. These are separate businesses that had been smushed together. So David split them up.

Processes:

They’ve done a lot to change their newsroom processors. Engineers are now in the newsroom. They use agile processes. The newsroom is moving toward an 18-24 hour cycle as opposed to the print cycle.


We do three types of journalism on our sites:


1. Digital first — the “bloggy stuff.” How do we add something new to those conversations that provides the Globe’s unique perspective? We don’t want to be writing about things simply because everyone else is. We want to bring something new to it. We have three digital first writers.


2. The news of the day. We do a good job with this, as demonstrated during the Marathon bombing.


3. Enterprise stuff — long investigations, etc. Those stories get incredible engagement. “It’s heartening.” They’re experimenting with release schedules: how do you maximize the exposure of a piece?

Resources:
In terms of resources: We’re looking at our content management system (CMS). Ezra Klein went to Vox in part because of their CMS. You need a CMS that gives reporters what they need and want. We also need better realtime analytics.


Priorities, Processes + Resources = organizational culture.

Q&A
Q: You’re optimistic…?


A: We’re now entering the third generation of journalism on line. First: [missed it]. Second: SEO. Third: the social phase, the network effect. How are we engaging our readers so that they feel responsible to help us succeed? We’re not in the business of selling impressions [=page views, etc.] but experiences. E.g., we have a bracket competition (“Munch Madness“) for restaurant reviews. We tell advertisers that you’re getting not just views but experiences.


Q: [alex jones] And these revenues are enough to enable the Globe to continue…?


A: It would be foolish of me to say yes, but …


Q: [alex jones] How does the Globe attract an audience that’s excited but civil?


A: Part of it is thinking about new ways of doing journalism. E.g., for the Tsarnaev trial, we created cards that appear on every page that give you a synopsis of the day’s news and all the witnesses and evidence online. We made those cards available to any publisher who wanted them. They’re embeddable. We reached out to every publisher in New England that can’t cover it in the depth that the Globe can” and offered it to them for free. “We didn’t get as much uptake as we’d like,” perhaps because the competitive juices are still flowing.


Then there are the comments. When news orgs first put comments on their site, they thought about them as digital letters to the editor. Comments serve another purpose: they are a product and platform in and of themselves where your community can talk about your product. They’re not really tied to the article. Some comments “make me weep because they’re so beautiful.”


Q: As journalists are being asked to do much more, what do you think about the pay scale declining?


A: I can’t speak for the industry. The Globe pays competitively. We’re creating jobs now. And there are so many more outlets out there that didn’t exist five years ago. Journalists today aren’t just writers. They’re sw engineers, designers, etc.


I’m increasingly concerned about the lack of women engineers entering the field. Newspapers have as much responsibility as any other industry to address this issue.


Q: How to monetize aggregators?


A: If we were to try to go to every org that aggregates us, it’d be a fulltime job. We released a story online on a Feb. afternoon about Jeb Bush at Andover. [This one?] By Friday night, it was all over. I don’t view it as a threat. We have a meter. My job is to make sure that our reporting is good enough that you’ll use your credit card and sign up. I’m in awe in the number of people who sign up every day. We have churn issues as does everyone, but the meter business has been a success.


Q: [me] As you redo your CMS, have you thought about putting in an API? If so, would you consider opening it to the public?


A: When I’ve opened up API sets, there has been minimal takeup.


Q: What other newspapers are doing a good job addressing digital issues? And does the ownership structure matter?


A: The Washington Post, and they have a very similar ownership structure as the Globe.


Q: [alex] What’s Bezo’s effect on the WaPo?


A: Having the Post appear on every Kindle is something we’d all like for ourselves.


Q: Release schedule?


A: Our newsroom’s phenomenal editors are recognizing and believing that we are not a platform-specific business. We find only one in four of our print subscribers logged on to the web site with any frequency. We have two different audiences.We’ve had no evidence that releasing stories earlier on digital cannibalizes our print business. I love print. But when I get the Sunday edition, I feel guilty if I recycle it before I’ve read it all. So why not give people the opportunity to read it when they want? If it’s ready on a Wed., let them read it on Wed. Different platforms have different reader habits.

Q: What’s native to the print version?

A: Some of the enterprise reporting perhaps. But it’s more obvious in format issues. E.g., the print showed the 30 charges Tsarnaev was charged with. It had an emotional impact that digital did not.


Q: Is your print audience entirely over the age of 50?


A: No. It’s a little older than our overall numbers, but not that much.


Q: What are you doing to reduce the churn rate? What’s worked on getting print and digital folks to understand each other?


A: I’m a firm believer in data. We’re not pushing for digital change because we want to but because data backs up our claims. About frictionlessness: It’s so easy to buy goods. Uber. Even buying a necklace. We’re working with a backend database that is complex. We have to tie that into our digital product. The front end complexities on how users can pay come from the complexity of the back end.


Q: [nick sinai] I appreciate your comments about bringing designers, developers, UX into the newsroom. That’s what we’re trying to do in the govt. for digital services. How about data journalism.


A: Data journalism lets you tell stories you didn’t know where there. My one issue: We’ve reached a barrier: we’re reliant on what datasets are available.


Q: How many reporters work for print, Boston.com, and BostonGlobe.com


A: 250 journalists or so work for the Globe and they all work for all platforms.


Q: Are different devices attracting different stories? E.g., a long enterprise story may do better on particular devices. Where is contradiction, nuance, subtlety in this environment? How much is constrained by the device?


A: Yes, there are form-specific things. But there are also social-specific things. If you’re coming from Reddit, your behavior is different from your behavior coming from Facebook, etc. Each provides its own unique expectation of the reader. We’re trying to figure out how to be smarter in detecting where you’re coming from and what assets we should serve up to you. E.g., if you’re coming from Reddit and are going back to talk about the article, maybe you’re never going to subscribe, but could we provide a FB Like button, etc.?


Q: Analytics?


A: The most important metric for me is journalistic impact. That’s hard to measure. Sheer number? The three legislators who can change a law? More broadly: At the top of the funnel, it’s how to grow our audience: page views, shares, unique visitors, etc. As you get deeper into the funnel it’s about how much you engage with the site: bounce rate, path, page views per visit,time spent, etc. Third metric: return frequency. If you had a really good experience, did you come back: return visits, subscribers, etc.


[Really informative talk.]

2 Comments »

November 18, 2010

[defrag] Jud Valeski

Jud Valeski of Gnip is talking about the rise of APIs. There’s ben a doubling of publicly available API’s in the past few years, Jud says. He shows Wired’s “The Web is dead” chart that shows the proportion of bits moving through the tubes. But, API usage shows the Web is not dead.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

He divides APIs into two buckets. Functional APIs make periodic calls for bits of information, and are heavily cacheable because there’s static data on the other side. Most of the problems are solved: REST has won, along with Curl.

Volume APIs are a different matter. Call frequency and throughput are high, “and things get wonky.” The call characteristics change. Local programming challenges fall out into the network and cause problems, e.g. queuing.

We don’t yet know how to deal with SLAs (service level agreements). Open network toopology APIs don’t have clear SLAs.

He minimizes the shock by leveragng best practices, finding comonoality in the frameowkr (mashery.com, apigee.com, gnip.com), builing APIs that set the standard.

Comments Off on [defrag] Jud Valeski