Joho the Blog » journalism

November 15, 2013

[liveblog] Noam Chomsky and Bart Gellman at Engaging Data

I’m at the Engaging Data 2013conference where Noam Chomsky and Pulitzer Prize winner (twice!) Barton Gellman are going to talk about Big Data in the Snowden Age, moderated by Ludwig Siegele of the Economist. (Gellman is one of the three people Snowden vouchsafed his documents with.) The conference aims at having us rethink how we use Big Data and how it’s used.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

LS: Prof. Chomsky, what’s your next book about?

NC: Philosophy of mind and language. I’ve been writing articles that are pretty skeptical about Big Data. [Please read the orange disclaimer: I’m paraphrasing and making errors of every sort.]

LS: You’ve said that Big Data is for people who want to do the easy stuff. But shouldn’t you be thrilled as a linguist?

NC: When I got to MIT at 1955, I was hired to work on a machine translation program. But I refused to work on it. “The only way to deal with machine translation at the current stage of understanding was by brute force, which after 30-40 years is how it’s being done.” A principled understanding based on human cognition is far off. Machine translation is useful but you learn precisely nothing about human thought, cognition, language, anything else from it. I use the Internet. Glad to have it. It’s easier to push some buttons on your desk than to walk across the street to use the library. But the transition from no libraries to libraries was vastly greater than the transition from librarites to Internet. [Cool idea and great phrase! But I think I disagree. It depends.] We can find lots of data; the problem is understanding it. And a lot of data around us go through a filter so it doesn’t reach us. E.g., the foreign press reports that Wikileaks released a chapter about the secret TPP (Trans Pacific Partnership). It was front page news in Australia and Europe. You can learn about it on the Net but it’s not news. The chapter was on Intellectual Property rights, which means higher prices for less access to pharmaceuticals, and rams through what SOPA tried to do, restricting use of the Net and access to data.

LS: For you Big Data is useless?

NC: Big data is very useful. If you want to find out about biology, e.g. But why no news about TPP? As Sam Huntington said, power remains strongest in the dark. [approximate] We should be aware of the long history of surveillance.

LS: Bart, as a journalist what do you make of Big Data?

BG: It’s extraordinarily valuable, especially in combination with shoe-leather, person-to-person reporting. E.g., a colleague used traditional reporting skills to get the entire data set of applicants for presidential pardons. Took a sample. More reporting. Used standard analytics techniques to find that white people are 4x more likely to get pardons, that campaign contributors are also more likely. It would be likely in urban planning [which is Senseable City Labs’ remit]. But all this leads to more surveillance. E.g., I could make the case that if I had full data about everyone’s calls, I could do some significant reporting, but that wouldn’t justify it. We’ve failed to have the debate we need because of the claim of secrecy by the institutions in power. We become more transparent to the gov’t and to commercial entities while they become more opaque to us.

LS: Does the availability of Big Data and the Internet automatically mean we’ll get surveillance? Were you surprised by the Snowden revelations>

NC: I was surprised at the scale, but it’s been going on for 100 years. We need to read history. E.g., the counter-insurgency “pacification” of the Philippines by the US. See the book by McCoy [maybe this. The operation used the most sophisticated tech at the time to get info about the population to control and undermine them. That tech was immediately used by the US and Britain to control their own populations, .g., Woodrow Wilson’s Red Scare. Any system of power — the state, Google, Amazon — will use the best available tech to control, dominate, and maximize their power. And they’ll want to do it in secret. Assange, Snowden and Manning, and Ellsberg before them, are doing the duty of citizens.

BG: I’m surprised how far you can get into this discussion without assuming bad faith on the part of the government. For the most part what’s happening is that these security institutions genuinely believe most of the time that what they’re doing is protecting us from big threats that we don’t understand. The opposition comes when they don’t want you to know what they’re doing because they’re afraid you’d call it off if you knew. Keith Alexander said that he wishes that he could bring all Americans into this huddle, but then all the bad guys would know. True, but he’s also worried that we won’t like the plays he’s calling.

LS: Bruce Schneier says that the NSA is copying what Google and Yahoo, etc. are doing. If the tech leads to snooping, what can we do about it?

NC: Govts have been doing this for a century, using the best tech they had. I’m sure Gen. Alexander believes what he’s saying, but if you interviewed the Stasi, they would have said the same thing. Russian archives show that these monstrous thugs were talking very passionately to one another about defending democracy in Eastern Europe from the fascist threat coming from the West. Forty years ago, RAND released Japanese docs about the invasion of China, showing that the Japanese had heavenly intentions. They believed everything they were saying. I believe these are universals. We’d probably find it for Genghis Khan as well. I have yet to find any system of power that thought it was doing the wrong thing. They justify what they’re doing for the noblest of objectives, and they believe it. The CEOs of corporations as well. People find ways of justifying things. That’s why you should be extremely cautious when you hear an appeal to security. It literally carries no information, even in the technical sense: it’s completely predictable and thus carries no info. I don’t doubt that the US security folks believe it, but it is without meaning. The Nazis had their own internal justifications.

BG: The capacity to rationalize may be universal, but you’ll take the conversation off track if you compare what’s happening here to the Stasi. The Stasi were blackmailing people, jailing them, preventing dissent. As a journalist I’d be very happy to find that our govt is spying on NGOs or using this power for corrupt self-enriching purposes.

NC: I completely agree with that, but that’s not the point: The same appeal is made in the most monstrous of circumstances. The freedom we’ve won sharply restricts state power to control and dominate, but they’ll do whatever they can, and they’ll use the same appeals that monstrous systems do.

LS: Aren’t we all complicit? We use the same tech. E.g., Prof. Chomsky, you’re the father of natural language processing, which is used by the NSA.

NC: We’re more complicit because we let them do it. In this country we’re very free, so we have more responsibility to try to control our govt. If we do not expose the plea of security and separate out the parts that might be valid from the vast amount that’s not valid, then we’re complicit because we have the oppty and the freedom.

LS: Does it bug you that the NSA uses your research?

NC: To some extent, but you can’t control that. Systems of power will use whatever is available to them. E.g., they use the Internet, much of which was developed right here at MIT by scientists who wanted to communicate freely. You can’t prevent the powers from using it for bad goals.

BG: Yes, if you use a free online service, you’re the product. But if you use a for-pay service, you’re still the product. My phone tracks me and my social network. I’m paying Verizon about $1,000/year for the service, and VZ is now collecting and selling my info. The NSA couldn’t do its job as well if the commercial entities weren’t collecting and selling personal data. The NSA has been tapping into the links between their data centers. Google is racing to fix this, but a cynical way of putting this is that Google is saying “No one gets to spy on our customers except us.”

LS: Is there a way to solve this?

BG: I have great faith that transparency will enable the development of good policy. The more we know, the more we can design policies to keep power in place. Before this, you couldn’t shop for privacy. Now a free market for privacy is developing as the providers now are telling us more about what they’re doing. Transparency allows legislation and regulation to be debated. The House Repubs came within 8 votes of prohibiting call data collection, which would have been unthinkable before Snowden. And there’s hope in the judiciary.

NC: We can do much more than transparency. We can make use of the available info to prevent surveillance. E.g., we can demand the defeat of TPP. And now hardware in computers is being designed to detect your every keystroke, leading some Americans to be wary of Chinese-made computers, but the US manufacturers are probably doing it better. And manufacturers for years have been trying to dsign fly-sized drones to collect info; that’ll be around soon. Drones are a perfect device for terrorists. We can learn about this and do something about it. We don’t have to wait until it’s exposed by Wikileaks. It’s right there in mainstream journals.

LS: Are you calling for a political movement?

NC: Yes. We’re going to need mass action.

BG: A few months ago I noticed a small gray box with an EPA logo on it outside my apartment in NYC. It monitors energy usage, useful to preventing brown outs. But it measures down to the apartment level, which could be useful to the police trying to establish your personal patterns. There’s no legislation or judicial review of the use of this data. We can’t turn back the clock. We can try to draw boundaries, and then have sufficient openness so that we can tell if they’ve crossed those boundaries.

LS: Bart, how do you manage the flow of info from Snowden?

BG: Snowden does not manage the release of the data. He gave it to three journalists and asked us to use your best judgment — he asked us to correct for his bias about what the most important stories are — and to avoid direct damage to security. The documents are difficult. They’re often incomplete and can be hard to interpret.

Q&A

Q: What would be a first step in forming a popular movement?

NC: Same as always. E.g., the women’s movement began in the 1960s (at least in the modern movement) with consciousness-raising groups.

Q: Where do we draw the line between transparency and privacy, given that we have real enemies?

BG: First you have to acknowledge that there is a line. There are dangerous people who want to do dangerous things, and some of these tools are helpful in preventing that. I’ve been looking for stories that elucidate big policy decisions without giving away specifics that would harm legitimate action.

Q: Have you changed the tools you use?

BG: Yes. I keep notes encrypted. I’ve learn to use the tools for anonymous communication. But I can’t go off the grid and be a journalist, so I’ve accepted certain trade-offs. I’m working much less efficiently than I used to. E.g., I sometimes use computers that have never touched the Net.

Q: In the women’s movement, at least 50% of the population stood to benefit. But probably a large majority of today’s population would exchange their freedom for convenience.

NC: The trade-off is presented as being for security. But if you read the documents, the security issue is how to keep the govt secure from its citizens. E.g., Ellsberg kept a volume of the Pentagon Papers secret to avoid affecting the Vietnam negotiations, although I thought the volume really only would have embarrassed the govt. Security is in fact not a high priority for govts. The US govt is now involved in the greatest global terrorist campaign that has ever been carried out: the drone campaign. Large regions of the world are now being terrorized. If you don’t know if the guy across the street is about to be blown away, along with everyone around, you’re terrorized. Every time you kill an Al Qaeda terrorist, you create 40 more. It’s just not a concern to the govt. In 1950, the US had incomparable security; there was only one potential threat: the creation of ICBM’s with nuclear warheads. We could have entered into a treaty with Russia to ban them. See McGeorge Bundy’s history. It says that he was unable to find a single paper, even a draft, suggesting that we do something to try to ban this threat of total instantaneous destruction. E.g., Reagan tested Russian nuclear defenses that could have led to horrible consequences. Those are the real security threats. And it’s true not just of the United States.

1 Comment »

October 23, 2013

[2b2k] Does the Net make us stoopid?

Yesterday I participated as a color commentator in a 90 minute debate between Clive Thompson [twitter:pomeranian99] and Steve Easterbrook [twitter:smeasterbrook], put on by the CBC’s Q program.The topic was “Does the Net Make Us Smart or Stupid?” It airs today, and you can hear it here.

It was a really good discussion between Clive and Steve, without any of the trumped up argumentativeness that too often mars this type of public conversation. It was, of course, too short, but with a topic like this, we want it to bust its bounds, don’t we?

My participation was minimal, but that’s why we have blogs, right? So, here are two points I would have liked to pursue further.

First, if we’re going to ask if the Net makes us smart or stupid, we have to ask who we’re talking about. More exactly, who in what roles? So, I’d say that the Net’s made me stupider in that I spend more of my time chasing down trivialities. I know more about Miley Cyrus than I would have in the old days. Now I find that I’m interested in the Miley Phenomenon — the media’s treatment, the role of celebrity, the sexualization of everything, etc. &#8212 whereas before I would never have felt it worth a trip to the library or the purchase of an issue of Tiger Beat or whatever. (Let me be clear: I’m not that interested. But that’s the point: it’s all now just a click away.)

On the other hand, if you ask if the Net has made scholars and experts smarter, I think the answer has to be an almost unmitigated yes. Find me a scholar or expert who would turn off the Net when pursuing her topic. All discussions of whether the Net makes us smarter I think should begin by considering those who are in the business of being smart, as we all are at some points during the day.

Now, that’s not really as clear a distinction as I’d like. It’s possible to argue that the Net’s made experts stupider because it’s enabled people to become instant “experts” on topics. (Hat tip to Visiona-ary [twitter:0penCV] who independently raised this on Twitter.) We can delude ourselves into thinking we’re experts because we’ve skimmed the Wikipedia article or read an undergrad’s C- post about it. But is it really a bad thing that we can now get a quick gulp of knowledge in a field that we haven’t studied and probably never will study in depth? Only if we don’t recognize that we are just skimmers. At that point we find ourselves seriously arguing with a physicist about information’s behavior at the event horizon of a black hole as if we actually knew what we were talking about. Or, worse, we find ourselves disregarding our physician’s advice because we read something on the Internet. Humility is 95% of knowledge.

Here’s a place where learning some of the skills of journalists would be helpful for us all. (See Dan Gillmor‘s MediActive for more on this.) After all, the primary skill of a particular class of journalists is their ability to speak for experts in a field in which the journalist is not her/himself expert. Journalists, however, know how to figure out who to consult, and don’t confuse themselves with experts themselves. Modern media literacy means learning some of the skills and all of the humility of good journalists.

Second, Clive Thompson made the excellent and hugely important point that knowledge is now becoming public. In the radio show, I tried to elaborate on that in a way that I’m confident Clive already agrees with by saying that it’s not just public, it’s social, and not just social, but networked. Jian Ghomeshi, the host, raised the question of misinformation on the Net by pointing to Reddit‘s misidentification of one of the Boston bombers. He even played a touching and troubling clip by the innocent person’s brother talking about the permanent damage this did to the family. Now, every time you look up “Sunil Tripathi” on the Web, you’ll see him misidentified as a suspect in the bombing.

I responded ineffectively by pointing to Judith Miller’s year of misreporting for the NY Times that helped move us into a war, to make the point that all media are error prone. Clive did a better job by citing a researcher who fact checked an entire issue of a newspaper and uncovered a plethora of errors (mainly small, I assume) that were never corrected and that are preserved forever in the digital edition of that paper.

But I didn’t get a chance to say the thing that I think matters more. So, go ahead and google “Sunil Tripathi”. You will have to work at finding anything that identifies him as the Boston Bomber. Instead, the results are about his being wrongly identified, and about his suicide (which apparently occurred before the false accusations were made).

None of this excuses the exuberantly irresponsible way a subreddit (i.e., a topic-based discussion) at Reddit accused him. And it’s easy to imagine a case in which such a horrible mistake could have driven someone to suicide. But that’s not my point. My point here is twofold.

First, the idea that false ideas once published on the Net continue forever uncorrected is not always the case. If we’re taking as our example ideas that are clearly wrong and are important, the corrections will usually be more obvious and available to us than in the prior media ecology. (That doesn’t relieve us of the responsibility of getting facts right in the first place.)

Second, this is why I keep insisting that knowledge now lives in networks the way it used to live in books or newspapers. You get the truth not in any single chunk but in the web of chunks that are arguing, correcting, and arguing about the corrections. This, however, means that knowledge is an argument, or a conversation, or is more like the webs of contention that characterize the field of living scholarship. There was an advantage to the old ecosystem in which there was a known path to authoritative opinions, but there were problems with that old system as well.

That’s why it irks me to take any one failure, such as the attempt to crowdsource the identification of the Boston murderers, as a trump card in the argument the Net makes us stupider. To do so is to confuse the Net with an aggregation of public utterances. That misses the transformative character of the networking of knowledge. The Net’s essential character is that it’s a network, that it’s connected. We therefore have to look at the network that arose around those tragically wrong accusations.

So, search for Sunil Tripathi at Reddit.com and you will find a list of discussions at Reddit about how wrong the accusation was, how ill-suited Reddit is for such investigations, and how the ethos and culture of Reddit led to the confident condemning of an innocent person. That network of discussion &#8212 which obviously extends far beyond Reddit’s borders &#8212 is the real phenomenon…”real” in the sense that the accusations themselves arose from a network and were very quickly absorbed into a web of correction, introspection, and contextualization.

The network is the primary unit of knowledge now. For better and for worse.

3 Comments »

October 10, 2013

[2b2k] Erik Martin on Reddit and journalism

Erik Martin is giving a talk at the Nieman Foundation. He’s the general manager of Reddit.com. (Disclosure: We’re friendly.) He tells us that Reddit gets 5 billion page views per month, and 70 million unique visitors.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

Erik gives us a tour and some background. Every morning he clicks on the “Random” button and visits the subreddits (= topically-based pages within the site) the button gives him. He does so now, hitting subreddits such as bitch, i’m a bus, ukele, battlestations (office desks), and what’s this plant. Reddit, he says, is like a giant message board. You can create a board (subreddit) about anything. There are over 100,000 that get at least a post a day, and 6,000 that have substantial activity. All the subreddits are created by users, who also can create the page design. All the posts are voted up or down by users. Users also set the rules for subreddits. For example, at the Coversong subreddit, users have apparently decided all posts have to be videos.

Now he’s interviewed by Justin Ellis.

JE: How did you get to Reddit?

EM: He worked for Mammoth Records. It got bought by Disney. Then hecame a documentary filmmaker. Then marketing films and distributing them online. He read Hackers and Painters by Paul Graham) [great book]. He then read about Paul Graham’s Y Combinator incubator. He applied to do a documentary about it, but was rejected. Still, he was hooked. Reddit came out of the first round of projects. He saw Reddit and loved the unpredictability of it. “Every link as a rabbit hole you might go down.” He got to know the cofounders and said “IU want to find a way to work with Reddit because that’s what I’m doing with all my time.” Alexis Ohanian asked him to work on a TV pilot that was going to incorporate Reddit into a news show. But it didn’t work; the Internet part was an add-on. Then he got hired as a community manager at Reddit.

JE: Reddit has a lot of geography. What does it mean to be a community manager?

EM: He looked at it as being the manager of a band. He’d promote promising items. He’d try to keep things functioning. And he tried to make sure that the community didn’t get taken advantage of, e.g., when people didn’t link back to Reddit.

JE: When you create a subreddit and a crowd shows up, how does that happen?

EM: Sometimes it’s obvious why. But others we can’t figure it out. One of our most popular subreddits is Explain Like I’m Five. That one you know what you’re going to get. Same for Ask Me Anything. Those explode when hot topics arise.

JE: How does this community stay together so long?

EM: Some of it is the customization of subreddits.

JE: Because anyone can create a subreddit, Reddit has gotten into trouble from time to time. There have been some very creepy subreddits. What’s the guiding principle for what is allowable?

EM: Our philosophy is that it’s a site that has 5B page views, and we have 35 employees [so we can’t moderate everything]. If you’re going to function you have to have some rules, but they have to be relatively finite, relatively easy to understand, and relatively self-enforceable. So, we have six rules. We have added one or two throughout the years. We try to keep them simple. No spam. You can’t try to break the site. You can’t try to cheat. You can’t put people’s personal info up. You can’t have anything illegal. We added that you can’t have material that sexualizes minors. If we had one that said “Don’t be a jerk,” it wouldn’t be enfrceable. No one would agree about how it applies. So there’s tons of stuff on the site that we find horrible and offensive, but the site works best when we keep it open and governed by those simple rules.

JE: What responsibility do you think you have if you see something that you personally feel is wrong?

EM: What I find offensive is different from others around the world or other positions. People don’t come here because they think we have the best judgment about what’s offensive. Plus, you have all the context. E.g., people complain about the PicsOfDeadChildren subreddit. That’s obviously very offensive. But what if it were called “Child Autopsy Photos” and it put itself forward as presenting medical training photos. Or a subreddit about death. Or a subreddit about combat video. It’s beyond offensive. It’s people being killed. It gets very tricky.

JE: There have been 3 major stories illustrative of Reddit and citizen journalism: The Aurora movie theater shooting, the Boston Marathon bombing, and the shooting at the Navy Yard in DC. In the first, there was first person reporting. With the second, there was that but also the spreading of info from elsewhere and then the misidentification of one of the suspects in the bombing. With the third, someone created a subreddit to investigate what was happening, but you guys shut that down. What have you learned?

EM: In those three situations, the response of the community was the same as what you’d see offline: People trying to figure out what went on. Telling their story. Making jokes. Speculating about all kinds of things. Trying to make sense of what happened. Later on they were trying to help in some way. With Boston, it was different because the authorities wanted help from the public: they said if you have photos, upload them, etc. There was a subreddit where people were trying to identify the bombers, and that got a lot of attention. The actual subreddit where the Brown Univ. student was misidentified by name was actually the normal Boston subreddit, and it was removed after about an hour. That wasn’t good enough. That led to horrible consequences for that family.

So, what have we learned? We learned that people want to share, to talk, to help, to be a part of these huge events any way they can. We learned people can be callous and cavalier by mentioning people’s name. The vast majority were careful and thoughtful, but some were not. The Navy Yard subreddit was a joke. It had six posts, most from journalists satirizing the Boston bombing subreddit. It went against our rules and we shut it down after an hour.

JE: But you apologized after the Boston bombings…

EM: Absolutely. We do post-mortems and followsup. We did one when President Obama came on. So, yes, we apologized and talked aout what we can do better. And we also talked about the amazing things people did: people bringing their pets to parks in case people needed cute animal therapy, the sending of pizzas to EMTs and the police… We are an open source site in policy as well as code.

JE: Is it enough to do a post mortem? Newspapers issue corrections.

EM: There are thousands of subreddts, so there isn’t a way to reach everyone. We’re a platform, not a newspaper. We’re like Twitter or Youtube or WordPress. We don’t have a position on the veracity of one thing or another. I hope people learn to be more empathetic nandlearn that what you say on line has repercussions. But I don’t think we’re like a publication, and we’re not an editorial team.

JE: How do you see the role of journalism on Reddit? Why are people doing self-reporting?

EM: They want to be part of the story. They don’t want to be passie about what’s happening in the world. Even if
it’s uploading a meme. They’ve seen something start and then get big in a single day. Of course they want to share what’s happening in their neighborhood or share their thoughts about what’s going on in their govt Redditors vote 20M time a day.

JE: What’s the relation of journalisms and Reddit?

EM: We’re agnostic about what you’re linking to. But original reporting is more important than ever because people can find an audience. What’s happening on Reddit and what’s happening in the mainstream media happen to be in different hemispheres now but ultimately it’s the same thing. I hope people doing reporting will be active in a comment thread on Reddit or elsewhere.

JE: But you are creating content in some way, e.g., the Ask Me Anything’s where anyone can come in answer questions from the community. It’s very much like what media companies do.

EM: And in other Reddits people share recipes or workout routines. It’s like what you get in the media. It’s communicating, it’s story telling.

JE: How do you make money? You have ads and Reddit gold memberships.

EN: We don’t need to make a lot of money. We’re very lean. Our NY office is in a coworking space. We basically have ads for big movies, mobile phones, etc. We also have ads from mom and pop companies. Reddit Gold is a premium membership, $24.99/year. You get some extra features but most people do it to support the site. We have a secret Santa program (Reddit Gifts) that has an e-commerce site to help those exchanges and to make money.

JE: Reddit was purchased by Conde Nast and then spun off in 2011. How is it different?

EM: We started in 2005. Bought by Conde Nast in 2006. I started in 2008. Reddit was basically neglected by Conde: we were growing but there was a hiring freeze. OTOH, no one told us what to do. An example of how it made a difference: Before we were spun out, our ad operations was done through Conde, which is great for major magazines, not for a weird site where all you need is $5 to run an ad. So it didn’t make sense for us. We wanted an ad server that was fast and open source, which now we have.

Q&A

Q: Any trends in the type of content being produced? Trending toward the absurd? Or what?

A: It gets harder and harder to think about overall trends because the site is becoming more fractious and disparate each day. I think people are really motivated by the unexpected. Our audience is increasingly cynical. We also have an audience that is increasingly idealistic. You see trends were people are more connected across national and geographical boundaries; if there’s a discussion on healthcare the top comments will be from people around the globe. And it’s always been possible to have the serious next to the ridiculous; the last remaining bulkheads are being whittled away.

Q: Can you remain content agnostic?

A: No, it’s not possible. We’re not content agnostic towards spam or personal information. We try to be as close to agnosstic as we can.

Q: How much does porn account for your content?

A: About 85% of the subreddits are safe for work. (The Trees subreddit is not because you could get in trouble looking at pictures of weed.) Porn is maybe 5-10%. Our biggest subreddits are the video subreddits, As Reddit, etc.

Q: Terrorists radicalize by looking at pictures of dead babies. Have you had to hand over who your users are to agencies trying to track people on Reddit trying to radicalize people?

A: User privacy is core but we comply with what we have to comply with.

Q: [me] Reddit used to have a strong culture. People knew the same references, were playing the same games, had the same general politics, etc. But that shared culture seems to be weakening as Reddit becomes more popular. Does this concern you??

A: Yes, there is a certain sense of shared community that’s being fractured. But it’s being migrated down the subreddits the way you’re more loyal to community or borough.

Q: [me] Can you say more about IAMA’s, which at their best are a quite remarkable journalist form of collaborative interview?

A: The exciting thing for me is to see that format seep into other subreddits. We actively are trying to encourage that. E.g., mayoral candidates should do AMAs in their city’s subreddit. Or scifi authors are doing them in the sf subreddits. It goes back to that idea of so much of the word being predictable. If you waatch watch an interview on even some of the great programs — Charlie Rose, for example — even if they’re really good, you know what to expect. With the Reddit AMA’s not only do you not know what sort of questions are going to be asked, since you can answer a question at any length, it ends up taking this unexpected terms. If you look at the calendar of upcoming IAMA’s, you don’t even know which ones are going to be popular, outside of a Bill Gates or Tom Hanks, but if you look at the top AMAs for a week it will be a celebrity, subway driver, person with a weird disease, and way down the list will be someone with a household name. It’s unpredictable, and it’s unpredictable to the person being interviewed. It’s very different from what you get on a press junket where people go into robot mode. The AMA format can be more fun for them the standard press interview.

Q: Tumbler did a lot of active outreach to media. You don’t go out to, say, Newsweek and ask if they want a subreddit.

A: Yes. It’s difficult for us to do. Tech News Today is a great subreddit. They don’t directly flog their content. PBS has done one. But it’s hard.

Q: A newspaper could have its own subreddit where their folks are doing AMA’s etc.

A: Yes. But curating and cultivating a subreddit is a lot of work. It’s hard enough getting journalists to participate in comments on their own site.

Q: Companies you wouldn’t expect have made editorial plays. E.g., Twitter has being hiring editorial staff. Why are they doing that?

A: We’ve done some of that to prime the pump. E.g., Adam Savage’s publicist would probably say no to a request for an AMA at a site that looks like it’s from the 1990s [like ours], but if I go out with a camera and ask him to respond to the top ten questions, they might say yes. But then they see that the AMA works. So we only do editorial work for pump priming.

Q: What’s up with the design?

A: Look at the big sites. Minimal but flexible platforms. When you start doing a more professional and complex design, you suddenly needing 10x more people, and then you need 10x the money…But subreddits can monkey with the CSS. They can even change the Gold button, our “buy” button. Rich text works.

Q: For a traditional news org, the misidentification of the Boston Bomber would have been very expensive. Who owns the error from a legal perspective, in the US and elsewhere?

A: In the US, platforms are not responsible for what people say. The person who says it is responsible. I don’t know if Reddit could exist as a Canadian company. People give us a non-exclusive contract to display their words.

Q: But because you have some rules, doesn’t that make you responsible?

A: The more you monitor, the more responsible you are. But everything on the site is determined by human behavior. We are a platform for people discussing things. We’re not a publication. We don’t have editorial control.

Q: Is one of your 35 people a lawyer?

A: No.

Q: So when you get subpoenas…?

A: We’ve had to learn more than we want. We also have very good lawyers we consult with when we need to.

Q: The site in 5 years?

A: I don’t know. The users have better ideas than we do. All we try to do is take ideas they develop and help make them happen. So, in 5 years I think Reddit will be in more countries, more cross-country conversation. We have great engineers so we’ll be doing more interesting things. In 5 years I hope there will be 1,000 Reddit apps, using Reddit in novel ways that I couldn’t come up with. I never imagined that Reddit would be useful for live events. People are using our “edit” button 50/hour for this, which is not what the button is intended for, and Reddit’s not even very good at. People have created a site that reorganizes Reddit in chronological order and they can do that because we’re open source and don’t send lawyers after them. If we evolve in 5 yrs it will be because people in the community take it in those new directions.

Q: Venture capitalists?

A: Y-Combinator’s original investment was $20K. We were self-sustaining until Conde Nast bought us. We also had a very small angel round in the past year, around $1M. Very small. We’ve never spent a lot of money so we’ve never had to raise a lot. We’re close to break even now.

Q: Have any news events truly originated with Reddit?

A: As far as I know, one of the first reports on the Aurora story was from someone at the theater, before there was anything known to the media. The biggest story where Reddit was involved in the story was probably the SOPA/PIPA blackouts. Someone started to go after GoDaddy: “I’m moving 75 domains from GoDaddy” and it grew, and the next day GoDaddy flipped its position. Also, someone went after Paul Ryan and he ended up changing his mind.

Q: How can I troll Reddit for news stories?

A: When a new Android comes out, reporters go to Reddit to see what’s new in that version. I don’t know why more reporters don’t go to the relevant subreddits and ask for help on a story.

Q: We reporters are competitive.

A: In the sports world, you routinely see stories getting updated based upon information at Reddit.

Q: News orgs are trying to figure out how to engage with their audiences via social media. Advice?

A: Popular Science killed comments. Fine. You don’t have to have comments. But if you have them, you should pay attention to them. E.g., Roger Ebert would edit your comment as an admin, which is a terrible practice, but people didn’t mind because he was doing so to respond to their comments. I don’t understand why in general comments in 2013 are not all threaded and vote-able. Most are still in reverse chron, highlighting the latest. And most seem to be trying to hide their comments.

2 Comments »

September 10, 2013

Future of journalism

I’m at a Riptide forum at Harvard’s Shorenstein Center on the “digital disruption of the news.” The place is packed. Digital Riptide consists of 60 interviews. The panel discussion is with Tim Armstrong, AOL; Caroline Little, Newspaper Association of America; Arthur Sulzberger Jr., The New York Times. It’s moderated by Martin Nisenholtz.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

[I came in a few minutes late…]

Caroline Little: Audiences aren’t the problem. They’re growing. Revenue is the challenge.

Arthur Sulzberg: “We’re losing our first teeth and growing our new teeth. It’s painful. But what’s coming will be bigger” in reach and impact.

Q [Martin Nisenholtz]: Tim, you paid $315M for The Huffington Post. Jeff Bezos paid $250M for The Washington Post. Did Bezos get a better deal than you?

Tim Armstrong: No, Huffpo is worth more than people thought. It’s gone from 0 to 100M video views, for example. It’s got 100% digital DNA. I once owned a Boston newspaper, but saw Mosaic at MIT and knew I had to get out of that business. “I think I got a great deal with Huffpo and I think Jeff got a good deal on the WaPo, depending on what he does with it.”

Q: Caroline, you ran the WaPo digital division. Was it inevitable that they’d sell to Bezos or could they have done someting to change that future?

CL: It wasn’t inevitable. Now newspapers really have to understand their audiences. Taking it private will remove some pressure. The Grahams were trying to put the WaPo in the best possible place for the future, and that took courage.

Q: What is the nature of authority in a world where there are tens of thousands of verticalized publications on every conceivable topic?

AS: Authority is still about accuracy, breadth, calling out your own mistakes, and having experienced people on the ground, not parachuting them in. Few news organizations have bureaus around the world or even the country. The joy of the digital era is the speed and the reach, and the ability to take in points of view very quickly and bring them into a story line. It’s a remarkable opportunity. The downside is clear. Suddenly everyone is looking at the photo of the Boston bomber. Everyone knows it’s him. But it’s not. That kind of accuracy is critical. [I think AS thinks his characterization of authority bolsters the newspaper’s case, but I don’t think it does. I trust the specialized bloggers/sites on particular topics more than I trust the newspaper. E.g., I get far more in depth and more authoritative coverage of telecom policy from blogs and mailing lists than I do from the NYT. As for Reddit’s misidentification of the Boston bomber: Yes, it was a dreadful mistake. But it was done transparently in public and was immediately corrected. <cough>Judith Miller</cough>.]

Q: Tim, you were interviewed and spoke enthusiastically about Patch. Since then you’ve downsized it. What’s so hard about local journalism?

Tim: We rolled it out to 900 of the best GDP [I think] communities in the US. It was about the authoritative nature of local journalism. Patch’s expansion was rapid. This summer we realized there are 500 Patches that work, and 400 with traffic but we’re not part of that business model. Patch will continue to go one post-partnerships because there’s such an acute need for local info. We’ll probably do partnerships. AOL will own some, and traditional media companies will own some.

Caroline: 85% of all media coverage of stories starts with newspapers.

Q: There’s never been a large, truly international paper. What’s the model for going global with a US news brand?

AS: The International Herald Tribune is owned by the NYT. We’re re-branding it in October as the International NYT. This is just a step. We’re fixing things; e.g., we didn’t used to let you subscribe using Euro currency. But: I met with aa Chinese general. She began by talking in an angry way. We had just begun to charge for Web access, and every morning she’d go to the NYT site but it wouldn’t accept her credit card. We fixed it, but the point is that a Chinese general was going to the NYT site the first thing in the morning. What an opportunity!

Q: Young people don’t seem to be willing to pay for media on the Web. As they mature, will they be willing to pay to for a digital subscription to the NYT?

AS: More and more young people, and all people, are willing to pay for an experience they value on the Web. Thank you, Steve Jobs! But it’s not as if 14 yr olds were ever buying newspapers.

Q: Tim, you create some content, and you do deals to provide access to your audience. How do you decide what you’ll cover and what others will?

TA: Our theory is that people care about a limited amount of things. As they get older, they spend more time with things that matter. We want to be the most human-based company in terms of what people care about. E.g., we’re running TechCrunch Disrupt now. 3,000 people. Just about all the major figures in the tech space. This is an influencer space that we have a major major major amount of mindshare in. The second generation of our strategy is to build out massive content partnerships. And a giant B2B strategy; we service about 40K other publishers with video, ads and content-sharing. Our content theory: let’s invest in the most important areas of journalism and content, build B2B, and have relationships with people in the most important areas of their lives. HuffPo is an influencer. It’s a global info source. It’s a trusted brand. E.g., see our coverage of the selection of the new Pope.

Q&A

Q: Will reporters inevitably have less time to research a story?

AS: The pace has certainly picked up. But we’re still engaged in long form journalism. E.g., Snowfall.

TA: 30-40% of the traffic is on mobiles. Mobile adds consumption, typically about 30%. This changes how news works: If you don’t have a brand like the NYT, you’ll mix low appeal with high appeal

CL: Newsrooms are smaller, but they’d send 15 people to cover the Olympics. Do you really need that? Many newspapers now are investing in longer-form reporting.

Q: What makes global journalism work? The size of the city?

CL: Trust and authenticity. People still trust newspapers.

Q: What do you look for in journalists’ skills?

AS: We didn’t focus fast enough on hiring engineers to build systems and tools.

Q: TV broadcasters are using time efficiently. E.g., Netflix released an entire series at once. How are you experimenting with how to bundle and release investigative reporting?

AS: We’re always experimenting. E.g., We printed Snowfall as a full section, but the experience on the Web was so immensely powerful because of the video, the sound, etc.

TA: We look at how you disrupt readers’ behavior. I always say you can’t beat ESPN Sports Center by being 5% better. You have to be 75% better. Also you need distribution partnerships; that disrupts when your content is released.

Q: [couldn’t hear]

TA: If you look at how people use phones and tablets, the fact is that the average TV is 22 mins of content and 8 mins of commercial. When you watch how people use content, you’ll see trusted brands and faster content. [I had trouble hearing this.] People want to be told what they want to see. The future will be very curated and disruptively time-based.

Q: How do you view social media? Does the capacity to share overcome the limitations of 140 characters?

CL: I think Twitter is like a caption to a photograph. If it’s engaging, you’ll go find more about it.

AS: Twitter is a powerful tool, both in and out.

TA: Twitter is best known for short content, but they’re going to be changing that.

CL: Twitter is great for sources.

AS: This isn’t new. People said you can’t trust what people would say over a telephone wire. Telegrams were thought by some to be the death of newspapers. Twitter is a tool that we’re all getting better and better at using.

Q: From Twitter: How do you choose your political angle?

TA: We have a news chooser that lets you customize the news. HuffPo started more with a political angle. Now there are lots of forums set up for people with different political views. But we [AOL? HuffPo?] have our own voice.

AS: The Net is bringing us back to the written word. Radio, TV, telephone all took us away from that. We’re learning that using any single method will fail.

Q: Tim, you said “not just everyone can be a journalist.” How about bloggers who steal content?

TA: Anyone can be a journalist if they want to be. But consumers are smart. They know who’s stealing information and who isn’t. What you see happening in the blogging community are people taking advantage of situations to be disruptive to gain audience. I would not undercut the ability of people building blogs on specific topics disrupting newspapers.

Q: 5B people may be coming on line in the next decades. How does that change the target audience for online journalism?

The possibilities of our growth and the value of the quality of the info we can provide are immense.

TA: As the developing world comes online, they’ll come online with higher bandwidth.

1 Comment »

August 9, 2013

[2b2k] Can Bezos beat 1:25?

I am a big fan of Reddit, as a reader, an occasional participant, and as an observer. As a reader, Reddit has gone downhill for me. Or perhaps I should say “as a lazy reader.” I don’t stray much from the home page which shows the top posts from a default set of sub-reddits, i.e., topically clustered posts. These days, there’s usual one post among the 25 on the home page that I find interesting in a way that matters, although maybe a half dozen I find click-worthy. Those half dozen are usually memes, or discussions of something in pop or Internet culture. The one in 25 that matters to me introduces me to an idea I hadn’t considered, with a discussion that goes pretty deeply into it — while always laced with glancing sub-threads and banter. But for a page that can be quickly skimmed, a 1:25 ratio is enough to bring me back several times a day.

One in 25 is probably about the ratio I find in The New York Times when I come upon a printed copy of it. That ratio goes higher if you count the sections that I skip entirely. For example, I apparently entirely lack the sports gene. The articles I read are usually ones that offer an interesting viewpoint on a topic I already care about, or that for some unpredictable reason stimulate my interest in something I didn’t know I cared about. I know this is very different from the behavior I’m supposed to exhibit. As a responsible citizen, I should be reading all the articles the paper tells me are important. But that’s how I am, that’s how I’ve always been, and I think it’s the way that most of us were even during the decades when reading the newspaper every day was our civic duty.

So, it worries me that Jeff Bezos may bring to the Washington Post the theory of reading that he has brought to Amazon. Amazon’s personalization works very well for me. The books it suggests are often in fact very appealing to me. It’s one reason I keep going back to Amazon. The suggestions don’t often take me far afield, but books are such a big investment of time and money that I don’t intuitively react against that. Intellectually I react against it, but my intuition and the finger that clicks the “buy” button don’t seem to mind at all.

Besides, I read most books as a matter of recreation. (Actually, that’s entirely false. In terms of numbers, I read most books as research that’s dictated by whatever project I’m working on. But we’re talking here about discretionary reading.) And here the Washington Post is different. We need it to help us learn what we need to know to be better citizens in a world that is increasingly inhospitable. A newspaper that works like Amazon would be intentionally creating a filter bubble, in Eli Pariser’s phrase. (And Eli Pariser’s book by that name is thoroughly worth reading, especially if you follow it up with Ethan Zuckerman’s Rewire.)

Bezos has a tremendous opportunity with the Washington Post. He can choose to restructure it so that it becomes the first truly networked newspaper, retaining the traditional virtues of a great newspaper while opening it up to the new virtues of our global participatory network. It can become a uniquely well-webbed supplier of news to the networked ecology, although the idea that any newspaper can “cover” all the “major” news has long ago gone pining for the fjords.

But this new webby news platform will miss the big chance to improve the ecosystem if Bezos applies to the Washington Post what he knows about personalization. The world doesn’t need another way to have our beliefs confirmed and our interests titilated. We don’t need The Daily Everyone Sucks But Us, and we really really don’t need The Washington Post and Sideboob.

What we instead need is personalization that doesn’t pander to our interests but expands them. That requires starting from where we are; posting lots of articles that are so outside our interests that we won’t read them won’t help. But the genius of Amazon’s personalization can be tuned so that we are presented with what pushes our interests forward without abandoning them. There’s lots of room for improvement in my current 1:25 ratio. In fact, there’s a statistical possibility of a 24x improvement.

We have billions of dollars’ worth of evidence that Jeff Bezos is one of the great business entrepreneurs of our era. But we also have good evidence that he has interests beyond maximizing corporate value. His taking the Washington Post private is a very good sign. I’m hopeful that something very good for us all is going to come out of his purchase — but only if Bezos can unlearn much of what Amazon has taught him about how to succeed.

1 Comment »

July 22, 2013

Paid content needs REALLY BIG metadata

HBR.com has just put up a post of mine about some new guidelines for “paid content.” The guidelines come from the PR and marketing communications company Edelman, which creates and places paid content for its clients. (Please read the disclosure that takes up all of paragraph 4 of my post. Short version: Edelman paid for a day of consulting on the guidelines. And, no, that didn’t include me agreeing to write about the guidelines)

I just read the current issue of Wired (Aug.) and was hit by a particularly good example. This issue has a two-page spread on pp. 34-35 that features an info graphic that is stylistically indistinguishable from another info graphic on p. 55. The fact that the two pager is paid content is flagged only by a small Shell logo in the upper left and the words “Wired promotion” in gray text half the height of the “article’s” subhead. It’s just not enough.

Worse, once you figure out that it’s an ad, you start to react to legitimate articles with suspicion. Is the article on the very next page (p. 36) titled “Nerf aims for girls but hits boys too” also paid content? How about the interview with the stars of the new comedy “The World’s End”? And then there’s the article on p. 46 that seems to be nothing but a plug for coins from Kitco. The only reason to think it’s not an ad in disguise is that it mentions a second coin company, Metallium. That’s pretty subtle metadata. Even so, it crossed my mind that maybe the two companies pitched in to pay for the article.

That’s exactly the sort of thought a journal doesn’t want crossing its readers’ minds. The failure to plainly distinguish paid content from unpaid content can subvert the reader’s trust. While I understand the perilous straits of many publications, if they’re going to accept paid content (and that seems like a done deal), then this month’s Wired gives a good illustration of why it’s in their own interest to mark their paid content clearly, using a standardized set of terms, just as the Edelman guidelines suggest.

(And, yes, I am aware of the irony – at best – that my taking money from Edelman raises just the sort of trust issues that I’m decrying in poorly-marked paid content.)

3 Comments »

July 11, 2013

Apple e-books decision explained – mainstream and Reddit

A judge has ruled that Apple is guilty of price-fixing in its attempt to get the major publishers to unite against Amazon’s discounting of e-books.

Now, that’s not a very helpful — and possibly not entirely accurate — explanation. If you want more, there’s a thread at Reddit that has some terrific explanations at various level of detail (e.g., this one), as well as bunches of questions asked and answered. And, of course, some digressions, hip shots, and smug wrongnesses.

There are certainly some helpful analyses and explanations from the mainstream: e.g., WSJ, Wired, Bloomberg. In fact, I’d be hard-pressed to choose among those three and the Reddit comment I linked to above. But the Reddit thread is — at least to my taste — a better way to explore the issue: a variety of views expressed at appropriate lengths, with questions posed at various levels of sophistication, and with a conversation that goes where it wants to without a fear of dead ends.

Now, I’m aware that if you go to the Reddit thread, you’ll be appalled by how much there is wrong with it. Yeah, I’m not blind to it. But consider what an amazing emergent artifact that thread is. It combines in one flow “explainers” and analysis as good as you’ll find from professionals, Q&A, and a a social froth that you can easily ignore if it is not to your liking. This is what journalism looks like — one of the ways it looks — when the old constraints of space, authorial ownership, and editorial process are lifted, and a larger We gets our hands on it. Pretty fascinating.

5 Comments »

June 30, 2013

Are there journalists?

Jeff Jarvis [twitter:jeffjarvis] has a good post asking us to think about journalism as a service provided to a culture, rather than worrying about who is or is not a journalist:

Thanks to the Snowden-Greenwald NSA story, we are headed into another spate of debate about who is and isn’t a journalist. I’ve long said it’s the wrong question now that anyone can perform an act of journalism: a witness sharing news directly with the world; an expert explaining news without need of gatekeepers; a whistleblower opening up documents to sunlight; anyone informing everyone. It’s the wrong question when we reconsider journalism not as the manufacture of content but instead as a service whose goal is an informed public.

There are of course times when we have to make a decision about who is a journalist and who isn’t. For example, who do you let into a press conference? But almost always those are decisions forced by the limitations of the physical world, and it’s a shame to let those limitations drive our understanding. Jeff’s way is more useful.

Why more useful? First, because it can forestall a whole bunch of pointless arguments premised on the idea that there is a precise definition of “journalist” that we can establish, that’s useful, and that we can agree upon. That’s not how language works. Second, and more important, worrying about journalism-as-service rather than who is a journalist lets us think about how to build an ecosystem that enables the maximum amount of useful journalism no matter who is doing it. From my point of view, this is a way of thinking about how we can build better and better knowledge networks for understanding what’s happening now in the context of history and a pluralism of cultures.

Now, the very best such ecosystem well might have paid professional journalists in it. In fact, I’ll be shocked if it does not. But thinking about journalism-as-service (or as ecosystem or as network) seems to me to be the more inclusive frame.

2 Comments »

April 20, 2013

[2b2k] What we can learn from what we don’t know

I wrote a piece in the early afternoon yesterday about what we can learn from watching how we fill in the blanks when we don’t know stuff…in this case, when we don’t know much about Suspect #1 and #2. It’s about the narratives that shape our unserstanding.

For example, it turns out that I only have three Mass Murderer Narratives: Terrorist, Anti-Social, or Delusional. As we learned more about Suspect #2 yesterday, he seemed not to fit well into any of them. Perhaps he will once we know more, or perhaps my brain will cram him into one even if he doesn’t fit. Anyway, you can read the post at CNN.

 


I find myself unwilling to use Suspect #2’s name today because Martin Richard is too much with me.

Be the first to comment »

April 16, 2013

[misc][2b2k] Making Twitter better for disasters

I had both CNN and Twitter on yesterday all afternoon, looking for news about the Boston Marathon bombings. I have not done a rigorous analysis (nor will I, nor have I ever), but it felt to me that Twitter put forward more and more varied claims about the situation, and reacted faster to misstatements. CNN plodded along, but didn’t feel more reliable overall. This seems predictable given the unfiltered (or post-filtered) nature of Twitter.

But Twitter also ran into some scaling problems for me yesterday. I follow about 500 people on Twitter, which gives my stream a pace and variety that I find helpful on a normal day. But yesterday afternoon, the stream roared by, and approached filter failure. A couple of changes would help:

First, let us sort by most retweeted. When I’m in my “home stream,” let me choose a frequency of tweets so that the scrolling doesn’t become unwatchable; use the frequency to determine the threshold for the number of retweets required. (Alternatively: simply highlight highly re-tweeted tweets.)

Second, let us mute based on hashtag or by user. Some Twitter cascades I just don’t care about. For example, I don’t want to hear play-by-plays of the World Series, and I know that many of the people who follow me get seriously annoyed when I suddenly am tweeting twice a minute during a presidential debate. So let us temporarily suppress tweet streams we don’t care about.

It is a lesson of the Web that as services scale up, they need to provide more and more ways of filtering. Twitter had “follow” as an initial filter, and users then came up with hashtags as a second filter. It’s time for a new round as Twitter becomes an essential part of our news ecosystem.

1 Comment »

« Previous Page | Next Page »