At the Tel Aviv headquarters of the Center for Educational Technology, an NGO I’m very fond of because of its simultaneous dedication to improving education and its embrace of innovative technology, I got to try an Oculus Rift.

They put me on a virtual roller coaster. My real knees went weak.

Holy smokes.

 

---

Earlier, I gave a talk at the Israeli Wikimedia conference. I was reminded — not that I actually need reminding — how much I like being around Wikipedians. And what an improbable work of art is Wikipedia.

1 Comment »

I’m at the Israeli Wikimedia conference. The chair of the Wikimedia Foundation, Jan-Bart De Vreede, is being interviewed by Shizaf Rafaeli.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

Jan introduces himself. Besides being the chair, in the Netherlands he works on open educational resources at Kinnesnent. He says that the Wikimedia Foundation is quite small compared to other organizations like it. Five members are elected by the community (anyone with enough edits can vote), there are four appointed members, and Jimmy Wales.

Q: The Foundation is based on volunteers, and it has a budget. What are the components of the future for Wikipedia?

A: We have to make sure we get the technology to the place where we’re prepared for the future. And how we can enable the volunteers to do whatever they want to achieve our mission of being the sum of all knowledge, which is a high bar? Enabling volunteers is the highest impact thing that we can do.

Q: Students just did a presentation here based on the idea that Wikipedia already has too much information.

A: It’s not up to us to decide how the info is consumed. We should make sure that the data is available to be presented any way people want to. We are moving toward WikiData: structured data and the relationship among that data. How can we make it easier for people to add data to WikiData without necessarily requiring people to edit pages? How can we enable people to tag data? Can we use that to learn what people find relevant?

Q: What’s most important?

A: WikiData. Then Wikipedia Zero, making WP access available in developing parts of the globe. We’re asking telecoms to provide free access to Wikipedia on mobile phones.

Q: You’re talking with the Israeli Minister of Education tomorrow. About what?

A: We have a project of Wikipedia for children, written by children. Children can have an educational experience — e.g., interview a Holocaust survivor — and share it so all benefit from it.

Q: Any interesting projects?

A: Wiki Monuments [link ?]. Wiki Air. So many ideas. So much more to do. The visual editor will help people make edits. But we also have to make sure that new editors are welcomed and are treated kindly. Someone once told Jan that she “just helps new editors,” and he replied that that scale smuch better than creating your own edits.

A: I’m surprised you didn’t mention reliability…

Q: Books feel trustworthy. The Net automatically brings a measure of distrust, and rightly so. Wikipedia over the years has come to feel trustworthy, but that requires lots of people looking at it and fixing it when its wrong.

Q: 15,000 Europeans have applied to have their history erased on Google. The Israeli Supreme Court has made a judgment along the same lines. What’s Wikipedia’s stance on this?

A: As we understand it, the right to be forgotten applies to search engines, not to source articles about you. Encyclopedia articles are about what’s public.

Q: How much does the neutral point of view count?

A: It’s the most important thing, along with being written by volunteers. Some Silicon Valley types have refused to contributed money because, they say, we have a business model that we choose not to use: advertising. We decided it’d be more important to get many small contributions than corrode NPOV by taking money.

A: How about paid editing so that we get more content?

Q: It’s a tricky thing. There are public and governmental institutions that pay employees to provide Open Access content to Wikipedia and Wiki Commons. On the other hand, there are organizations that take money to remove negative information about their clients. We have to make sure that there’s a way to protect the work of genuine volunteers from this. But even when we make a policy about, the local Wikipedia units can override it.

Q: What did you think of our recent survey?

A: The Arab population was much more interested in editing Wikipedia than the Israeli population. How do you enable that? It didn’t surprise me that women are more interested in editing. We have to work against our systemic bias.

Q: Other diversity dimensions we should pay more attention to?

A: Our concept of encyclopedia itself is very Western. Our idea of citations is very Western and academic. Many cultures have oral citations. Wikipedia doesn’t know how to work with that. How can we accommodate knowledge that’s been passed down through generations?

Q&A

Q: Wikipedia doesn’t allow original research. Shouldn’t there be an open access magazine for new scientific research?

A: There are a lot of OA efforts. If more are needed, they should start with volunteers.

Q: Academics and Wikipedia have a touchy relationship. Wikipedia has won that battle. Isn’t it time to gear up for the next battle, i.e., creating open access journals?

A: There are others doing this. You can always upload and publish articles, if you want [at Wiki Commons?].

19 Comments »

Jérôme Hergeux is giving a Berkman lunch talk on “Cooperation in a peer prodiuction economy: experimental evidence from Wikipedia.” He lists as co-authors: Yann Algan, Yochai Benkler, and Mayo Fuster-Morell.

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

Jérôme explains the broader research agenda behind the paper. People are collaborating on the Web, sometimes on projects that compete with or replace major products from proprietary businesses and institutions. Standard economic theory doesn’t have a good way of making sense of this with its usual assumptions of behavior guided by perfect rationality and self-interest. Instead, Jérôme will look at Wikipedia where people are not paid and their contributions have no signaling value on the labor market. (Jérôme quotes Kizor: “The problem with Wikipedia is that it only works in practice. In theory it can never work.”)

Instead we should think of contributing to Wikipedia as a Public Goods dilemma: contributing has personal cost and not enough countervailing personal benefit, but it has a social benefit higher than the individual cost. The literature has mainly focused on the “prosocial preferences” that lead people to include the actions/interets of others, which leads them to overcome the Public Goods dilemma.

There are three classes of models commonly used by economists to explain prosocial behavior:

First, the altruism motive. Second, reciprocity: you respond in kind to kind actions of others. Third, “social image”: contributing to the public good signals something that brings you other utility. (He cites Napoleon: “Give me enough meals and I will win you any war.”)

His research’s method: Elicit the social prefs of a representative sample of Wikipedia contributors via an online experiment, and use those preferences to predict subjects’ field contributions to the Wikipedia project.

To check the reciprocity motive, they ran a simple public goods game. Four people in a group. Each has $10. Each has to decide how much to invest in a public project. You get some money back, but the group gets more. You can condition your contribution on the contributions of the other group members. This enables the researchers to measure how much the reciprocity motive matters to you. [I know I’m not getting this right. Hard to keep up. Sorry.] They also used a standard online trust game: You get some money from a partner, and can respond in kind.

Q: Do these tests correlate with real world behavior?

A: That’s the point of this paper. This is the first comprehensive test of all three motives.

For studying altruism, the dictator game is the standard. The dictator can give as much as s/he wants to the other person. The dictator has no reason to transfer the money. This thus measures altruism. But people might contribute to Wikipedia out of altruism just to their own Wikipedia in-group, not general altruism (“directed altruism”). So they ran another game to measure in-group altruism.

Social image is hard to measure experimentally, so they relied on observational data. “Consider as ‘social signalers’ subjects who have a Wikipedia user page whose size is bigger than the median in the sample.” You can be a quite engaged contributor to Wikipedia and not have a personal user page. But a bigger page means more concern with social image. Second, they looked at Barnstars data. Barnstars are a “social rewarding practice” that’s mainly restricted to heavy contributors: contribute well to a Wikipedia article and you might be given a barnstar. These shows up on Talk pages. About half of the people move it to their user page where it is more visible. If you move one of those awards manually to your user page, Jérôme will count you as a social signaller, i.e., someone who cares about his/her image.

He talks about some of the practical issues they faced in doing this experiment online. They illustrated the working of each game by using some simple Flash animations. And they provided calculators so you could see the effect of your decisions before you make them.

The subject pool came from registered Wikipedia users, and looked at the number of edits the user has made. (The number of contributions at Wikipedia follows a strong power law distribution.) 200,000 people register at Wikipedia account each month (2011) but only 2% make ten contributions in the their first month, and only 10% make one contribution or more within the next year. So, they recruited the cohort of new Wikipedia contributors (190,000 subjects), the group of engaged Wikipedia contributors (at least 300 edits) (18,989), and Wikipedia administrators (1,388 subjects). To recruit people, they teamed up with the Wikimedia Foundation to put a banner up on a Wikipedia page if the user met the criteria as a subject. The banner asked the reader to help with research. If readers click through, they go to the experiment page where they are paid in real money if they complete the 25 minute experiment within eight hours.

The demographics of the experiment’s subjects (1,099) matched quite closely the overall demographics of those subject pools. (The pool had 9% women, and the experiment had 8%).

Jérôme shows the regression tables and explains them. Holding the demographics steady, what is the relation between the three motives and the number of contributions? For the altruistic motive, there is no predictive power. Reciprocity in both games (public and trust) is a highly significant predictive. This tells us that reciprocal preference can lead you from being a non-contributor to being an engaged contributor; once you’re an engaged contributor, it doesn’t predict how far you’re going to go. Social image is correlated with the number of contributions; 81% of people who have received barnstars are super-contributors. Being a social signaler is associated with a 130% rise in the number of contributions you make. By both user-page length and barnstar, social image motivates for more contributions even among super-contributors.

Reciprocity incentivizes contributions only for those who are not concerned about their social image. So, reciprocity and social image are both at play among the contributors, but among separate groups. I.e., if you’re motivated by reciprocity, you are likely not motivated by social image, and vice versa.

Now Jérôme focuses on Wikipedia administrators. Altruism has no predictive value. But Wikipedia participation is negatively associated with reciprocity; perhaps this is because admins have to have thick skins to deal with disruptive users. For social image, the user page has significant revelance for admins, but not barnstars. Social image is less strong among admins than among other contributors.

Jérôme now explores his “thick skin hypothesis” to explain the admin results. In the trust game, look at how much the trustor decides how much to give to the stranger/partner. Jérôme ’s hypothesis: Among admins, those who decide to perform more of their policing role will be less trusting of strangers. There’s a negative correlation among admins between the results from the trust game and their contributions. The more time they say they do admin edits, the less trusting they are of strangers in the tests. That sort of make sense, says Jérôme. These admins are doing a valuable job for which they have self-selected, but it requires dealing with irritating people.

QA

Q: Maybe an admin is above others and is thus not being reciprocated by the group.

A: Perfectly reasonable explanation, and it is not ruled out by the data.

Q: Did you come into this with an idea of what might motivate the Wikipedians?

A: These are the three theories that are prevalent. We wanted to see how well they map onto actual field behavior.

Q: Maybe the causation goes the other way: working in Wikipedia is making people more concerned about social image or reciprocity?

A: The correlations could go in either direction. But we want to know if those explanations actually match what people do in the field.

Q: Heather Ford looks at why articles are deleted for non-Western topics. She found the notability criteria change for people not close to the topics. Maybe the motives change depending on how close you are to the event.

A: Sounds fascinating.

Q: Admins have an inherent bias in that they focus on the small percentage of contributors who are annoying jerks. If you spend your time working with jerks, it affects your sense of trust.

A: Good point. I don’t have the data to answer it.

Q: [me] If I’m a journalist I’m likely to take away the wrong conclusions from this talk, so I want to make sure I’m understanding. For example, I might conclude that Wikipedia admins are not motivated by altruism, whereas the right conclusion is (isn’t it?) that the standard altruism test doesn’t really measure altruism. Why not ask for self-reports to see?

A: Economists are skeptical about self-reports. If the reciprocity game predicts a correlation, that’s significant.

Yochai Benkler: Altruism has a special meaning among economists. It refers to any motivation other than “What’s in it for me?” [Because I asked the question, I didn’t do a good job recording the answers. Sorry.]

Q: Aren’t admins control freaks?

A: I wouldn’t say that. But control is not a pro-social motive, and I wanted to start with the theories that are current.

Q: You use the number of words someone writes on a user page as a sign of caring about social image, but this is in an context where people are there to write. And you’re correlating that to how much they write as editors and contributors. Maybe people at Wikipedia like to write. And maybe they write in those two different places for different reasons. Also, what do you do with these findings? Economists like to figure out which levers we pull if we’re not getting enough contributors.

Q: This sort of data seems to work well for large platforms with lots of users. What’s the scope of the methods you’re using? Only the top 100 web sites in the world?

A: I’d like to run this on all the peer production platforms in the world. Wikipedia is unusual if only because it’s been so successful. We’re already working on another project with 1,000 contributors at SourceForge especially to look at the effects of money, since about half of Open Source contributions are for money.

---

Fascinating talk. But it makes me want to be very dumb about it, because, well, I have no choice. So, here goes.

We can take this research as telling us something about Wikipedians’ motivations, about whether economists have picked the right three prosocial motivations, or about whether the standard tests of those motivations actually correlate to real-world motivations. I thought the point had to do with the last two alternatives and not so much the first. But I may have gotten it wrong.

So, suppose instead of talking about altruism, reciprocity, and social image we instead talk about the correlation between the six tests the researchers used and Wikipedia contributions. We would then have learned that Test #1 is a good predictor of the contribution levels of beginner Wikipedians, Test #2 predicts contributions by admins, Test #3 has a negative correlation with contributions by engaged Wikipedians, etc. But that would be of no interest, since we have (ex hypothesis) not made any assumptions about what the tests are testing for. Rather, the correlation would be a provocation to more research: why the heck does playing one of these odd little games correlate to Wikipedian productivity? It’d be like finding out that Wikipedian productivity is correlated to being a middle child or to wearing rings on both hands. How fascinating!… because these correlations have no implied explanatory power.

Now let’s plug back in the English terms that indicate some form of motivation. So now we can say that Test #3 shows that scoring high in altruism (in the game) does not correlate with being a Wikipedia admin. From this we can either conclude that Wikipedia admins are not motivated by altruism, or that the game fails to predict the existing altruism among Wikipedia admins. Is there anything else we can conclude without doing some independent study of what motivates Wikipedia admins? Because it flies in the face of both common sense and my own experience of Wikipedia admins; I’m pretty convinced one reason they work so hard is so everyone can have a free, reliable, neutral encyclopedia. So my strong inclination – admittedly based on anecdote and “common sense” (= “I believe what I believe!”) – is to conclude that any behavioral test that misses altruism as a component of the motivation of someone who spends thousands of hours working for free on an open encyclopedia…well, there’s something hinky about that behavioral test.

Even if the altruism tests correlate well with people engaged in activities we unproblematically associate with altruism – volunteering in a soup kitchen, giving away much of one’s income – I’d still not conclude from the lack of correlation with Wikipedia admins that those admins are not motivated by altruism, among other motivations. It just doesn’t correlate with the sort of altruism the game tests for. Just ask those admins if they’d put in the same amount of time creating a commercial encyclopedia.

So, I come out of Jérôme’s truly fascinating talk feeling like I’ve learned more about the reliability of the tests than about the motivations of Wikipedians. Based on Jérôme’s and Yochai’s responses, I think that’s what I’m supposed to have learned, but the paper also seems to be putting forward interesting conclusions (e.g., admins are not trusting types) that rely upon the tests not just correlating with the quantity of edits, but also being reliable measures of altruism, self-image, and reciprocity as motives. I assume (and thus may be wrong) that’s why Jérôme offered an hypothesis to explain the lack-of-trust result, rather than discounting the finding that admins lack trust (to oversimplify it).

(Two concluding comments: 1. Yochai’s The Leviathan and the Penguin uses behavioral tests like these, as well as case studies and observation, to make the case that we are a cooperative species. Excellent, enjoyable book. (Here’s a podcast interview I did with him about it.) 2. I’m truly sorry to be this ignorant.)

1 Comment »

Amanda Filipacchi has a great post at the New York Times about the problem with classifying American female novelists as American female novelists. That’s been going on at Wikipedia, with the result that the category American novelist was becoming filled predominantly with male novelists.

Part of this is undoubtedly due to the dumb sexism that thinks that “normal” novelists are men, and thus women novelists need to be called out. And even if the category male novelist starts being used, it still assumes that gender is a primary way of dividing up novelists, once you’ve segregated them by nation. Amanda makes both points.

From my point of view, the problem is inherent in hierarchical taxonomies. They require making decisions not only about the useful ways of slicing up the world, but also about which slices come first. These cuts reflect cultural and political values and have cultural and political consequences. They also get in the way of people who are searching with a different way of organizing the topic in mind. In a case like this, it’d be far better to attach tags to Wikipedia articles so that people can search using whatever parameters they need. That way we get better searchability, and Wikipedia hasn’t put itself in the impossible position of coming up with a taxonomy that is neutral to all points of view.

Wikipedia’s categories have been broken for a long time. We know this in the Library Innovation Lab because a couple of years ago we tried to find every article in Wikipedia that is about a book. In theory, you can just click on the “Book” category. In practice, the membership is not comprehensive. The categories are inconsistent and incomplete. It’s just a mess.

It may be that a massive crowd cannot develop a coherent taxonomy because of the differences in how people think about things. Maybe the crowd isn’t massive enough. Or maybe the process just needs far more guidance and regulation. But even if the crowd can bring order to the taxonomy, I don’t believe it can bring neutrality, because taxonomies are inherently political.

There are problems with letting people tag Wikipedia articles. Spam, for example. And without constraints, people can lard up an object with tags that are meaningful only to them, offensive, or wrong. But there are also social mechanisms for dealing with that. And we’ve been trained by the Web to lower our expectations about the precision and recall afforded by tags, whereas our expectations are high for taxonomies.

Go tags.

9 Comments »

Ars Technica has a post about Wikidata, a proposed new project from the folks that brought you Wikipedia. From the project’s introductory page:

Many Wikipedia articles contain facts and connections to other articles that are not easily understood by a computer, like the population of a country or the place of birth of an actor. In Wikidata you will be able to enter that information in a way that makes it processable by the computer. This means that the machine can provide it in different languages, use it to create overviews of such data, like lists or charts, or answer questions that can hardly be answered automatically today.

Because I had some questions not addressed in the Wikidata pages that I saw, I went onto the Wikidata IRC chat (http://webchat.freenode.net/?channels=#wikimedia-wikidata) where Denny_WMDE answered some questions for me.

[11:29] hi. I’m very interested in wikidata and am trying to write a brief blog post, and have a n00b question.

[11:29] go ahead!

[11:30] When there’s disagreement about a fact, will there be a discussion page where the differences can be worked through in public?

[11:30] two-fold answer

[11:30] 1. there will be a discussion page, yes

[11:31] 2. every fact can always have references accompanying it. so it is not about “does berlin really have 3.5 mio people” but about “does source X say that berlin has 3.5 mio people”

[11:31] wikidata is not about truth

[11:31] but about referenceable facts

When I asked which fact would make it into an article’s info box when the facts are contested, Denny_WMDE replied that they’re working on this, and will post a proposal for discussion.

So, on the one hand, Wikidata is further commoditizing facts: making them easier and thus less expensive to find and “consume.” Historically, this is a good thing. Literacy did this. Tables of logarithms did it. Almanacs did it. Wikipedia has commoditized a level of knowledge one up from facts. Now Wikidata is doing it for facts in a way that not only will make them easy to look up, but will enable them to serve as data in computational quests, such as finding every city with a population of at least 100,000 that has an average temperature below 60F.

On the other hand, because Wikidata is doing this commoditizing in a networked space, its facts are themselves links — “referenceable facts” are both facts that can be referenced, and simultaneously facts that come with links to their own references. This is what Too Big to Know calls “networked facts.” Those references serve at least three purposes: 1. They let us judge the reliability of the fact. 2. They give us a pointer out into the endless web of facts and references. 3. They remind us that facts are not where the human responsibility for truth ends.

4 Comments »

In the straight-up match between paper and Web, the Encyclopedia Britannica lost. This was as close to a sure thing as we get outside of the realm of macro physics and Meryl Streep movies.

1. The EB couldn’t cover enough: 65,000 topics compared to the almost 4M in the English version of Wikipedia.

2. Topics had to be consistently shrunk or discarded to make room for new information. E.g., the 1911 entry on Oliver Goldsmith was written by no less than Thomas Macaulay, but with each edition, it got shorter and shorter. EB was thus in the business of throwing out knowledge as much as it was in the business of adding knowledge.

3. Topics were confined to rectangles of text. This is of course often a helpful way of dividing up the world, but it is also essentially false. The “see also’s” and the attempts at synthetic indexes and outlines (Propædi) helped, but they were still highly limited, and cumbersome to use.

4. All the links were broken.

5. It was expensive to purchase.

6. If you or your family did not purchase it, using it required a trip to another part of town where it was available only during library hours.

7. It was very slow to update — 15 editions since 1768 — even with its “continuous revision” policy.

8. Purchasers were stuck with an artifact that continuously became wronger.

9. Purchasers were stuck with an artifact that continuously became less current.

10. It chose topics based on agendas and discussions that were not made public.

11. You could not see the process by which articles were written and revised, much less the reasoning behind those edits.

12. It was opaque about changes and errors.

13. There were no ways for readers to contribute or participate. For example, underlining in it or even correcting errors via marginalia would get you thrown out of the library. It thus crippled the growth of knowledge through social and networked means.

14. It was copyrighted, making it difficult for its content to be used maximally.

Every one of the above is directly or indirectly a consequence of the fact that the EB was a paper product.

Paper doesn’t scale.

Paper-based knowledge can’t scale.

The Net scales.

The Net scales knowledge.

 

---

I should probably say something nice about the Britannica:

1. Extremely smart, very dedicated people worked on it.

2. It provided a socially safe base for certain sorts of knowledge claims.

3. Owning it signaled that one cared about knowledge, and it’s good for our culture for us to be signaling that sort of thing.

 

---

The inestimably smart and wise Matthew Battles has an excellent post on the topic (which I hesitate to recommend only because he refers to “Too Big to Know” overly generously).

55 Comments »

Jimmy Wales has proposed that Wikipedia might black out its English-language pages for a short period to register opposition to the SOPA law that would allow the US government to shut down access to sites that provide access to material that infringes copyright. These shutdowns would occur without the need for any judicial procedure, without notice, and without appeal.

I think Jimmy’s idea is great and that all sites that could be affected by SOPA — which is to say any site — ought to join in. Just name the date and time, and many of us would turn out our sites’ lights.

[Minutes later: Through a failure in my command of in-page searching, I missed Cory Doctorow’s proposing exactly this on BoingBoing. Go Jimmy! Go Cory!]

(Here’s Rebecca MacKinnon’s op-ed on SOPA and its Senate version, which together would constitute a Great Firewall of America, as she says. [A couple of hours later: Rebecca and Ivan Sigal just posted a terrific op-ed on the topic at CNN.com)

Be the first to comment »

My podcast interview of Yochai Benkler about his excellent new book, The Penguin and the Leviathan has been posted. Yochai makes brilliantly (of course) a case that shouldn’t need making, but that in fact does very much need to be made: that we are collaborative, social, cooperative creatures. Your unselfish genes will thoroughly enjoy this book.

And, Joseph Reagle has promulgated the following email about his excellent, insightful book that explores the subtleties of the social structures that enable Wikipedia to accomplish its goal of being a great encyclopedia:

I’m pleased to announce that the Web/CC edition of *Good Faith Collaboration* is now available. In addition to all of the book’s complete content, hypertextual goodness, and fixed errata, there is a new preface discussing some of the particulars of this edition.

2 Comments »

In what is certainly the coolest proof that philosophy is the Queen of the Sciences, and also the Duke and all of the in-laws, xefer shows the relation of any Wikipedia topic to its “Philosophy” article. (Hat tip to Hal Roberts.)

1 Comment »

I just shared a cab with James Bridle, a UK publisher and digital activist (my designation, not his) who is the brilliance behind the printing out of the changes to the Wikipedia article on the Iraq War. It turns out that those changes — just the changed portions — fill up twelve volumes.

What does the project show? “The argument,” James says. Of course it also shows the power of the cognitive surplus: we just casually created twelve volumes of changes in our spare time. If only all users of Wikipedia all understood how it’s put together! (Rather than banning students from using Wikipedia, it’d be far better if teachers required students to click on the “Discussion” tab.)

1 Comment »