Amanda Alvarez has a provocative post at GigaOm:
There’s an epidemic going on in science: experiments that no one can reproduce, studies that have to be retracted, and the emergence of a lurking data reliability iceberg. The hunger for ever more novel and high-impact results that could lead to that coveted paper in a top-tier journal like Nature or Science is not dissimilar to the clickbait headlines and obsession with pageviews we see in modern journalism.
The article’s title points especially to “dodgy data,” and the item in this list that’s by far the most interesting to me is the “data reliability iceberg,” and its tie to the rise of Big Data. Amanda writes:
…unlike in science…, in big data accuracy is not as much of an issue. As my colleague Derrick Harris points out, for big data scientists the abilty to churn through huge amounts of data very quickly is actually more important than complete accuracy. One reason for this is that they’re not dealing with, say, life-saving drug treatments, but with things like targeted advertising, where you don’t have to be 100 percent accurate. Big data scientists would rather be pointed in the right general direction faster — and course-correct as they go – than have to wait to be pointed in the exact right direction. This kind of error-tolerance has insidiously crept into science, too.
But, the rest of the article contains no evidence that the last sentence’s claim is true because of the rise of Big Data. In fact, even if we accept that science is facing a crisis of reliability, the article doesn’t pin this on an “iceberg” of bad data. Rather, it seems to be a melange of bad data, faulty software, unreliable equipment, poor methodology, undue haste, and o’erweening ambition.
The last part of the article draws some of the heat out of the initial paragraphs. For example: “Some see the phenomenon not as an epidemic but as a rash, a sign that the research ecosystem is getting healthier and more transparent.” It makes the headline and the first part seem a bit overstated — not unusual for a blog post (not that I would ever do such a thing!) but at best ironic given this post’s topic.
I remain interested in Amanda’s hypothesis. Is science getting sloppier with data?
, too big to know
Tagged with: 2b2k
• big data
Date: May 26th, 2013 dw
Bora Zivkovic, the blog editor at Scientific American, has a great post about bad comment threads. This is a topic that has come up every day this week, which may just be a coincidence, or perhaps is a sign that the Zeitgeist is recognizing that when it talks to itself, it sounds like an idiot.
Bora cites a not-yet-published paper that presents evidence that a nasty, polarized comment thread can cause readers who arrive with no opinion about the paper’s topic to come to highly polarized opinions about it. This is in line with off-line research Cass Sunstein cites that suggests echo chambers increase polarization, except this new research indicates that it increases polarization even on first acquaintance. (Bora considers the echo chamber idea to be busted, citing a prior post that is closely aligned with the sort of arguments I’ve been making, although I am more worried about the effects of homophily — our tendency to hang out with people who agree with us — than he is.)
Much of Bora’s post is a thoughtful yet strongly voiced argument that it is the responsibility of the blog owner to facilitate good discussions by moderating comments. He writes:
So, if I write about a wonderful dinner I had last night, and somewhere in there mention that one of the ingredients was a GMO product, but hey, it was tasty, then a comment blasting GMOs is trolling.
Really? Then why did Bora go out of his way to mention that it was a GMO product? He seems to me to be trolling for a response. Now, I think Bora just picked a bad example in this case, but it does show that the concept of “off-topic” contains a boatload of norms and assumptions. And Bora should be fine with this, since his piece begins by encouraging bloggers to claim their conversation space as their own, rather than treating it as a public space governed by the First Amendment. It’s up to the blogger to do what’s necessary to enable the type of conversations that the blogger wants. All of which I agree with.
Nevertheless, Bora’s particular concept of being on-topic highlights a perpetual problem of conversation and knowledge. He makes a very strong case — nicely argued — for why he nukes climate-change denials from his comment thread. Read his post, but the boiled down version is: (a) These comments are without worth because they do not cite real evidence and most of them are astroturf anyway. (b) They create a polarized environment that has the bad effect of raising unjustified doubts in the minds of readers of the post (as per the research he mentions at the beginning of his post). (c) They prevent conversation from advancing thought because they stall the conversation at first principles. Sounds right to me. And I agree with his subsequent denial of the echo chamber effect as well:
The commenting threads are not a place to showcase the whole spectrum of opinions, no matter how outrageous some of them are, but to educate your readers, and to, in turn, get educated by your readers who always know something you don’t.
But this is why the echo chamber idea is so slippery. Conversation consists of the iteration of small differences upon a vast ground of agreement. A discussion of a scientific topic among readers of Scientific American has value insofar as they can assume that, say, evolution is an established theory, that assertions need to be backed by facts of a certain evidentiary sort (e.g., “God told me” doesn’t count), that some assertions are outside of the scope of discussion (“Evolution is good/evil”), etc. These are criteria of a successful conversation, but they are also the marks of an echo chamber. The good Scientific American conversation that Bora curates looks like an echo chamber to the climate change deniers and the creationists. If one looks only at the structure of the conversation, disregarding all the content and norms, the two conversations are indistinguishable.
But now I have to be really clear about what I’m not saying. I am not saying that there’s no difference between creationists and evolutionary biologists, or that they are equally true. I am not saying that both conversations follow the same rules of evidence. I am certainly not saying that their rules of evidence are equally likely to lead to scientific truths. I am not even saying that Bora needs to throw open the doors of his comments. I’m saying something much more modest than that: To each side, the other’s conversation looks like a bunch of people who are reinforcing one another in their wrong beliefs by repeating those beliefs as if they were obviously right. Even the conversation I deeply believe is furthering our understanding — the evolutionary biologists, if you haven’t guessed where I stand on this issue — has the structure of an echo chamber.
This seems to me to have two implications.
First, it should keep us alert to the issue that Bora’s post tries to resolve. He encourages us to exclude views challenging settled science because including ignorant trolls leads casual visitors to think that the issues discussed are still in play. But climate change denial and creationist sites also want to promote good conversations (by their lights), and thus Bora is apparently recommending that those sites also should exclude those who are challenging the settled beliefs that form the enabling ground of conversation — even though in this case it would mean removing comments from all those science-y folks who keep “trolling” them. It seems to me that this leads to a polarized culture in which the echo chamber problem gets worse. Now, I continue to believe that Bora is basically right in his recommendation. I just am not as happy about it as he seems to be. Perhaps Bora is in practice agreeing with Too Big to Know’s recommendation that we recognize that knowledge is fragmented and is not going to bring us all together.
Second, the fact that we cannot structurally distinguish a good conversation from a bad echo chamber I think indicates that we don’t have a good theory of conversation. The echo chamber fear grows in the space that a theory of conversation should inhabit.
I don’t have a theory of conversation in my hip pocket to give you. But I presume that such a theory would include the notion, evident in Bora’s post, that conversations have aims, and that when a conversation is open to the entire world (a radically new phenomenon…thank you WWW!) those aims should be explicitly stated. Likewise for the norms of the conversation. I’m also pretty sure that conversations are never only about they say they’re about because they are always embedded in complex social environments. And because conversations iterate on differences on a vast ground of similarity, conversations rarely are about changing people’s minds about those grounds. Also, I personally would be suspicious of any theory of conversation that began by viewing conversations as composed fundamentally of messages that are encoded by the sender and decoded by the recipient; that is, I’m not at all convinced that we can get a theory of conversation out of an information-based theory of communication.
But I dunno. I’m confused by this entire topic. Nothing that a good conversation wouldn’t cure.
The letters of
Lord Alfred Russel Wallace, co-discoverer of the theory of evolution by natural selection, are now online. As the Alfred Russel Wallace Correspondence Project explains, the collection consists of 4,000 letters gathered from about 100 different institutions, with about half in the British Natural History Museum and British Library.
The Correspondence Project has, admirably, been releasing the scans without waiting for transcription; more faster is better! Predictably annoyingly, the letters, written by a man who died ten years before the Perpetual Copyright date of 1923, seem to be (but are they?) carefully obstructed by copyright: The Natural History Museum, which houses the collection, asserts copyright over “data held in the Wallace Letters Online database (including letter summaries)” [pdf — oddly unreadable in Mac Preview]. Beyond the summaries, exactly what data is this referring to? Not sure. Don’t know.
But that isn’t the full story anyway, for the NHM sends us to the Wallace Fund for more information about the copyright. That page tells us that the unpublished letters are copyrighted until 2039, with this very helpful footnote:
Unless the work was published with the permission of his Literary Estate before 1 August 1989, in which case the work will be in copyright for 70 years after Wallace’s death, unless he died more than 20 years before the work’s publication, in which case copyright would expire 50 years after publication.
Eventually it gets to some good news:
Authors wishing to publish such works would ordinarily need to obtain permission from the copyright holder before doing so. However, on July 31st 2011, in an attempt to facilitate the scholarly study of ARW’s writings, the co-executors of ARW’s Literary Estate agreed to allow third parties to publish ARW’s copyright works non-commercially without first having to ask the Literary Estate for permission, under the terms and conditions of Creative Commons license “Attribution-NonCommercial-ShareAlike 3.0 Unported”
So, are the letters published on the NHM site actually available under a Creative Commons non-commercial license? The Wallace Fund that aggregated them seems to think so. The NHM that published them maybe thinks not.
Because copyright is just so magical.
TWO HOURS LATER: Please see the first comment, from George Beccaloni, Director of the Wallace Correspondence Project. Thanks, George.
He explains that the transcribed text is available under a Creative Commons non-commercial license, but the digitized images are not. Plus some further complications, such as the content of the database being under copyright, although it is not clear from the site what data that is.
Since the aim of CC is to make it easier for people to re-use material, may I suggest (in the friendliest of fashions) that this be prominently clarified on the sites themselves?
Well, I learned a bunch of stuff, but I’ll only mention two.
First, NASA is as totally awesome as you think it is. I went to the Langley centerfor a one day visit, and got a morning tour, and it is a nerd-heaven work space, with no Star Wars white plastic, but lots and lots of dented workbenches covered with sprays of components. And it adds up to our species looking down on our planet. Ultra ultra cool.
Second, I got a tour of the National Transonic Facility by Bill Bisset, who manages the place. They test models in the world’s most sophisticated wind tunnel — they fill it with liquid nitrogen (which they make themselves) that’s blown in by the world’s most powerful horizontally-mounted electrical motor (that consumes an eighth of the output of a local nuclear generator), and they measure up to 5,000 different parameters. So, naturally, I ran an urban myth past Bill, because that’s an excellent use of his time.
I had been told by someone sometime that those little upturned wing tips you sometimes see on planes were discovered more than invented: Someone tried them out, and they turned out to increase the efficiency of the plane, but no one knew why.
Nope, nope, and nope. They’re called winglets. Here’s the story, from a NASA page:
The concept of winglets originated with a British aerodynamicist in the late 1800s, but the idea remained on the drawing board until rekindled in the early 1970s by Dr. Richard Whitcomb when the price of aviation fuel started spiraling upward.
Bill explained that winglets work by altering the vortex that forms when air rushes over a wing. “Winglets…produce a forward thrust inside the circulation field of the vortices and reduce their strength,” as the NASA page says. They increase efficiency by 6-9%. Bill said they also effectively increase the wingspan of the plane, but without extending the wings horizontally, which matters to airlines because they pay airports based upon the horizontal length of the wings.
So,yes, everything I’d heard was wrong. And, yes, it was in Wikipedia all along.
(And yes, I learned a whole lot more. It was for me a wonderful day.)
Tagged with: nasa
Date: January 9th, 2013 dw
An article in published in Science on Thursday, securely locked behind a paywall, paints a mixed picture of science in the age of social media. In “Science, New Media, and the Public,” Dominique Brossard and Dietram A. Scheufele urge action so that science will be judged on its merits as it moves through the Web. That’s a worthy goal, and it’s an excellent article. Still, I read it with a sense that something was askew. I think ultimately it’s something like an old vs. new media disconnect.
The authors begin by noting research that suggests that “online science sources may be helping to narrow knowledge gaps” across educational levels. But all is not rosy. Scientists are going to have “to rethink the interface between the science community and the public.” They point to three reasons.
First, the rise of online media has reduced the amount of time and space given to science coverage by traditional media .
Second, the algorithmic prioritizing of stories takes editorial control out of the hands of humans who might make better decisions. The authors point to research that “shows that there are often clear discrepancies between what people search for online, which specific areas are suggested to them by search engines, and what people ultimately find.” The results provided by search engines “may all be linked in a self-reinforcing informational spiral…” This leads them to ask an important question:
Is the World Wide Web opening up a new world of easily accessible scientific information to lay audiences with just a few clicks? Or are we moving toward an online science communication environment in which knowledge gain and opinion formation are increasingly shaped by how search engines present results, direct traffic, and ultimately narrow our informational choices? Critical discussions about these developments have mostly been restricted to the political arena…
Third, we are debating science differently because the Web is social. As an example they point to the fact that “science stories usually…are embedded in a host of cues about their accuracy, importance, or popularity,” from tweets to Facebook “Likes.” “Such cues may add meaning beyond what the author of the original story intended to convey.” The authors cite a recent conference  where the tone of online comments turned out to affect how people took the content. For example, an uncivil tone “polarized the views….”
They conclude by saying that we’re just beginning to understand how these Web-based “audience-media interactions” work, but that the opportunity and risk are great, so more research is greatly needed:
Without applied research on how to best communicate science online, we risk creating a future where the dynamics of online communication systems have a stronger impact on public views about science than the specific research that we as scientists are trying to communicate.
I agree with so much of this article, including its call for action, yet it felt odd to me that scientists will be surprised to learn that the Web does not convey scientific information in a balanced and impartial way. You only are surprised by this if you think that the Web is a medium. A medium is that through which content passes. A good medium doesn’t corrupt the content; it conveys signal with a minimum of noise.
But unlike any medium since speech, the Web isn’t a passive channel for the transmission of messages. Messages only move through the Web because we, the people on the Web, find them interesting. For example, I’m moving (infinitesimally, granted) this article by Brossard and Scheufele through the Web because I think some of my friends and readers will find it interesting. If someone who reads this post then tweets about it or about the original article, it will have moved a bit further, but only because someone cared about it. In short, we are the medium, and we don’t move stuff that we think is uninteresting and unimportant. We may move something because it’s so wrong, because we have a clever comment to make about it, or even because we misunderstand it, but without our insertion of ourselves in the form of our interests, it is inert.
So, the “dynamics of online communication systems” are indeed going to have “a stronger impact on public views about science” than the scientific research itself does because those dynamics are what let the research have any impact beyond the scientific community. If scientific research is going to reach beyond those who have a professional interest in it, it necessarily will be tagged with “meaning beyond what the author of the original story intended to convey.” Those meanings are what we make of the message we’re conveying. And what we make of knowledge is the energy that propels it through the new system.
We therefore cannot hope to peel the peer-to-peer commentary from research as it circulates broadly on the Net, not that the Brossard and Scheufele article suggests that. Perhaps the best we can do is educate our children better, and encourage more scientists to dive into the social froth as the place where their research is having its broadest effect.
Notes, copied straight from the article:
 M. A. Cacciatore, D. A. Scheufele, E. A. Corley, Public Underst. Sci.; 10.1177/0963662512447606 (2012).
 C. Russell, in Science and the Media, D. Kennedy, G. Overholser, Eds. (American Academy of Arts and Sciences, Cambridge, MA, 2010), pp. 13–43
 P. Ladwig et al., Mater. Today 13, 52 (2010)
 P. Ladwig, A. Anderson, abstract, Annual Conference of the Association for Education in Journalism and Mass Communication, St. Louis, MO, August 2011; www.aejmc. com/home/2011/06/ctec-2011-abstracts
, social media
, too big to know
Tagged with: 2b2k
Date: January 5th, 2013 dw
The American Psychiatric Association has approved its new manual of diagnoses — Diagnostic and Statistical Manual of Mental Disorders — after five years of controversy [nytimes].
For example, it has removed Aspberger’s as a diagnosis, lumping it in with autism, but it has split out hoarding from the more general category of obsessive-compulsive disorder. Lumping and splitting are the two most basic activities of cataloguers and indexers. There are theoretical and practical reasons for sometimes lumping things together and sometimes splitting them, but they also characterize personalities. Some of us are lumpers, and some of us are splitters. And all of us are a bit of each at various times.
The DSM runs into the problems faced by all attempts to classify a field. Attempts to come up with a single classification for a complex domain try to impose an impossible order:
First, there is rarely (ever?) universal agreement about how to divvy up a domain. There are genuine disagreements about which principles of organization ought to be used, and how they apply. Then there are the Lumper vs. the Splitter personalities.
Second, there are political and economic motivations for dividing up the world in particular ways.
Third, taxonomies are tools. There is no one right way to divide up the world, just as there is no one way to cut a piece of plywood and no one right thing to say about the world. It depends what you’re trying to do. DSM has conflicting purposes. For one thing, it affects treatment. For example, the NY Times article notes that the change in the classification of bipolar disease “could ‘medicalize’ frequent temper tantrums,” and during the many years in which the DSM classified homosexuality as a syndrome, therapists were encouraged to treat it as a disease. But that’s not all the DSM is for. It also guides insurance payments, and it affects research.
Given this, do we need the DSM? Maybe for insurance purposes. But not as a statement of where nature’s joints are. In fact, it’s not clear to me that we even need it as a single source to define terms for common reference. After all, biologists don’t agree about how to classify species, but that science seems to be doing just fine. The Encyclopedia of Life takes a really useful approach: each species gets a page, but the site provides multiple taxonomies so that biologists don’t have to agree on how to lump and split all the forms of life on the planet.
If we do need a single diagnostic taxonomy, DSM is making progress in its methodology. It has more publicly entered the fray of argument, it has tried to respond to current thinking, and it is now going to be updated continuously, rather than every 5 years. All to the good.
But the rest of its problems are intrinsic to its very existence. We may need it for some purposes, but it is never going to be fully right…because tools are useful, not true.
And one more thing about my previous post: I understand that when Heidegger was writing Being and Time in the 1920s, it was important to try to relax our culture’s commitment to scientific objectivity in order to allow more types of truths to appear – more ways that the world shows itself to us.
Almost a hundred years later, with a brand new medium for knowledge, truth, and disclosure, it is time to re-assert science’s privileged (yet still human and imperfect) position as we try to come to agreement across cultures about what we need to do in order to live together on this earth.
In my opinion.
, too big to know
Tagged with: 2b2k
Date: October 28th, 2012 dw
Last night I gave a talk at the Festival of Science in Genoa (or, as they say in Italy, Genova). I was brought over by Codice Edizioni, the publisher of the just-released Italian version of Too Big to Know (or, as they say in Italy “La Stanza Intelligente” (or as they say in America, “The Smart Room”)). The event was held in the Palazzo Ducale, which ain’t no Elks Club, if you know what I mean. And if you don’t know what I mean, what I mean is that it’s a beautiful, arched, painted-ceiling room that holds 800 people and one intimidated American.
After my brief talk, Serena Danna of Corriere della Serra interviewed me. She’s really good. For example, her first question was: If the facts no longer have the ability to settle arguments the way we hoped they would, then what happens to truth?
Yeah, way to pitch the ol’ softballs, Serena!
I wasn’t satisfied with my answer, which had three parts. (1) There are facts. The world is one way and not all the other ways that it isn’t. You are not free to make up your own facts. [Yes, I’m talking to you, Mitt!] (2) The basing of knowledge primarily on facts is a relatively new phenomenon. (3) I explicitly invoked Heidegger’s concept of truth, with a soupçon of pragmatism’s view of truth as a tool intended to serve a purpose.
Meanwhile, I’ve been watching The Heidegger Circle mailing list contort itself trying to understand Heidegger’s views about the world that existed before humans entered the scene. Was there Being? Were there beings? It seems to me that any answer has to begin by saying, “Of course the world existed before we did.” But not everyone on the list is comfortable with a statement that simple. Some seem to think that acknowledging that most basic fact somehow diminishes Heidegger’s analysis of the relation of Being and disclosure. Yo, Heideggerians! The world shows itself to us as independent of us. We were born into it, and it keeps going after we’ve died. If that’s a problem for your philosophy, then your philosophy is a problem. And for all of the problems with Heidegger’s philosophy, that just isn’t one. (To be fair, no one on the list suggests that the existence of the universe depends upon our awareness of it, although some are puzzled about how to maintain Heidegger’s conception of “world” (which does seem to depend on us) with that which survives our awareness of it. Heidegger, after all, offers phenomenological ontology, so there is a question about what Being looks like when there is no one to show itself to.)
So, I wasn’t very happy with what I said about truth last night. I said that I liked Heidegger’s notion that truth is the world showing itself to us, and it shows itself to us differently depending on our projects. I’ve always liked this idea for a few reasons. First, it’s phenomenologically true: the onion shows itself differently whether you’re intending to cook it, whether you’re trying to grow it as a cash crop, whether you’re trying to make yourself cry, whether you’re trying to find something to throw at a bad actor, etc. Second, because truth is the way the world shows itself, Heidegger’s sense contains the crucial acknowledgement that the world exists independently of us. Third, because this sense of truth look at our projects, it contains the crucial acknowledgement that truth is not independent of our involvement in the world (which Heidegger accurately characterizes not with the neutral term “involvement” but as our caring about what happens to us and to our fellow humans). Fourth, this gives us a way of thinking about truth without the correspondence theory’s schizophrenic metaphysics that tells us that we live inside our heads, and our mental images can either match or fail to match external reality.
But Heidegger’s view of truth doesn’t do the job that we want done when we’re trying to settle disagreements. Heidegger observes (correctly in my and everybody’s opinion) that different fields have different methodologies for revealing the truth of the world. He speaks coldly (it seems to me) of science, and warmly of poetry. I’m much hotter on science. Science provides a methodology for letting the world show itself (= truth) that is reproducible precisely so that we can settle disputes. For settling disputes about what the world is like regardless of our view of it, science has priority, just as the legal system has priority for settling disputes over the law.
This matters a lot not just because of the spectacular good that science does, but because the question of truth only arises because we sense that something is hidden from us. Science does not uncover all truths but it uniquely uncovers truths about which we can agree. It allows the world to speak in a way that compels agreement. In that sense, of all the disciplines and methodologies, science is the closest to giving the earth we all share its own authentic voice. That about which science cannot speak in a compelling fashion across all cultures and starting points is simply not subject to scientific analysis. Here the poets and philosophers can speak and should be heard. (And of course the compulsive force science manifests is far from beyond resistance and doubt.)
But, when we are talking about the fragmenting of belief that the Internet facilitates, and the fact that facts no longer settle arguments across those gaps, then it is especially important that we commit to science as the discipline that allows the earth to speak of itself in its most compelling terms.
Finally, I was happy that last night I did manage to say that science provides a model for trying to stay smart on the Internet because it is highly self-aware about what it knows: it does not simply hold on to true statements, but is aware of the methodology that led us to see those statements as true. This type of meta awareness — not just within the realm of science — is crucial for a medium as open as the Internet.
Someday I’ll figure out the threads that bind the mere sentences that make me fill with tears. Sometimes it’s sadness, but surprisingly often it’s joy.
Here’s today’s joy:
Look in the upper right for a crescent-shaped smudge. That’s Phobos, one of Mars’ two moons.
Emily Lakdawalla writes in her blog:
Think about this for a moment — we’re seeing a different moon from the surface of a different world. And this moon is weird not just for its lumpiness, but also because it orbits so close to Mars that it outpaces Mars’ rotation. That means it rises in the west and sets in the east, more than twice every Martian day. Completely alien. And awesome, in the literal sense of the world.
It turns me into a soppy ol’ Boehner.
Here’s a close-up of Phobos:
I would not have noticed this image were it not for the ever-watchful members of unmannedspaceflight.com (user “fredk” this time). I’m so grateful for that community. We’re running a fundraiser right now to support our hosting costs — if you, too, value the beautiful images and constant attentiveness of this community of volunteers and amateurs, please consider making a donation to support it.
, too big to know
Tagged with: curiosity
Date: September 30th, 2012 dw
From TheHeart.org, an article by Lisa Nainggolan:
Gothenburg, Sweden – Further support for the concept of the obesity paradox has come from a large study of patients with acute coronary syndrome (ACS) in the Swedish Coronary Angiography and Angioplasty Registry (SCAAR) . Those who were deemed overweight or obese by body-mass index (BMI) had a lower risk of death after PCI [percutaneous coronary intervention, aka angioplasty] than normal-weight or underweight participants up to three years after hospitalization, report Dr Oskar Angerås (University of Gothenburg, Sweden) and colleagues in their paper, published online September 5, 2012 in the European Heart Journal.
Can confirm. My grandmother in the 1930s was instructed to make sure she fed her husband lots and lots of butter to lubricate his heart after a heart attack. This proved to work extraordinarily well, at least until his next heart attack.
I refer once again to the classic 1999 The Onion headline: Eggs Good for You This Week.
, too big to know
Tagged with: 2b2k
Date: September 10th, 2012 dw
« Previous Page | Next Page »