Monthly Archives: March 2012

JPSM Distinguished Lecture

Tomorrow the Joint Program in Survey Methodology is having a special lecture at the University of Maryland.

Do survey respondents lie?

Situated cognition and socially desirable responding Prof. Norbert Schwarz University of Michigan

Survey researchers commonly assume that people know what they do, know what they believe, and can report on it with candor and accuracy, as Angus Campbell put it. From this perspective, many findings suggest that survey respondents are less than candid. The best known example is the observation that answers to racial attitude questions vary as a function of the interviewers race. Challenging this interpretation, a large body of social psychological research shows similar context effects under conditions that do not lend themselves to this interpretation, including conditions that use implicit attitude measures, which are not subject to deliberate “faking”.

From a situated cognition perspective, such findings reflect that attitude questions assess context sensitive evaluations that respondents form on the spot, drawing on information that is accessible at that point in time. The underlying processes operate in daily life as well as in survey interviews and reflect the situated nature of human judgment rather than a deliberate attempt to report a socially desirable answer.

I review relevant findings and discuss their implications for survey measurement.

Friday, March 30, 2012, 3:00 PM – 5:00 PM

2205 LeFrak Hall, University of Maryland, College Park MD USA

Metro stop: College Park on the Green line See http://www.jpsm.umd.edu/jpsm/?geninfo/directions.htm for directions and parking information.

 

Discussants: Paul Beatty, NCHS and David Cantor, Westat

 

A reception follows the lecture.

Advertisement

Don’t fear Big Data

I really enjoyed this RTI blog post about embracing big data:

https://blogs.rti.org/surveypost/2012/03/22/why-you-should-not-fear-but-embrace-the-age-of-big-data/

I suspect that oftentimes fear of big data is motivated by a concern that new, less tested, still evolving methods will replace the time tested methods that we have grown to have so much faith in. I sincerely believe that the foundation that we have is a strong one, and the knowledge we have developed through those processes should be embraced, especially the quality controls. But SUPPLEMENTING an analysis through a measured combination of data sources can lead to a more complete picture.

This week I spent some time analyzing Pew’s report on the Kony 2012 video. I believe that this report is an excellent example of what researchers are capable of when they look outside the artificial divisions of research group (this was a collaborative effort) and research methodology. Seven days after the release of the video, Pew was able to reconstruct a comprehensive narrative of the video’s dissemination, using traditional survey methods, sentiment analytic snapshots over time, and a careful breakdown of the media coverage of influential parties.

 

Dana Boyd also has an interesting analysis of the Kony phenomena on her Apophenia blog:

http://www.zephoria.org/thoughts/

Fostering Creativity at Work

This book looks fantastic. Whenever I need to do a lot of thinking at work, I’ll go for a walk or hit the gym. Or start reading about a similar topic. Or stare out the window. We don’t have Ping Pong tables, but we do have floor to ceiling windows overlooking a wooded patch. I can’t tell you how any cumulative hours I’ve spent watching the trees wave in the wind and working my way through a stumbling block.

 

http://www.npr.org/2012/03/21/148607182/fostering-creativity-and-imagination-in-the-workplace?ft=3&f=122101520&sc=nl&cc=sh-20120324

Zen as a Research Ethic

I have a Zen calendar on my desk for 2012. It has such gems as: “Although the world is full of suffering, it is also full of the overcoming of it” (Helen Keller)

The more I look at the calendar, the more it relates to everything I think about.

I read “To see is to forget the name of the thing one sees,” (Paul Valery) and I think of the Charles Goodwin paper I cited in a recent post about Professional Vision. He talks about ways of seeing as kind of coding structures, inculturation, or ways of foregrounding certain parts of what we see. Truly, being able to see deeper than that requires shedding that inculturation and observing more closely. As researchers, we often become so deeply incultured into our way of thinking, that we lose sight of our research goals. As survey researchers, we can easily fall into the pattern of first asking “who should we survey?” and “what should we ask?” before taking more time to consider whether a survey is even an appropriate methodology for the specific topic of focus. Of course, not this action based on praxis is not limited to survey researchers. Far from it! Every person, every field, every community of practice, every language has a way of thinking. And often instead of seeing or observing, we quickly begin to navigate our networks of inculturation.

These two are similarly meaningful in my interpretation:

“Zen is not to confuse spirituality with thinking about God while one is peeling potatoes. Zen is just to peel the potatoes.” (Alan Watts)

“If all beings are Buddha, why all this striving?” (Dogen)

These are a reminder to boil things down to what they simply are and not try to describe them as what you want them to be. In survey research, this comes up often in the process of reporting research results. If I know that I intended to measure something about Project Based Learning or STEM education, it is easily for me to begin to frame my findings by my intentions. But that is not true to my findings or my methodology, and it doesn’t make for good research. I can’t say that 10% of my respondents were using project based learning methods in the classroom if I asked about the number of group activities they conducted. I must simply say that 10% were using group activities (daily/monthly/occasionally- whatever the answer choices were)

In this way, my Zen calendar not only provides something to think about in a larger sense, but it keeps my research anchored.

Why Social Media couldn’t predict Super Tuesday

This piece is a nice reminder not only, as the authors conclude, that sentiment analysis has not fully matured, but also that sentiment analysis and social media analysis probably don’t accomplish what they think they are accomplishing:

 

http://www.retargeter.com/political-advertising/why-social-media-couldnt-predict-super-tuesday

Taking it to the source

As a survey researcher at the American Institute of Physics, I often say that I don’t study physics, I study people who study physics. In that respect, this is an interesting way to take it back:

http://www.businessweek.com/articles/2012-03-22/book-review-the-idea-factory-by-jon-gertner

This graphic shows how I owe a debt of gratitude to the physicists I study for making the devices through which I study them.

I guess some would say that sooner or later, it has to come back to physics (physics being the root of all things)!

Seriously, though. It’s an interesting and fun graphic. Check it out.

Telling your story in qualitative and quantitative terms

This is a really interesting piece by Robery Krulwich:

http://www.npr.org/blogs/krulwich/2012/03/21/149095154/mirror-mirror-on-the-wall-do-the-data-tell-it-all?sc=fb&cc=fp

In it, he asks a writer and a physicist to talk about their lives. They describe their lives in vastly different terms. The writers tells his story in narrative form, and the physicist describes his in terms of data. In fact, the physicist has amassed a sizable collection of what he calls ‘personal data.’ He provides an interesting an unusual perspective and begs the question ‘what does this data mean?’ or, as Dr Castillo Ayometzi would ask, “what is being constituted here?”

Enjoy!

Language as a way of seeing

A conversation in my Intercultural Communication class on Tuesday inspired me to think about language in a different way.

There is a fantastic paper by Charles Goodwin called Professional Vision that shows how people train each other into ways of seeing. The example he uses is an anthropological investigation of soil, and he shows how the new anthropologist is trained by the more experienced anthropologist to notice and describe gradations in the color or quality of the soil. These gradations have a specific meaning within their investigation, and being able to describe them is an important tool of the study.

Language provides two important functions that I want to focus on today. The one we think of most often is the tool function. Language gives us a way to communicate what we see. But it is much less often that we think about language as a way of seeing.

I’ve talked about our constant stimulation before on this blog. We are constantly exposed to more sensory data than we could ever process. Language provides a lens through which to see the world in front of us. Languages have inherent sets of coding structures to apply to the world.

There is another function of language that corpus and computational linguistics really highlights, and that’s habituation. We may have the tools within a language to describe something in any number of ways, but very few of these ways are actually used with any frequency. We learn in corpus linguistics that language is never random. In many ways, in fact, language follows a zipfian distribution, with a high concentration of high frequency words or expressions, and each after the most frequent exponentially less frequent.The language itself may be large, but the language we use is much smaller.

This is an important element to consider in the conceptualization of translation. One person who reviewed the original Russian translation of our Global Survey of Physicists complained that if they did not know English, they would not have understood the Russian. The more I learn about language, the more I understand how that could be possible. Language is not just grammatical, it is habitual, and it holds a way of seeing the world.

I don’t believe this is an idea whose time has come

I ran across this presentation of a Living Social style performance review system:

http://i.bnet.com/video/itunes/12z0315_GeorgeHu_Salesforce_sd_DL.m4v

I’m pretty sure that it is not yet April 1st, so I’m going to assume this program is for real. If it catches on, it represents a giant step toward the gamification of performance reviews.

We are currently in the process of adapting to a new performance review system at my workplace, and the process is immensely complicated. There is definitely some crowd sourcing potential in these systems and room for improvement. But I think that what’s really missing from these systems is a basis for internal motivation. I’m a firm believer that people will naturally try to excel. If people are not trying to excel, it should be taken as a sign that something is disrupting the system. Given that basic assumption, the best kind of performance review system is one that naturally allows, rewards and encourages people to strive to do their best, one that respects and recognizes peoples’ natural talents and interests. Gamification of performance just makes a disrespectful joke out of this. I have to wonder if this is a consequence of an increasingly pervasive artificial system of punishments and rewards. Instead of people finding and experiencing the natural consequences of their actions, are people instead looking for external rewards, such as badges on social media interfaces?

While I truly believe in embracing technology, I don’t believe that this is a step forward. In my ideal work situation, we would all be encouraged to do our best, credited for the work that we do, and given work that we’re most enthusiastic about.

If you think that is naive, please prove me wrong!