Tag Archives: research methodology

Janet Harkness- the passing of a great mind in survey research

I just received this announcement from WAPOR. This is sad news. I attended an AAPOR short course that she helped to teach in the Spring of 2010, and she was very sharp, insightful and kind. At the time I was researching multilingual, multinational and multicultural surveys, and her writing was one of my mainstays.

“Janet Harkness died on Memorial Day (May 28, 2012) in Germany at age 63.  Harkness was the Director of the Survey Research and Methodology graduate program and Gallup Research Center, and holder of the Donald and Shirley Clifton Chair in Survey Science at the University of Nebraska-Lincoln.  She was the founder and Chair of the Organizing Committee on the International Workshop on Comparative Survey Design and Implementation (CSDI).  Her many contributions to cross-national and cross-cultural survey research included service as Head of the International Social Survey Programme’s Methodology Committee (1997-2008),  board member of the National Science Foundation’s (USA) Social, Behavioral & Economic Sciences Advisory Board (2008-present),  board member of the  Deutsches Jugendinstitut (Germany) Advisory Board (2009-present), Co-initiator of the Cross-Cultural Survey Guidelines Initiative, Chair of the Organizing Committee for the International Conference on Survey Methods in Multicultural, Multinational and Multiregional Contexts (3MC, Berlin 2008), and member of the European Social Survey’s (ESS) Central Coordinating Team. The ESS was awarded the European Union’s top annual science award, the Descartes Prize, in 2005.  She has been a member of WAPOR since 2009.

Besides her substantial contributions and organizational achievements in cross-national survey research, Harkness made major contributions to the scholarly literature including Cross-Cultural Survey Equivalence (1998), Cross-Cultural Survey Methods (with F.J.R. Van de Vijver and P. Ph. Mohler, 2003), and Survey Methods in Multicultural, Multinational, and Multiregional Contexts (with M. Braun, B. Edwards, T.P. Johnson, L.F Lyberg, P. PH. Mohler, B. Pennell and T.W. Smith, 2010).

As her professional colleague, Don Dillman, Regents Professor at Washington State Unversity, noted of Janet “I don’t know of anyone who has done as much thinking as she has about cross-cultural surveys, and how measurement differs across languages and countries…That’s one of the major challenges we now face in doing surveys as we increasingly shift to a world-wide emphasis in survey design.”

She is survived by her husband Peter Ph. Mohler.”

——-

10/15/12 Edited to add some great news:

Announcement for the Janet A. Harkness Student Paper Award

The Janet A. Harkness Student Paper Award will be issued annually, starting in 2013, by the World Association for Public Opinion Research (WAPOR) and the American Association for Public Opinion Research (AAPOR) to honor the memory of Dr. Harkness and the inspiration she brought to her students and colleagues.

In particular, WAPOR and AAPOR will consider papers related to the study of multi-national/ multi-cultural/multi-lingual survey research (aka 3M survey research), or to the theory and methods of 3M survey research, including statistics and statistical techniques used in such research. Paper topics might include: (a) methodological issues in 3M surveys; (b) public opinion in 3M settings; (c) theoretical issues in the formation, quality, or change in 3M public opinion; (d) or substantive findings about 3M public opinion. The competition committee encourages submissions that deal with the topics of the annual conferences, for which the call for papers are posted on both associations’ websites in the fall.

Submissions to the Harkness award competition are anticipated to be 15-25 pages in length. A prize of $750 will be awarded to the winning paper and the author(s) of the paper will be invited to deliver it as a part of either the annual WAPOR conference, AAPOR conference, or in certain years the WAPOR-AAPOR joint conference.

For a winning paper with one author, WAPOR and AAPOR will pay for the author’s travel expenses to and from the nearest WAPOR or AAPOR annual conference for that year. However, for a winning submission with multiple authors, WAPOR and AAPOR will pay only for the primary author (or his/her designee, who must be a co-author) to present the paper. Up to two other papers each year may receive an Honorable Mention designation with each receiving a $100 cash prize (though no travel expenses).

All authors must be current students (graduate or undergraduate) at the time of the submission, or must have received their degree during the preceding calendar year. The research must have been substantially completed while the author was (all authors were) enrolled in a degree program. Preference will be given to papers based on research not presented elsewhere.

A panel of public opinion researchers from WAPOR and AAPOR’s membership – drawn from academic, government, and commercial sectors – will judge the papers.

The 2013 Call-for-Submission of papers for the Harkness Award will be issued in the near future.

Memory is incomplete experience

Today’s quote on my zen calendar is perfect.

“Memory is incomplete experience.” J. Krishnamurti

This is a great reminder for researchers and for people in general, because we all forget and keep forgetting how incomplete our memories and the memories of people we come into contact are. How many survey questions could be better written with this advice? How much better is ethnography when we base our observations on repeated viewings, rather than trying to reconstruct a vague memory? How many arguments could be avoided, if we could just remember that memories are incomplete?

I sometimes participate in a video discussion group. I am amazed that each viewing of a short segment of video brings a different set of interpretations, and I am amazed that the other participants continually notice different aspects of the video. This experience really drives the point home about how little we see in our everyday lives. We are so inundated with information that we simply couldn’t, and wouldn’t want to, process it all.

Research is the process of recovering and reconstructing. Of observing carefully. Of noticing things that we would never or could never have accessed through normal observation and we absolutely could never access through our memories. Being a researcher does not make us any more able to analyze that which we experience in a single pass- we’re still human. Being a researcher simply means that we have the capacity to observe and investigate things more closely.

Academic register: Are we smarter than a 5th grader?

Another important area of study in Linguistics is register. Among other things, register refers to the degree of formality with which we communicate. We speak differently with our friends and family than we do at work. We speak differently in a courtroom than in a courtroom lobby. And we speak differently in academia than we do elsewhere.

In an interview about Betty Friedan’s ‘The Feminine Mystique,’ Naomi Wolf discusses the groundbreaking impact of Friedan’s classic work. She praises Friedan for having the courage to release countless hours of research to a wider audience, rather than an academic audience. To do this, Friedan sacrificed the academic recognition that could have accompanied her work in order to reach a broader population who could potentially benefit from her work. The choice to write in a less respected register opened Friedan to criticism from academics, but led to a broad, longstanding appeal.

Wolf takes this point a step further by suggesting that academics write in such a way that we don’t even understand each other (!!).

This was a surprising admission to see from an academic. Academics often really embrace the large words and complicated nature of their findings. They manage to encode large amounts of information and complicated ideas in relatively small amounts of space. But are the conclusions and information that we publish limited in their usefulness by the academic register itself?

I’ve mentioned before that Linguistics is a very broad area of study. Coming to the field with absolutely no prior background, I was really struck by the different definitions of terms I used regularly in my profession, like reliability, validity, sampling, representative sample, … It took a while for me to adjust to the different context of those terms, and to the different lexicon and areas of focus in linguistics. And the more areas of linguistics I study, the more I find words and concepts from fields I have little experience with. I remember reading and rereading papers in my conversation analysis class, trying to understand what they were doing and why- and it took the whole semester for me to be able to imitate that academic genre and understand its power.

Clearly, the more experience we have with specific words and methods, the more easily we can understand a specific genre of academic writing. This is the academic genre at its best, and it enables us to reach complicated conclusions that we might not be able to make otherwise. But it is also quite restrictive. Nobody can be an expert in all fields, and research could potentially benefit from feedback from a much wider variety of fields than it often receives. This enforces a linguistic segregation represents is the academic genre at its worst.

Yesterday I attended a talk that involved areas and methods of study that I had never encountered before. I heard a talk about textual features that evoke emotion. The talk was heady, showing logical expressions and cognitive space diagrams, and involving some of what I believe is called semantic formalism. The text examples were mostly poems, which naturally add to the complexity of the analysis. The main points were that the use of complexity and negation in text add to the emotional wallop of a body of text. He used hypotheticals as an example of constructed negation that evokes emotion. After trying and trying to wrap my head around his points and their wider applicability, I thought of funerals and memorial services. I thought of how we use hypotheticals to make us cry and help the grieving process. I mentioned to the speaker that, as difficult as it is to wrap my head around his talk, I realized that we use these devices as tools to evoke emotion regularly in those situations.

In my mind, research is of the best quality when it is anchored in something palpable or readily accessible. As a poet, I have a distinct sense of trying to create contrasts and develop layers of complication as poetic devices. But that sense isn’t as visceral or accessible as grieving communication is.

I wonder what his research would have gained by borrowing from other registers. In my own research, I believe that explaining my work to my family or friends is a critical part of my research process. It helps me to make grounded conclusions, and it guides my research questions and methods. For yesterday’s speaker, surely it would help him to generate better, more wide ranging feedback from a wider variety of people?

At the end of the day, I went out for dinner with my kids. I mentioned the talk to my 10 year old, who loves to discuss emotions. I was surprised to see that not only did she ‘get it’ in one or two sentences of explanation, but she was able to generate some really excellent examples of these devices in a 5th grade register.