The future doesn’t belong to big or small data. It belongs to the disruptors.

Research is evolving fast. There is less support and more doubt for traditional methods, a fast- changing set of expectations from end-users, and a fast-evolving field of nontraditional methods and approaches. The future of research does not belong to big data or small data. It belongs to the disruptors. It belongs to those that can recognize and challenge the assumptions underlying their methodologies. The future belongs to creative approaches, connected data, and collaboration.


Research requires listening and understanding.

In order to create research that is useful, there needs to be a deep understanding of end-users, clients, and the context of the products we create. This requires listening, understanding, and creating opportunities to learn more, both by representing end users and clients more directly in the development process and by qualitative research methods. Qualitative research provides methods of collecting and analyzing information about people, in-person, virtually and through behavioral data sources, and it must provide a vitally important role in evolving research methods.


Research requires ever-changing analytic capabilities and creative, open minds.

We live in an era when data is plentiful. But the data looks different from what we saw in the past. We need capable and versatile technical workers who are able to process data. And we need the creativity to put the data to use in ways that benefit end-users.


Research must embrace diversity.

Creative strategy and good user-focus can’t spring from echo chambers. We need to connect to divergent experiences and views early and often in order to create good products. People with divergent views can raise questions earlier in the development process and allow us to integrate holistic solutions for problems we could not have thought of alone. Diverse experiences allow us to be more creative because they provide more material to inspire us. And diversity is crucial for us to successfully compete in the global marketplace.


Research design must be iterative.

If we want to create new ways of analyzing and connecting data, we have to be free to experiment with new methods, test new methods, and allow end-users to test proposed solutions. Often what we create doesn’t function the way that we expect it to. In an era where data does not need to be designed and collected, we have the flexibility to find creative ideas (“ugly babies”), nurture them, test them out, and tweak what doesn’t work.


Silos no longer make sense in research

It no longer makes sense to separate end users from developers or quantitative from qualitative. The best disruptive, creative potential lies in the mingling of methods and people. The most useful products are the ones that can be created collaboratively.


Research can be agile.

Agile development has become standard practice in much of the software development world, but it makes sense for research as well. Agile teams can involve end-users, UX researchers, quantitative methodologists and qualitative methodologists. Research can be built by agile, creative teams that feel free to question and inspire each other.


Creating high performing teams has never mattered more.

There is a growing body of great research about what makes a highly effective team. Effective teams are empathetic and open. They consider each other’s interests. They are practical and focused on the end product. They are comfortable asking questions and brainstorming solutions. They work collaboratively, and they celebrate their accomplishments.


The future of research is bigger than any one person or silo. It requires us to come together in new ways. I already see some firms moving in this direction- kudos to them. A new era is here, and I’m excited for us all!

Academic register: Are we smarter than a 5th grader?

Another important area of study in Linguistics is register. Among other things, register refers to the degree of formality with which we communicate. We speak differently with our friends and family than we do at work. We speak differently in a courtroom than in a courtroom lobby. And we speak differently in academia than we do elsewhere.

In an interview about Betty Friedan’s ‘The Feminine Mystique,’ Naomi Wolf discusses the groundbreaking impact of Friedan’s classic work. She praises Friedan for having the courage to release countless hours of research to a wider audience, rather than an academic audience. To do this, Friedan sacrificed the academic recognition that could have accompanied her work in order to reach a broader population who could potentially benefit from her work. The choice to write in a less respected register opened Friedan to criticism from academics, but led to a broad, longstanding appeal.

Wolf takes this point a step further by suggesting that academics write in such a way that we don’t even understand each other (!!).

This was a surprising admission to see from an academic. Academics often really embrace the large words and complicated nature of their findings. They manage to encode large amounts of information and complicated ideas in relatively small amounts of space. But are the conclusions and information that we publish limited in their usefulness by the academic register itself?

I’ve mentioned before that Linguistics is a very broad area of study. Coming to the field with absolutely no prior background, I was really struck by the different definitions of terms I used regularly in my profession, like reliability, validity, sampling, representative sample, … It took a while for me to adjust to the different context of those terms, and to the different lexicon and areas of focus in linguistics. And the more areas of linguistics I study, the more I find words and concepts from fields I have little experience with. I remember reading and rereading papers in my conversation analysis class, trying to understand what they were doing and why- and it took the whole semester for me to be able to imitate that academic genre and understand its power.

Clearly, the more experience we have with specific words and methods, the more easily we can understand a specific genre of academic writing. This is the academic genre at its best, and it enables us to reach complicated conclusions that we might not be able to make otherwise. But it is also quite restrictive. Nobody can be an expert in all fields, and research could potentially benefit from feedback from a much wider variety of fields than it often receives. This enforces a linguistic segregation represents is the academic genre at its worst.

Yesterday I attended a talk that involved areas and methods of study that I had never encountered before. I heard a talk about textual features that evoke emotion. The talk was heady, showing logical expressions and cognitive space diagrams, and involving some of what I believe is called semantic formalism. The text examples were mostly poems, which naturally add to the complexity of the analysis. The main points were that the use of complexity and negation in text add to the emotional wallop of a body of text. He used hypotheticals as an example of constructed negation that evokes emotion. After trying and trying to wrap my head around his points and their wider applicability, I thought of funerals and memorial services. I thought of how we use hypotheticals to make us cry and help the grieving process. I mentioned to the speaker that, as difficult as it is to wrap my head around his talk, I realized that we use these devices as tools to evoke emotion regularly in those situations.

In my mind, research is of the best quality when it is anchored in something palpable or readily accessible. As a poet, I have a distinct sense of trying to create contrasts and develop layers of complication as poetic devices. But that sense isn’t as visceral or accessible as grieving communication is.

I wonder what his research would have gained by borrowing from other registers. In my own research, I believe that explaining my work to my family or friends is a critical part of my research process. It helps me to make grounded conclusions, and it guides my research questions and methods. For yesterday’s speaker, surely it would help him to generate better, more wide ranging feedback from a wider variety of people?

At the end of the day, I went out for dinner with my kids. I mentioned the talk to my 10 year old, who loves to discuss emotions. I was surprised to see that not only did she ‘get it’ in one or two sentences of explanation, but she was able to generate some really excellent examples of these devices in a 5th grade register.

Calling Respondents Stupid

From a Politico article titled “How much do voters know?” by Alexander

“The first lesson you learn as a pollster is that people are stupid,”
said Tom Jensen of Public Policy Polling, a Democratic polling firm. “I
tell a client trying to make sense of numbers on a poll that are
inherently contradictory that at least once a week.”

Full article at:

Message to Tom Jenson:
It’s not so much that people are stupid as it is that Tom Jenson is ignorant of the cognitive underpinnings of response methodology and filling in his gaps with unnecessary condescension. Truly, all survey researchers deal with these contradictions. We might be able to garner more consistency in our response sets if we could instead survey computers or robots, but, alas, that’s not why we conduct surveys.

Renewing my Vows?

Anne Steen came to our Proseminar class last week to discuss personality types, the strong skills inventory, and the search for an ideal job. She used an excellent analogy to describe the difference between doing work that we are and are not suited for. She said that doing work that we are well suited for is like using our dominant hand to write. We do it without really thinking about it. Writing with our other hand is possible, but it is more difficult. I have always felt that way about my work. I remember when I first began to work in research. I had done other kinds of work before, some of which I had enjoyed more than others. But work in research was almost meditative, because it came so naturally to me.

Because of that feeling, it has always been hard for me to see outside my field. I don’t feel as though I have a great understanding of what other people do in their work on a day to day basis, and I can’t easily envision myself doing any different kind of work. The results of the strong skills inventory and MBTI reinforced my contentedness with my current position. Many of my daily work activities were listed as work activities that I would particularly enjoy. I feel like, professionally speaking, I married my first love and am too content in my marriage to imagine being with anyone else. In fact, I work at an organization where people tend to stay for their entire careers, so the metaphor is particularly apt. In a culture where people don’t consider looking elsewhere or changing jobs, it feels particularly backhanded to explore other options. In this way, I feel like I have seen my matches on a dating website, and my partner was right on top of the list of potential matches.

Because of this, I feel like I need to go deeper in order to ‘think outside the box.’

One aspect of my current job as a survey methodologist that I really love is designing surveys. For paper surveys, I really love obsessing about the color of the survey and the mood it will immediately evoke. I love playing with space. I love the way that each aspect of the space is meaningful and every design element must be consistent, because it will be taken as meaningful. I love the way rearranging the questions changes the context and meaning of the questions themselves. I love reading the research about what people like to encounter first, and what you need to squirrel away last, and what questions increase respondent’s confidence in the survey. I love the crisp, professional look of a well-designed survey, and I love the way each aspect of the design is based on research.

I also love designing web surveys. Many of my other job activities are based on programming, so working with CSS really resonates with me. I started out working with html and then discovered CSS, so the flexibility of CSS was a real revelation. It turned out that I could set text styles, just like on a gui editor, design all aspects of their appearance, and tweak them with ease. Tweaking margin spaces until I feel content looking at them appeals to my artistic sensibilities. I really love survey design for its technical challenges and artistic rewards. When I am in a design phase, I even tend to read design books, like Norman Cook’s The Design of Everyday Things or Steve Krug’s web design classic Don’t Make Me Think, in my spare time.

Although I really love most aspects of my job, survey design is the one aspect that I have volunteered to specialize in. My small department’s workflow style has been best described as “jack of all trades, master of none,” but I have approached my directors and told them that we could use an expert in survey design, because it’s such an important part of collecting quality answers. Since then our tech guru has created a survey template, so I no longer get to do much survey design. I am interested in finding new ways to apply those skills, or ways to bring those skills back into my work life.

However, I worry about this kind of a step, because I’m so interested in research methodology.

I suppose that having too many interests is not a bad problem to have!