Category Archives: research methodology

More rundown on Academedia

So I promised more on Academedia (note: they will add more video and visual resources to the Academedia website in the next few days)…

First, some of Robert Cannon’s (employed with the FCC and a member of Panel B “New Media: A closer look at what works”) insightful gems

Re: internet “a participatory market of free speech”

Re: kids& social media “It’s not a question of whether kids are writing. Kids are writing all the time. It’s whether parents understand that.”

“The issue is not whether to use Wikipedia, but how to use Wikipedia”
Next, the final panel, “Digital Tools for Communication:” http://gnovis-conferences.com/panel-c/
Hitlin (Pew Project for Excellence in Journalism)
People communicate differently about issues on different kinds of media sources.
Re: Trayvon Martin case –> largest issue by media source

  •      Twitter: 21% Outrage @ Zimmerman
  •      Cable & Talk radio: 17% Gun control legislation
  •      Blogs: 15% Role of race

Re: Crimson Hexagon
Pew is different, because they’re in a partnership with Crimson Hexagon to measure trends in Traditional media sources. Also because their standard of error is much higher, and they have a team of hand coders available.

Crimson Hexagon is different, because it combines human coding with machine learning to develop algorithms. It may actually overlap pretty intensely with some of the traditional qualitative coding programs that allow for some machine learning. I can imagine that this feature would appeal especially to researchers who are reluctant to fully embrace machine coding, which is understandable, given the current state of the art. I wonder if, by hosting their users instead of distributing programs, they’re able to store and learn from the codes developed by the users?

CH appears to measure two main domains: topic volume over time and topic sentiment over time. Users get a sense of recall and precision in action as they work with the program, by seeing the results of additions and subtractions to a search lexicon. Through this process, Hitlin got a sense of the meat of the problems with text analysis. He said that it was difficult to find examples that neatly fit into boxes, and that the computer didn’t have an eye for subtlety or things that fit into multiple categories. What he was commenting about was the nature of language in action, or what sociolinguists call Discourse! Through the process of categorizing language, he could sense how complicated it is. Here I get to reiterate one of the main points of this blog: these problems are the reason why linguistics is a necessary aspect of this process. Linguistics is the study of patterns in language, and the patterns we find are inherently different from the patterns we expect to find. Linguistics is a small field, one that people rarely think of. But it is critically essential to a high quality analysis of communication. In fact, we find, when we look for patterns in language, that everything in language is patterned, from its basic morphology and syntax, to its many variations (which are more systematic than we would predict), to methods like metaphor use and intertextuality, and more.

Linguistics is a key, but it’s not a simple fit. Language is patterned in so many ways that linguistics is a huge field. Unfortunately, the subfields of linguistics divide quickly into political and educational camps. It is rare to find a linguist trained in cognitive linguistics, applied linguistics and discourse analysis, for example. But each of these fields are necessary parts of text analysis.

Just as this blog is devoted to knocking down borders in research methods, it is devoted to knocking down borders between subfields and moving forward with strategic intellectual partnerships.

This next speaker in the panel thoroughly blew my mind!

Rami Khater from Al Jazeera English talked about the generation of ‘The Stream,’ an Al Jazeera program that is entirely driven by social media analysis.

Rami can be found on Twitter: @ramisms , and he shared a bit.ly with resources from his talk: bit.ly/yzST1d

The goal of The Stream is to be “a voice of the voiceless,” by monitoring how the hyperlocal goes global. Rami gave a few examples of things we never would have heard about without social media. He showed how hash tags evolve, by starting with competing tags, evolving and changing, and eventually converging into a trend (incidentally, Rami identified the Kony 2012 trend as synthetic from the get go by pointing that there was no organic hashtag evolution. It simply started and nded as #Kony2012). He used TrendsMap to show a quick global map of currently trending hashtags. I put a link to TrendsMap on the tools section of the links on this blog, and I strongly encourage you to experiment with it. My daughter and I spent some time looking at it today, and we found an emerging conversation in South Africa about black people on the Titanic. We followed this up with another tool, Topsy, which allowed us to see what the exact conversation was about. Rami gets to know the emerging conversations and then uses local tools to isolate the genesis of the trend and interview people at its source. Instead, my daughter and I looked at WhereTweeting to see what the people around us are tweeting about. We saw some nice words of wisdom from Iyanla Vanzant that were drowning in what appeared to me to be “a whole bunch of crap!” (“Mom-mmy, you just used the C word!”)

Anyway, the tools that Rami shared are linked over here —->

I encourage you to play around with them, and I encourage you and me both to go check out the recent Stream interview with Ai Wei Wei!

The final speaker on the panel was Karine Megerdoomian from MITRE. I have encountered a few people from MITRE recently at conferences, and I’ve been impressed with all of them! Karine started with some words that made my day:

“How helpful a word cloud is is basically how much work you put into it”

EXactly! Great point, Karine! And she showed a particularly great word cloud that combined useful words and phrases into a single image. Niiice!

Karine spoke a bit about MITRE’s efforts to use machine learning to identify age and gender among internet users. She mentioned that older users tended to use noses in their smilies 🙂 and younger users did not 🙂 . She spoke of how older Iranian users tended to use Persian morphology when creating neologisms, and younger users tended to use English, and she spoke about predicting revolutions and seeing how they are propagated over time.

After this point, the floor was opened up for questions. The first question was a critically important one for researchers. It was about representativeness.

The speakers pointed out that social media has a clear bias toward English speakers, western educated people, white, mail, liberal, US & UK. Every network has a different set of flaws, but every network has flaws. It is important not to just use these analyses as though they were complete. You simply have to go deeper in your analysis.

 

There was a bit more great discussion, but I’m going to end here. I hope that other will cover this event from other perspectives. I didn’t even mention the excellent discussions about education and media!

Don’t fear Big Data

I really enjoyed this RTI blog post about embracing big data:

https://blogs.rti.org/surveypost/2012/03/22/why-you-should-not-fear-but-embrace-the-age-of-big-data/

I suspect that oftentimes fear of big data is motivated by a concern that new, less tested, still evolving methods will replace the time tested methods that we have grown to have so much faith in. I sincerely believe that the foundation that we have is a strong one, and the knowledge we have developed through those processes should be embraced, especially the quality controls. But SUPPLEMENTING an analysis through a measured combination of data sources can lead to a more complete picture.

This week I spent some time analyzing Pew’s report on the Kony 2012 video. I believe that this report is an excellent example of what researchers are capable of when they look outside the artificial divisions of research group (this was a collaborative effort) and research methodology. Seven days after the release of the video, Pew was able to reconstruct a comprehensive narrative of the video’s dissemination, using traditional survey methods, sentiment analytic snapshots over time, and a careful breakdown of the media coverage of influential parties.

 

Dana Boyd also has an interesting analysis of the Kony phenomena on her Apophenia blog:

http://www.zephoria.org/thoughts/

Zen as a Research Ethic

I have a Zen calendar on my desk for 2012. It has such gems as: “Although the world is full of suffering, it is also full of the overcoming of it” (Helen Keller)

The more I look at the calendar, the more it relates to everything I think about.

I read “To see is to forget the name of the thing one sees,” (Paul Valery) and I think of the Charles Goodwin paper I cited in a recent post about Professional Vision. He talks about ways of seeing as kind of coding structures, inculturation, or ways of foregrounding certain parts of what we see. Truly, being able to see deeper than that requires shedding that inculturation and observing more closely. As researchers, we often become so deeply incultured into our way of thinking, that we lose sight of our research goals. As survey researchers, we can easily fall into the pattern of first asking “who should we survey?” and “what should we ask?” before taking more time to consider whether a survey is even an appropriate methodology for the specific topic of focus. Of course, not this action based on praxis is not limited to survey researchers. Far from it! Every person, every field, every community of practice, every language has a way of thinking. And often instead of seeing or observing, we quickly begin to navigate our networks of inculturation.

These two are similarly meaningful in my interpretation:

“Zen is not to confuse spirituality with thinking about God while one is peeling potatoes. Zen is just to peel the potatoes.” (Alan Watts)

“If all beings are Buddha, why all this striving?” (Dogen)

These are a reminder to boil things down to what they simply are and not try to describe them as what you want them to be. In survey research, this comes up often in the process of reporting research results. If I know that I intended to measure something about Project Based Learning or STEM education, it is easily for me to begin to frame my findings by my intentions. But that is not true to my findings or my methodology, and it doesn’t make for good research. I can’t say that 10% of my respondents were using project based learning methods in the classroom if I asked about the number of group activities they conducted. I must simply say that 10% were using group activities (daily/monthly/occasionally- whatever the answer choices were)

In this way, my Zen calendar not only provides something to think about in a larger sense, but it keeps my research anchored.