🔬Communication Research Methods Unit 9 – Content Analysis: Textual Research Methods

What's Content Analysis?

• Systematic and replicable technique for compressing many words of text into fewer content categories based on explicit rules of coding
• Allows researchers to sift through large volumes of data with relative ease in a systematic fashion
• Can be a useful technique for allowing us to discover and describe the focus of individual, group, institutional, or social attention
• Allows inferences to be made which can then be corroborated using other methods of data collection
• Consists of coding raw messages (i.e., textual material, visual images, illustrations) according to a classification scheme
• Useful for examining trends and patterns in documents
• Looks at documents, text, or speech to see what themes emerge, what people talk about the most, and how ideas are related

Why Use Content Analysis?

• Looks directly at communication via texts or transcripts, and hence gets at the central aspect of social interaction
• Can allow for both quantitative and qualitative operations
• Allows a closeness to text which can alternate between specific categories and relationships and also statistically analyzes the coded form of the text
• Can be used to interpret texts for purposes such as the development of expert systems (since knowledge and rules can both be coded in terms of explicit statements about the relationships among concepts)
• Provides insight into complex models of human thought and language use
• When done well, is considered as a relatively "exact" research method (based on hard facts, as opposed to Discourse Analysis)
• Enables researchers to include large amounts of textual information and systematically identify its properties (e.g., the frequencies of most used keywords)

Key Steps in Content Analysis

1. Decide on the level of analysis
   • Code for a single word, a set of words, or phrases
   • Code for a concept, a theme, or an assertion about some subject matter
2. Decide how many concepts to code for
   • Develop a pre-defined or interactive set of concepts and categories
   • Decide on the level of generalization (e.g., concepts like cooperation, or more specific concepts like task coordination)
3. Decide whether to code for existence or frequency of a concept
   • Code for existence: unobtrusive and easy to do
   • Code for frequency: gives more information but takes more time
4. Decide on how you will distinguish among concepts
   • Decide on the level of implication you'll allow (e.g., coding only explicit appearances of a word vs. coding both implicit and explicit appearances)
   • Decide on whether to code for the generalization of a concept or for subtypes of the concept (e.g., coding for all references to emotion or coding for positive and negative emotions)
5. Develop rules for coding your texts
   • Create translation rules (i.e., coding rules) that allow you to streamline and organize the coding process so that you are coding for exactly what you want to know
6. Decide what to do with "irrelevant" information
   • Decide whether irrelevant information should be ignored (e.g., common English words like "the" or "and"), or used to reexamine and/or alter the coding scheme
7. Code the texts
   • Manually code the text or use specialized software
   • Coding can be done by hand, by computer, or by both
8. Analyze your results
   • Draw conclusions and generalizations where possible
   • Relate your results back to your research question(s)

Coding Schemes and Categories

• Coding schemes are the rules used to classify the text
• Categories are the "boxes" into which the coded text is placed
• Coding schemes should be:
  • Mutually exclusive: A text can only be placed into one category
  • Exhaustive: Every text should fit into a category
  • Reliable: Different coders should code the same text in the same way
• Categories can be:
  • A priori: Determined beforehand (e.g., based on a theory)
  • Emergent: Developed through the coding process
• Coding schemes can be:
  • Deductive: Codes are predetermined and then looked for in the data
  • Inductive: Codes emerge from the data and are then applied
• Coding schemes and categories should be:
  • Relevant to the research question(s)
  • Exhaustive (i.e., cover all relevant aspects)
  • Mutually exclusive (i.e., no overlap between categories)
  • Independent (i.e., assignment to one category does not influence assignment to another)

Sampling Techniques

• Depends on the research question and the nature of the data
• Random sampling: Each unit has an equal chance of being selected
  • Simple random sampling: Selecting units at random from the population
  • Stratified random sampling: Dividing the population into strata (subgroups) and then selecting units at random from each stratum
• Non-random sampling: Units are selected based on certain characteristics
  • Purposive sampling: Selecting units that are judged to be typical or representative of the population
  • Convenience sampling: Selecting units that are easily accessible
  • Quota sampling: Selecting units until a predetermined number (quota) is obtained for each category
• Sample size depends on the research question, the nature of the data, and the resources available
  • Larger samples are more representative but require more resources
  • Smaller samples are less representative but require fewer resources

Reliability and Validity

• Reliability refers to the consistency of the coding
  • Intra-coder reliability: The same coder codes the same text in the same way at different times
  • Inter-coder reliability: Different coders code the same text in the same way
• Validity refers to the extent to which the coding scheme measures what it is intended to measure
  • Face validity: The coding scheme appears to measure what it is intended to measure
  • Content validity: The coding scheme covers all relevant aspects of the concept being measured
  • Criterion validity: The coding scheme is related to an external criterion (e.g., another measure of the same concept)
  • Construct validity: The coding scheme is related to other variables as predicted by theory
• Reliability and validity can be improved by:
  • Using clear and precise coding rules
  • Training coders and providing them with coding manuals
  • Conducting pilot studies to test and refine the coding scheme
  • Using multiple coders and assessing inter-coder reliability
  • Comparing the results with other measures of the same concept (triangulation)

Tools and Software

• Manual coding: Coding is done by hand, usually using a coding sheet and a codebook
  • Advantages: Allows for more flexibility and interpretation
  • Disadvantages: Time-consuming and prone to human error
• Computer-assisted coding: Coding is done using specialized software (e.g., NVivo, ATLAS.ti, MAXQDA)
  • Advantages: Faster and more consistent than manual coding
  • Disadvantages: Requires learning how to use the software and may limit flexibility
• Dictionary-based approaches: Coding is done using a pre-defined dictionary of words and phrases
  • Advantages: Fast and easy to use
  • Disadvantages: Limited to the words and phrases in the dictionary and may miss context and nuance
• Machine learning approaches: Coding is done using algorithms that "learn" from a set of training data
  • Advantages: Can handle large amounts of data and can identify patterns that humans may miss
  • Disadvantages: Requires a large amount of training data and may be difficult to interpret
• Choice of tool depends on the research question, the nature of the data, and the resources available
  • Manual coding may be best for small datasets or when flexibility is needed
  • Computer-assisted coding may be best for large datasets or when consistency is important
  • Dictionary-based approaches may be best for simple, well-defined concepts
  • Machine learning approaches may be best for complex, nuanced concepts or very large datasets

Challenges and Limitations

• Sampling bias: The sample may not be representative of the population
  • Solution: Use random sampling techniques when possible
• Coding bias: The coding scheme may be biased or inconsistently applied
  • Solution: Use clear and precise coding rules, train coders, and assess reliability
• Interpretation bias: The interpretation of the results may be biased
  • Solution: Be aware of one's own biases and seek alternative explanations
• Lack of context: The coding scheme may miss important contextual information
  • Solution: Use a combination of quantitative and qualitative methods (e.g., content analysis and discourse analysis)
• Changing meanings over time: The meaning of words and phrases may change over time
  • Solution: Be aware of historical context and use appropriate time periods
• Lack of generalizability: The results may not be generalizable to other contexts
  • Solution: Replicate the study in different contexts and with different samples
• Labor-intensive: Content analysis can be time-consuming and labor-intensive
  • Solution: Use computer-assisted tools when possible and plan for adequate time and resources
• Requires a large amount of data: Content analysis may require a large amount of data to be meaningful
  • Solution: Ensure that the sample size is adequate and consider using data reduction techniques