How the researchers would have ensured trustworthiness and/or reliability and validity when conducting the research?

The methods used by quantitative and qualitative researchers to establish trustworthiness differ in many ways.  For qualitative researchers, the methods used to establish trustworthiness include credibility, transferability, dependability, and confirmability.  For quantitative researchers, the methods used to establish trustworthiness include internal validity, external validity, reliability, and objectivity.  

Criteria: Truth Value

    Credibility is one method used by qualitative researchers to establish trustworthiness by examining the data, data analysis, and conclusions to see whether or not the study is correct and accurate.  For qualitative researchers, credibility is a method that includes researchers taking on activities that increase probability so that there will be trustworthy findings.  The following are procedures qualitative researchers can use to increase credibility in qualitative studies:

1.  Prolonged engagement is an activity qualitative researchers use to learn traditions and customs of the participants and build trust.  It is crucial for researchers to spend a good amount of time at a site and examine any distortions, including perceptual, selective, and misconstruction of investigator’s questions.

2.  Persistent observation is used to examine credibility by looking in-depth at what the researchers are examining and investigating factor in detail.

3.  Triangulation is an activity used to examine a substantial amount of various sources (i.e., interviewing and observation), methods, investigators, and theories.  Several different investigators are used to examine if one researcher is more or less honest from other team members.  Multiple theories are examined because theories can be interrelated, and findings could result in a function of the similarity of theories.  Contextual validation plays a role in triangulation because it examines the validity of a piece of a study by comparing it with other kinds of evidence on same points to find a similar characteristic style or distortion in a source.

4.    Peer debriefing is used to help make sure none of the researchers are using their biased opinion.  This method consists of researchers asking a colleague or another person to look over the study for credibility and determine if the results seem to align from the data.

5.  Negative case analysis is used to show that not all the data will provide the same result. This improves the credibility of a study because it shows that the researchers are looking over the cases thoroughly, and it allows researchers to present information from a study that does not align with other themes, patterns, and overall results.

6.  Referential adequacy is a method used to store raw data in records to examine later and compare to other future studies to show the credibility of data.

7.  Members checking is used for participants to review the data, analytic categories, interpretations, and conclusions tested with the participants.  This allows qualitative researchers to examine the overall accuracy of the study, and verifying data results.

In contrast to qualitative researchers credibility methods, quantitative researchers use internal validity methods to establish trustworthiness.  Quantitative researchers evaluate trustworthiness by how well the threats to internal validity have been controlled, and the validity of the instruments and measurements used in a study.  These researchers analyze data through using statistical test measures.  Internal validity is supported when changes in the dependent variable happen from only the independent variable, not from other confounding variables.  It is important for quantitative researchers to remember the following possible threats to internal validity: history, selection, maturation, pretesting, instrumentation, treatment replications, subject attrition, statistical regression, diffusion of treatment, experimenter effects, and subject effects.

Criteria: Applicability

    Transferability is another method used by qualitative researchers to establish trustworthiness.  In qualitative studies, transferability means applying research results to other contexts and settings in order to get at generalizability.  Qualitative researchers use this method to provide a detailed description of the study’s site, participants, and procedures used to collect data in order for other researchers to assess whether or not applying the results of one study is a good match, and makes sense to generalize.

In contrast to transferability, quantitative researchers use the method of external validity to establish trustworthiness.  External validity is used to generalize from the research sample to the larger population.  It is crucial for quantitative researchers to examine the sampling technique in determining the trustworthiness of a study.  Researchers use external validity in the form of such things as statistical confident limits to make reasonably accurate statements.  Quantitative researchers must look into the following factors that could affect external validity and generalizability: subjects, situation, time, intervention, and measures.

Criteria: Consistency

    Dependability is a method qualitative researchers used to show consistency of findings.  Qualitative researchers describe in detail the exact methods of data collection, analysis, and interpretation.  This is so the study could be auditable to describe the situation, and for another researcher to follow the study.   The following are ways to show dependability:

1.  There can be no validity without reliability, and no credibility without dependability.

2.  “Overlap methods” as a direct technique to exemplify a kind of triangulation.

3.  “Stepwise replication” as a process of establishing reliability.  This approach requires an inquiry team of at least two people or more who can be separated into two inquiry teams.  The two teams deal with data sources separately and perform their studies apart from one another.  Then, the results between the two teams are compared.

4.  Inquiry audit for a researcher auditor to examine the process of the study and determine its acceptability to the dependability of the study.  The researcher auditor looks into the data, findings, interpretations, and recommendations and looks into whether the study is supported by data and is trustworthy.

For quantitative researchers, reliability is a method used to established trustworthiness.  Quantitative researchers use reliability by examining the consistency of a group of measurements or measuring instruments used in a study (also known as internal consistency).  Researchers also use the test-retest method (also known as stability) to prove reliability by administering one measure to one group of individuals, wait for a certain amount of time, and then readminister the same instrument to the same group.  Equivalence is a measure than can be used to administer two forms of the same test to one group of individuals and then correlate the scores from the two administrations.  Equivalence and stability estimate is another way to examine reliability by administering one form of an instrument and then a second form of the instrument after a certain amount of time to the same group of individuals.  Agreement is another way reliability is measured by raters observing the same behavior and examining whether or not they have similar and consistent results to one another.  Reliability is important to quantitative researcher because it is a basis for validity, and measures whether or not a study obtains the same results each time.

Criteria: Neutrality

    Confirmability is a method used by qualitative researchers to establish trustworthiness.  Confirmability includes an audit trail that includes raw data, such as electronically recorded materials, written field notes, documents, and records.  This method is used for another researcher to be able to verify the study when presented with the same data.  Confirmability is achieved when findings of a study reflect from the participants of the study and make sure the data speaks for itself, and is not based on biases and assumptions of the researchers.

Unlike qualitative researchers method of using confirmability to establish trustworthiness, quantitative researchers use the method of objectivity. Objectivity is used through the methodology of measurements, data collection, and data analysis through which reliability and validity are established.  Objectivity is performed through methodological procedures such as instrumentation and randomization.  Quantitative researchers focus on the facts.  Objectivity also refers to the appropriate distance between a researcher and participants that lessens bias.  The objective researcher is distant so that the researcher is not influenced by the participants, and does not influence the study.      

One last method both qualitative and qualitative researchers can use to establish trustworthiness is the reflexive journal.  This method is a kind of diary in which both qualitative and quantitative researchers can use on a daily basis or as needed for records of a variety of information.  This method can be useful for qualitative researchers in particular in order to to provide information about reflecting upon what is happening in terms of the researcher’s own beliefs and thoughts about the study.  The method of using a reflexive journal can also be useful for quantitative researchers because they can provide information about methodological decisions made and reasons for choosing certain methods, instruments, and data analyses of the study.

How will you ensure reliability and validity in your research?

How do researchers ensure trustworthiness?

How do you ensure validity and trustworthiness in qualitative research?