FIC0154 Basic Comm Research (16th October 2012)
Reference materials: Lecture notes 4 and relevant text book
chapters(Mass Media Research; An Introduction, 9th edition by Roger D. Wimmer & Joseph R. Dominick; Pg 26-31).
Question 1
Scientific research must be internally and externally valid.
What do Internal Validity and External Validity mean?
ANSWER:
Note that internal validity and external validity are not the opposite of each other.
Internal validity happens when the researchers are able to control all the variables efficiently so that the studies and results are not affected by any unwanted variables. Factors that affect internal validity are called artifacts.
External validity refers to how much the results of a study can be generalized or extended to others. For example, the results of a study cannot be generalized to the rest of the population if it was only conducted on a certain gender or race.
ANSWER:
Note that internal validity and external validity are not the opposite of each other.
Internal validity happens when the researchers are able to control all the variables efficiently so that the studies and results are not affected by any unwanted variables. Factors that affect internal validity are called artifacts.
External validity refers to how much the results of a study can be generalized or extended to others. For example, the results of a study cannot be generalized to the rest of the population if it was only conducted on a certain gender or race.
Question 2
What do researchers need to take into account when ensuring
that their research results are externally and internally valid?
ANSWER:
To ensure their research results are valid, researchers must first be aware of the factors that are affecting it, including internal and external validity.
INTERNAL VALIDITY
Factors that can affect internal validity include history, maturation, testing, instrumentation and experimental mortality.
a) History:
This means that a lot of different kinds of events that happened prior or during the study that could affect the subject's opinions, attitudes and/or behaviour.
Examples:
i) When a famous and somewhat considered 'legendary' singer like Michael Jackson passed away, it would affect the subject's original answer when asked for their favourite pop singer (i.e; the answer could have been Justin Bieber if Michael Jackson have not passed away at that moment).
ii) When researchers are trying to conduct a study on accident rates due to reckless driving in Malaysia during the monsoon season, their results would be inaccurate. This is because accidents are more prone to happen because of the heavy rain, rather than because of reckless driving.
A way to effectively counter this factor would be to conduct the research at the right timing and make sure there are no events that could manipulate the subject's answers before conducting a study.
This means that a lot of different kinds of events that happened prior or during the study that could affect the subject's opinions, attitudes and/or behaviour.
Examples:
i) When a famous and somewhat considered 'legendary' singer like Michael Jackson passed away, it would affect the subject's original answer when asked for their favourite pop singer (i.e; the answer could have been Justin Bieber if Michael Jackson have not passed away at that moment).
ii) When researchers are trying to conduct a study on accident rates due to reckless driving in Malaysia during the monsoon season, their results would be inaccurate. This is because accidents are more prone to happen because of the heavy rain, rather than because of reckless driving.
A way to effectively counter this factor would be to conduct the research at the right timing and make sure there are no events that could manipulate the subject's answers before conducting a study.
b) Maturation:
This happens when there are biologically and psychologically characteristics changes during the course of the research. Subjects may be tired, hungry or even older and this would influence greatly on the study.
Example:
i) If asked to complete a survey after having spent 4 hours on the computer(or even doing the survey for four hours straight), the data would be inaccurate because the respondents would be too tired and would just randomly click on the answers for the sake of finishing the survey. When that happens, the survey would be completely pointless.
To counter this factor, the subjects should be given a few break sessions (by providing coffee, tea or plain water and snacks) in between completing the surveys so that the respondents would be more alert and do not feel too bored/tired and hungry. Also, the temperature where the respondents are completing the survey should neither be too cold nor too warm (*roughly 21-23°C). This is to provide the respondents a comfortable and conducive environment.
*Standard room temperature
c) Testing:
This means that researches do a pretest(survey questions) so that they could improve their questions. However, this could backfire and be considered as an artifact that could affect the results.
Example:
i) If the subjects have already done the pretests, the real test would be predictable and the subjects might give biased answer, since they already know how to answer the questions.
This happens when there are biologically and psychologically characteristics changes during the course of the research. Subjects may be tired, hungry or even older and this would influence greatly on the study.
Example:
i) If asked to complete a survey after having spent 4 hours on the computer(or even doing the survey for four hours straight), the data would be inaccurate because the respondents would be too tired and would just randomly click on the answers for the sake of finishing the survey. When that happens, the survey would be completely pointless.
To counter this factor, the subjects should be given a few break sessions (by providing coffee, tea or plain water and snacks) in between completing the surveys so that the respondents would be more alert and do not feel too bored/tired and hungry. Also, the temperature where the respondents are completing the survey should neither be too cold nor too warm (*roughly 21-23°C). This is to provide the respondents a comfortable and conducive environment.
*Standard room temperature
c) Testing:
This means that researches do a pretest(survey questions) so that they could improve their questions. However, this could backfire and be considered as an artifact that could affect the results.
Example:
i) If the subjects have already done the pretests, the real test would be predictable and the subjects might give biased answer, since they already know how to answer the questions.
To counter this factor would be to use different set of subjects for the pretest and the actual test. That way, the subjects who are doing the actual test would not be able to predict the questions and the results would not be affected.
d) Instrumentation:
It is also known as instrument decay. This refers to the sudden failure of the instruments or methods that you use to measure your variables.
Example:
i) Instrument wears out; When trying to measure the time taken(speed) for a group of teenagers to type a passage, the stopwatch stops working after 10 seconds. This will result in failure to collect data.
ii) Research assistants too casual; This is when the researchers (or their assistants) themselves are too casual in recording their observations or they might be too tired that their attention span was shorten during an interview. This will lead to having results that would most likely not be accurate.
Ways to counter this would be to double check the instruments (stop watch, recorder, typewriter etc.) to make sure they are in tip top condition before use and also have backup instruments. Researchers should also make sure they have enough sleep and are not in a tired state when conducting a study.
i) Instrument wears out; When trying to measure the time taken(speed) for a group of teenagers to type a passage, the stopwatch stops working after 10 seconds. This will result in failure to collect data.
ii) Research assistants too casual; This is when the researchers (or their assistants) themselves are too casual in recording their observations or they might be too tired that their attention span was shorten during an interview. This will lead to having results that would most likely not be accurate.
Ways to counter this would be to double check the instruments (stop watch, recorder, typewriter etc.) to make sure they are in tip top condition before use and also have backup instruments. Researchers should also make sure they have enough sleep and are not in a tired state when conducting a study.
e) Experimental mortality:
It should be expected that it is possible for subjects to drop out/quit from being part of a research study for one reason or another.
It should be expected that it is possible for subjects to drop out/quit from being part of a research study for one reason or another.
Example:
i) In search of the perfect vaccine to cure a certain illness, doctors might have been conducting his study on 10 patients. However, if the 8 of the patients passed away or decided that they do not what to be part of the research anymore, it would effect the study greatly.
ii) If 500 surveys were to be given to people to fill, it is possible that only 50% of the surveys that came back to the researchers were fully answered.
A way to counter this would be to have at least two times(2x) more research subjects than required. If there are more than enough subjects, the results could be greatly improved and if there are any dropouts, it would not affect the results.
EXTERNAL VALIDITY
The only factors that affect external validity are the interaction in an analysis of variables such as subject selection, instrument and experimental conditions.
Example:
i) When researches want to conduct a study on Malaysians who have diabetes, their sample only consists of middle aged women where 80% of the subjects are Malays and 20% Chinese with no samples of Indian race. This study cannot be generalized to the population of Malaysia and would be considered inaccurate because of the wrong usage of samples.
Ways to encounter this would be:
a) Use random samples.
This enables the researchers to collect date from a variety of different subjects rather than collecting data from people who share the same attitudes, opinions and lifestyles. Using this, everyone would have an equal chance to be selected as part of the research study.
b) Use heterogeneous samples and replicated (repeat) the study several times.
Results are only able to be generalized if the information gathered by different subjects(ie; different race) show the same results. *By repeating the study several times with the same results, we can conclude that the results are accurate.
The only factors that affect external validity are the interaction in an analysis of variables such as subject selection, instrument and experimental conditions.
Example:
i) When researches want to conduct a study on Malaysians who have diabetes, their sample only consists of middle aged women where 80% of the subjects are Malays and 20% Chinese with no samples of Indian race. This study cannot be generalized to the population of Malaysia and would be considered inaccurate because of the wrong usage of samples.
Ways to encounter this would be:
a) Use random samples.
This enables the researchers to collect date from a variety of different subjects rather than collecting data from people who share the same attitudes, opinions and lifestyles. Using this, everyone would have an equal chance to be selected as part of the research study.
b) Use heterogeneous samples and replicated (repeat) the study several times.
Results are only able to be generalized if the information gathered by different subjects(ie; different race) show the same results. *By repeating the study several times with the same results, we can conclude that the results are accurate.
c) Select a sample that is representative of the group to which the results will be generalized.
This just means that a study conducted on a particular group of people cannot be considered applicable to another. For example, the results on a study on what housewives like to watch cannot be applicable to teenagers or children.
d) Conduct research over a long period of time.
Opinions can be changed in a short period of time, thus, if a research is done for well over weeks or months, it can be safe to conclude that any changes would be minimal or nonexistent. This can also be called taking into account the "sleeper" effect.
This just means that a study conducted on a particular group of people cannot be considered applicable to another. For example, the results on a study on what housewives like to watch cannot be applicable to teenagers or children.
d) Conduct research over a long period of time.
Opinions can be changed in a short period of time, thus, if a research is done for well over weeks or months, it can be safe to conclude that any changes would be minimal or nonexistent. This can also be called taking into account the "sleeper" effect.
*Insanity is doing the same thing over and over again and expecting different results - Albert Einstein*
[Refer to b)]
Uploaded by,
Tee Mieng (MANDY) (0309219)
Tee Mieng (MANDY) (0309219)