according to http://www.socialresearchmethods.net/kb/quasird.php, this design is suitable for evaluation of effectiveness of program.
using the http://weelookang.blogspot.sg/2013/08/experimental-research-designs.html
characteristics:
characteristics:
- Usually used for evaluating program effectiveness that cannot do random assignment
- Participants are assigned to programs based on a pre-program measure—advantageous as program can be given to those who need/deserve it.
- Treatment and comparison are intentionally made different
- Inferences which are drawn from a well-implemented RDD are comparable in internal validity to conclusions from randomized experiments (Shadish, Galindo, Wong, Steiner, & Cook, 2011)
From Shadish et al, 2002
- Assignment variable (AV) must not be caused by treatment
- AV often assesses merit or need
- more than one AV can be used
- AV best when continuous variable
- Need more sample (about 3x the randomized expts) (Capelleri et al., 1994)
WEAK 3 Two groups, Non-random Selection, Non-equivalent, Pre-test, Post-test
Group
|
Pre-test
|
Treatment
|
Post-test
|
Experimental group = E
|
O
|
X
|
O
|
Control Group = C
|
O
|
O
|
 |
| Regression-Discontinuity Design with roughly a 4 percentage improvement point with cut-off of below 18/24 marks for Treatment Effect (Use of Ripple Tank simulation we customized in 2012) over the control group with traditional lesson. https://www.dropbox.com/s/july6ai0blznek3/regressiondiscontinutymodel.xlsx |
Threats to internal validity
2.1 Ambiguous temporal precedence Which is independent variable to cause the effect on which dependent variable. My research is clearly use of computer models (IV) to cause gains in learning (DV) and affective domains such as perceptions of enjoying learning (DV2).
2.2 Confounding existence of third variable to cause effect dependent variable to change. interviews with students/teachers suggests nil third variable.
2.3 Selection bias Differences in control/experimental groups during pre-test. mostly convenient samplings where we try to ensure equivalent groups through O level physics grade, pre-test scores.
2.4 History Events outside of the study/experiment or/and post-tests between repeated measures of the dependent variable. duration of treatment is short 3 weeks, with 1 post test only, minimises history threats
2.5 Maturation Subjects change during the course of the experiment or even between measurements. duration of treatment is short 1- 3 week(s) typically, should be minimal maturation of subject threat
2.6 Repeated testing (also referred to as testing effects) Repeatedly measuring the participants may lead to bias. only 1 post test.
2.7 Instrument change (instrumentality) instrument used during the testing process can change the experiment. post test change agreed! what i found in pre/post test questions different from treatment, need to craft the post test question to correctly test the learning in the treatment.
2.8 Regression toward the mean extreme outliers on individual scores are more likely to be captured in one instance of testing but will likely evolve into a more normal distribution with repeated testing. data will speak to me but it is likely to happen for high scores. have a control group. Xpre = T(true score)+E(random error), statistical adjustment Xpre = Xpre +(1-rxx) (X mean-Xpre)
2.9 Mortality/differential attrition only those participants that have participated from the start to the end are measured. safely assumption minimum for 1 week to 3 weeks treatment
2.10 Selection-maturation interaction minimum for 1 week to 3 weeks treatment
2.11 Diffusion treatment effects spread from experimental/treatment groups to control groups. tell subjects everyone will experience the same treatment, just the order different.
2.12 Compensatory rivalry/resentful demoralization control group members may work extra hard to see that expected superiority of the experimental group. agreed! we didn't check/noted
2.13 Experimenter bias teachers who are conducting an experiment inadvertently affect the outcome by non-consciously behaving differently to members of control and experimental groups.since teacher is key to learning, it is definitely going to affect the learning gains if the experimenter is unfamiliar with the learning practices in treatment. agreed! noted.
2.14 Instrument/research re-activity (Neumann, 2006) act of measuring produces change is DV. no observer in the class, so the chance of the subject knowing they are being treated is low. Teacher conduct lessons like normal.
2.2 Confounding existence of third variable to cause effect dependent variable to change. interviews with students/teachers suggests nil third variable.
2.3 Selection bias Differences in control/experimental groups during pre-test. mostly convenient samplings where we try to ensure equivalent groups through O level physics grade, pre-test scores.
2.4 History Events outside of the study/experiment or/and post-tests between repeated measures of the dependent variable. duration of treatment is short 3 weeks, with 1 post test only, minimises history threats
2.5 Maturation Subjects change during the course of the experiment or even between measurements. duration of treatment is short 1- 3 week(s) typically, should be minimal maturation of subject threat
2.6 Repeated testing (also referred to as testing effects) Repeatedly measuring the participants may lead to bias. only 1 post test.
2.7 Instrument change (instrumentality) instrument used during the testing process can change the experiment. post test change agreed! what i found in pre/post test questions different from treatment, need to craft the post test question to correctly test the learning in the treatment.
2.8 Regression toward the mean extreme outliers on individual scores are more likely to be captured in one instance of testing but will likely evolve into a more normal distribution with repeated testing. data will speak to me but it is likely to happen for high scores. have a control group. Xpre = T(true score)+E(random error), statistical adjustment Xpre = Xpre +(1-rxx) (X mean-Xpre)
2.9 Mortality/differential attrition only those participants that have participated from the start to the end are measured. safely assumption minimum for 1 week to 3 weeks treatment
2.10 Selection-maturation interaction minimum for 1 week to 3 weeks treatment
2.11 Diffusion treatment effects spread from experimental/treatment groups to control groups. tell subjects everyone will experience the same treatment, just the order different.
2.12 Compensatory rivalry/resentful demoralization control group members may work extra hard to see that expected superiority of the experimental group. agreed! we didn't check/noted
2.13 Experimenter bias teachers who are conducting an experiment inadvertently affect the outcome by non-consciously behaving differently to members of control and experimental groups.since teacher is key to learning, it is definitely going to affect the learning gains if the experimenter is unfamiliar with the learning practices in treatment. agreed! noted.
2.14 Instrument/research re-activity (Neumann, 2006) act of measuring produces change is DV. no observer in the class, so the chance of the subject knowing they are being treated is low. Teacher conduct lessons like normal.
http://en.wikipedia.org/wiki/External_validity
"A threat to external validity is an explanation of how you might be wrong in making a generalization."[5]Generally, generalizability is limited when the cause (i.e. the independent variable) depends on other factors; therefore, all threats to external validity interact with the independent variable.
- Aptitude–treatment Interaction: agreed! he sample may have certain features that may interact with the independent variable, limiting generalization. For example, students' aptitude and belief may not be in sync with the treatment. I notice some students may not take treatment well because for examination, they need only to know the 'right'answer, but not the process of learning like a scientist using computer.
- Situation: agreed! All situation specifics (e.g. treatment conditions, time, location, lighting, noise, treatment administration, investigator, timing, scope and extent of measurement, etc. etc.) of a study potentially limit generalizability. every research involving humans being is contextual specific and situational with people with agency to act and behave in a way difference from intended treatment outcome/result.
- Pre-test effects: agreed! if cause-effect relationships can only be found when pre-tests are carried out, then this also limits the generality of the findings.
- Post-test effects: If cause-effect relationships can only be found when post-tests are carried out, then this also limits the generality of the findings.
- Reactivity (placebo, novelty, and Hawthorne effects): agreed! if cause-effect relationships are found they might not be generalizable to other settings or situations if the effects found only occurred as an effect of studying the situation.
- Rosenthal effects: Inferences about cause-consequence relationships may not be generalizable to other investigators or researchers. Which is why teacher belief about the effectiveness of treatment is key to social research in education. U believe, therefore, it is happening.
in general, we try to talk to the students/teachers to gather insights to whether the threats are in effect.
like in my open research, you should try out the lesson yourself instead of accepting my research findings are valid.
enjoy!
 |
| MIN = xsource MAX =Math.sqrt((xpoint-xsource)*(xpoint-xsource)+(ypoint-ysource)*(ypoint-ysource))+xsource X(x,t)= "xsource+sign*((x-xsource)*cs-0.1*A1*Math.cos(omega*t-k*(x-xsource))*sc)" Y(x,t) ="ysource+sign*((x-xsource)*sc+0.1*A1*Math.cos(omega*t-k*(x-xsource))*cs)" where angle = Math.atan((ypoint-ysource)/(xpoint-xsource)); cs=Math.cos(angle); / /lookang & FKH sc=Math.sin(angle);//lookang & FKH cs2=Math.cos(angle2); // use to rotate to universal axes sc2=Math.sin(angle2); // use to rotate to universal axes using if ((xsource the using new feature with new wave form to allow counting of number of wavelength with RVHS tat leong in ripple tank sim plus some bug fixes. made dt smaller 0.01 for smooth graph http://weelookang.blogspot.sg/2013/03/ripple-tank-model-wee-duffy.html Ripple Tank Model (Wee, Duffy, Aguirregabiria, Hwang & Lee, 2012) with simplified physics equations modeled, realistic 2D and 3D (shown) visualizations, hints and scientific measurement tools for inquiry activities and data gathering for inquiry learning https://dl.dropboxusercontent.com/u/44365627/lookangEJSworkspace/export/ejs_Ripple_Tank_Interferencewee12.jar older version working https://dl.dropbox.com/u/44365627/lookangEJSworkspace/export/ejs_Ripple_Tank_Interferencewee07try.jar worksheets by (lead) IJC: https://www.dropbox.com/s/ssfismpu1683l3k/RippleTankIJC.zip IJC: https://www.dropbox.com/s/dyvzrhuhzecxx7c/RippleTankIJC2013.docx RVHS: https://www.dropbox.com/s/pnbi0k6ww1zcmv8/RippleTankRVHS.zip YJC: https://www.dropbox.com/s/khlnwerjoienknh/RippleTankYJC.zip |
No comments:
Post a Comment