PSY 3094 Psychological Measurement – Scale Construction Project
The aim of this project is to develop a scale measuring a construct related to psychology and to report its psychometric properties. You will work in groups and prepare a report including the following componets given in the table below. The Assessment Rubric summarizes the details that should be included in each part of the report and how they will be graded. Please submit a copy of this rubric with your paper for grading.
Title Page
including the title of the project, the Group No/Name, the Name, Surname and Student No of the group members
Main Text
organized in subtitles of Introduction, Preparation of the Scale, Data Collection, Data Analysis, Evaluation of the Scale
References
in APA style
Appendices:
You should refer to the relevant appendices in text
Adapted from Improving Student Learning through Classroom Action Research: A Guide to Becoming an Action Research; Florida Department of Education https://tr.scribd.com/doc/132663674/action-research-assignment-and-rubric
SCALE CONSTRUCTION PROJECT ASSESSMENT RUBRIC
Group No / Name: ____________________________________________________ Title of Project: _________________________________________________________
Section
REQUIREMENTS & POINTS
POINTS RECEIVED
Introduction
5 points
3 points
0 points
…
A detailed description/definition of the construct that you are measuring
An adequate description/definition of the construct that you are measuring, but some important details are missing.
No definition of the construct that you are measuring
5 points
3 points
0 points
…
A detailed – clear description of the importance of measuring the construct
The importance of measuring the construct is stated, but why it is important is not clear.
The importance of measuring the construct and its reasons are not stated.
10 points
5 points
0 points
…
A detailed-clear description/definition of the variable that will be measured for criterion-related/construct validity and why it is selected/how it is related to the construct that you are measuring.
The variable that will be measured for the criterion-related/construct validity is described/defined, but why it is selected/how it is related to the construct that you are measuring is not clear.
No description/definition of the variable that will be measured for the criterion-related/construct validity and no statement about why it is selected
Preperation of the Scale
First version of the scale
- – Detailed description of how the first items are created (3 points)
- – presentation of the items (in Appendix) (5 points)
- – information about the scaling method (2 points)
…
Analysis of Content validity
- – Characteristics of experts (number, age, gender, their expertise level)(2 points)
- – How experts rated the items (2 points)
- – The results of experts’ ratings (3 points)
- – Content validity coefficient (3 points)
…
Finalization of the Scale
- – the solution for the items on which the experts do not agree (2 points)
- – the final list of the items (in Appendix) (2 points)
- – how the scoring of the scale should be done. ( 1point)
…
Data Collection
Sample
- – the number of participants (1 points)
- – how the participants are recruited ( 2 points)
- – demographic characteristics of participants (required : Age (M, SD, Min, Max), gender distribution; optional:
education level, SES, other characteristics that are important for the construct you are measuring etc.) (2 points)
…
Measures/Instruments
– demographic information form & presentation in Appendix (3 points)
Information about how the variable that is included for the criterion-related/construct validity is measured. If via a questionnaire, then
- – when and by whom the questionnaire is developed (1 point)
- – the number of items and the scale used for the rating (2 points)
- – how the score is computed (1 point)
- – psychometric properties of the original questionnaire and also in your data (2 points)
- – presentation of the questionnaire in Appendix (1 point)
…
Procedure
- – consent form (1 point)
- – presentation of consent form in Appendix (1 point)
- – order of the measures (1 point)
- – paper-pencil vs. online (1 point)
- – duration (1 point)
…
Data Analysis
Descriptive statistics of the scale that you are constructing and other scales used for validity analysis – M, SD, Min, Max (3 points)
Information about the analysis of inter-item consistency
- – how it is calculated (2 points)
- – report of the coefficient (3 points)
- – Interpretation of the score for inter-item consistency (high-low etc, whether it will change if some items are
deleted) (3 points) - – Report of SEM (3 points)
Information about the analysis of the criterion-related/construct validity - – How it is calculated (3 points)
- – Report of the validity coefficient (3 points)
- – Interpretation of the score (5 points)
Optional: Information about the analysis of split-half reliability (5 additional points)
…
Evaluation of the Scale
Evaluation of the scale that you are developing in terms of its reliability and validity (5 points) Limitations of the study (sample, problem of restricted range, procedural problems etc.) (3 points) A general conclusion ( 2 points)
…
References
5 points
2 points
0 points
…
All citations appearing the text match the reference list. The reference list is consistent in APA style and error free.
References partially match text citations. Reference list is inconsistent with regard to APA style.
References missing or do not match citations. Reference list is not consistent with APA style.