Psychological - Lecture 3 - Test Development 1: part2 - Construction (DN)

• Measurement: the assigning of numbers to some attribute

• Scaling: the method (mechanism) by which we assign numbers (rule setting)

• Nominal rule
  classifies only

• Ordinal rule
  slightly more sophisticated

  • it is ordered

  • train example - order says nothing about distances between stations

  • has both nominal & ordinal properties

• Interval rule
  has nominal & ordinal properties

  • equal distance between

  • no absolute zero

  • zero is just arbitrary

  • e.g., celsius (zero = freezing, but not the absence of tempretaure)

• Ratio rule
  true zero point

  • not a lot in psych testing

57:00

Ratio Rule

Nominal rule

Interval rule

Ordinal rule

• At least Interval

• 1:09*

• ordinal-level data

• because they do not actually have equal levels between responses
  however they are treated as interval (summative) scale

  • because they behave interval like (although theoretically there are not equal distances between strongly disagree & disagree)

• nominal

• not as much info as a likert, but its simpler

1:13:25

• good for children

• they dont understand things in the same way adults do

1:14:20

• method of paired comparisons (1:17:19)
  

  must choose one on basis of some rule

• comparative scaling (1:19:19)
  

  • sorting/ranking along a dimension

  • what is worse cheating on taxes or cheating on test

• categorical scaling (can be nominal data)
  

  • sorting stimuli according to rule

  • good or bad / best or worst

  • essential - not essential (our questionnaire)

  • depends on how question is framed as to whether you get ordinal or nominal data (with categorical)

• Constructed-response format
  

  • Completion item

  • Short-answer

  • Essay

• Selected-response format
  

  • Matching items

  • True-False

  • Multiple choice

1. Completion item (1:25)

• provides word to complete sentence

• must be a specific correct answer

• poor design can lead to scoring problems

2. Short-answer (1:31:15)
   * word, sentence, paragraph in answer to a highly specific question

3. Essay (132:10)

1. Matching
   2. True-False (1:34:20)

• must contain a single, unambigous idea & be short

• use kuder-richardson for binary

3. Multiple choice (1:35:35)

• a stem

• a correct item

• multiple incorrect alternative options (distractors, foils)

• stem - expectancy tables are used to evaluate this
  

  • could use kuser-richardson & look at it internally

  • or look at all 4 and use chronbach alpha

• Classification

• If-then conditions

• Multiple conditions

• Multiple true-false

• Oddity

Previous research (Talbot et. al.)

• used Likert (ordinal)

• made it summative (assumed interval) to get a composite score

We will use

• Likert (assume interval)
  how felt if not enough sleep

• Categorical
  emotion is essential, useful, unnecessary

  • to behaviour of whether you get enough sleep

• one of four complex multiple-choice format

• The test-taker must decide the consequence of one or more conditions being present

“If the true variance of a test increases but the error variance remains constant, which of the following will occur?”

a. Reliability will increase

b. Reliability will decrease
c. Observed variance will decrease
d. Neither reliability nor observed variance will change

if variance has an increased proportion of true variance - this will always increase reliability

• it is one of the four complex multiple-choice formats

• The test-taker uses two or more conditions in the statements listed to draw a conclusion

“Given that Mary’s raw score on a test is 60, the test mean is 59, and the standard deviation is 2, what is Mary’s z score?”

a. -2.00
b. - .50

c. .50

d. 2.00

z score = difference between raw score & mean / standard deviation

1 / 2 = .50

• one of four complex multiple-choice formats

The test-taker decides whether one, all, or none of the two or more conditions or statements listed in the stem are correct

“Is it true that (1) Alfred Binet was the father of intelligence testing and the (2) his first intelligence test was published in 1916?”

a. Both 1 and 2
b. 1 but not 2
c. 2 but not 1
d. Neither 1 nor 2

• one of four complex multiple-choice formats

• The test-taker indicates which option does not belong with the others

  • which is the odd one out?

“Which of the following names does not belong with the others?”

a. Alfred Adler

b. Sigmund Freud
c. Carl Jung
d. Carl Rogers

Freud, Jung & Rogers = listeners

Adler = confronter

• create an item bank or pool

• start off with at least twice as many as you want to end up with

• one of four complex multiple-choice formats

• The test-taker classifies a person, object or condition into one of several categories designated in the stem

Jean Piaget is best described as a _____________ psychologist

a. Clinical

b. Developmental

c. Psychometric
d. Social