[Note - This several-page length presentation, as part of the overall PSSPQ Web Site, whose
          address is:  http://www.home.earthlink.net/~lastone2/psspq.html, is intended to provide an 
          important educational understanding to anyone who will be, is currently, or who has already gone
          through the investigation and adjudication process involved in being evaluated for high-level
          security clearance status.  It is important for such persons to be aware that the adjudication 
          process, is carried out by federal security employees (and/or sometimes assisted by contractor
          employees), is not fallible or always results in the 'best,' 'right,' or correct decision.  The below
          presentation presents an objective and scientific basis, founded on well-accepted and sound 
          psychometric theory (upon which most well-regarded psychological tests are founded) of the final 
          decision making associated with the official decision, made by the government, whether persons 
          will or will not be granted their hoped for security clearances.

         Unfortunately, the following presentation, most likely, will ONLY be appreciated or fully 
         understood by persons who possess an introductory level comprehension of psychometric
         statistical/mathematical knowledge.  With this said, then it is hoped that, at least, Ph.D.
         trained psychologists will, hopefully, full appreciate what is argued in the below paragraphs
         For those readers who do not have this kind of knowledge background, then if the direction
         of what is argued below truly interests you, it is suggested that you attempt to find a
         sufficiently trained psychologist or someone who also possesses similar type knowledge
         regarding  the statistical/mathematical basis of measurement to assist you towards better
         comprehension of what is herein presented.].

     Readers of this PSSPQ-associated presentation, which is mainly focused upon reliability/validity concerns, who truly have a psychometric background and who are not mere 'youngsters' in the field, are likely to have some familiarity in the theory of measurement error (as found in Gulliksen, 1950 or Nunnally, 1967) and will quite easily understand what is argued in the following paragraph. If one is willing to assume in that with any two sets of measures, errors from each set are uncorrelated between sets and that error on either set is uncorrelated with ‘true’ scores, then the measurement model for the correction for attenuation can be developed. Students of theories of measurement error are quite familiar with the fact that from the classic formula for computing the correction for attenuation another formula can be derived provided that one makes the assumption that the correlation between ‘true’ scores (i.e., without measurement error being involved) from both the variables involved be equal to unity. This particular derived formula is:

                                   r12 = ( Ö r11) ( Ö r22)

In other words, the upper limits size of the correlation between variables 1 and 2 (e.g., for example, the PSSPQ and the decision to grant or not grant TS-SCI security clearance status) is equal to the product of the square root of the reliability of variable 1 and the square root of the reliability of variable 2.  In this fashion, then it can be seen that a correlation coefficient between any two variables is a function of the reliabilities of the involved variables.  One of the major uses of the above shown formula is that its elements can be rearranged so as to allow for the estimation of a reliability coefficient for one set of measurements if the correlation coefficient between the two measurement sets is known or established in some fashion and if the reliability coefficient for the other set of measurements has been established. Such is the situation with our PSSPQ data at the present time. A correlation coefficient of 0.79 (i.e., the multiple correlation coefficient) has been clearly described as having been found to describe the relationship between a weighted sum of information associated with the so-called ‘best’ PSSPQ items with the final adjudication decision (made by the U. S. Government) to grant or not grant high-level security clearances. Also developed have been two different estimates of the reliability of the PSSPQ total scores measure; i.e., 0.94 based on the test-retest paradigm and 0.97 based on the Kuder-Richardson internal consistency model. If one accepts that the value of 0.95 (the average of the two reliability estimates) can represent a reliability coefficient for the PSSPQ, then the here-to-fore unknown and never-before estimated reliability for Government made adjudicational decisions, regarding the granting or non-granting of TS-SCI access, can now be known (or at least, argued)

     When 0.95 (the PSSPQ’s averaged reliability estimation) and the 0.79 (the Multiple R involving 25 of the PSSPQ’s items with the final adjudication decision) are entered into the correction for attenuation formula given above in one of the previous paragraphs, then the up-to-now, unknown reliability for the adjudication decision variable (favorable or unfavorable) can be determined and specified. This particular determined reliability value, in the present situation can be easily shown to be equal to 0.656, which is not an overly impressive reliability coefficient. However, for a measurement that is only dichotomous in kind, such a sized reliability estimation value seems to be not at all surprising to those who have studied judgmental reliability for diagnostic and prognostic evaluations regarding human beings. It can be noted that that this estimated reliability coefficient, having a numerical value of 0.656, should only be thought of only as an ‘upper-limit’ value, as this estimation value was obtained by having to assume that the correlation of ‘true’ PSSPQ scores with the ‘true’ adjudication decisions was perfect and equal to unity. Psychometrically speaking, it is almost certain that the actual reliability for the Governmentally made adjudication decision is, in reality, much lower than the 0.656 value presented and discussed here.

     The major conclusion that can be made, based upon information and logic described in the couple preceding paragraphs, is that if one wishes to increase the correlational prediction accuracy of the PSSPQ with respect to its ability to predict subsequent adjudication decisions, then it would be far more productive to attempt to increase the reliability for the making or formulating the adjudication decision-making rather than attempting to change, modify, or add-to the PSSPQ itself. All of the psychometric evidence points to a rather clear conclusion that the PSSPQ possesses excellent psychological test reliability. If there is a reliability problem in what is being measured, such is much more a problem with the governmentally-made adjudication decision variable.

     This is highly consistent with what has been observed frequently in the past in many of the very numerous applied areas of psychology. When using well-constructed psychometric tests in applied situations, it is also the rule and not the exception, that the tests possess far superior reliability than do the involved validity criterion measures themselves. A good example, are the correlational relationships that are found between extremely well-developed and constructed mental ability tests with school grades. Academic grading has been notorious for being known to possess very poor measurement reliability. Another example that has been well discussed, in the industrial psychology literature, relates to poor reliability for the job interview type situation (e.g., Guilford, 1959; Ulrich & Trumbo, 1965) The advice in this type of situation is not that we need to improved psychological tests in order to more accurately predict final adjudication decisions, but rather what is needed are better adjudication decision-making procedures that can allow for improved reliability of such decision making. Readers of this paper are urged to NOT tell or suggest to Dr. Stone (the developer and current purveyor of the PSSPQ) that more research should be done so that his PSSPQ testing instrument should be improved, added-to, or in some way changed. If one wants to improve the accuracy of using PSSPQ scorings to predict success/failure of individuals to be granted or not granted TS-SCI clearance status, the way to increase the prediction accuracy is very simple – IMPROVE THE RELIABILITY OF THE GOVERNMENTAL DECISION MAKING PROCEDURE FOR ADJUDICATING DECISIONS TO GRANT OR NOT GRANT HIGH-LEVEL SECURITY CLEARANCE STATUS!!!

     In case you have read the above presentation, but you feel that you really did not understand what was stated, then this more simple explanation is intended to perhaps remedy this situation.
Within an hour or so, you repeatedly get on a weight scale and it repeatedly gives you a markedly different weight readings, you would not hesitate to think that the weight information given this scale was worthless as it must be broken in some fashion.  You would correctly believe that any weight information from the scale was incorrect as all of the weight measures were very different when you knew that your true weight really had not changed, but more than a fraction of an ounce.  In other words, you would doubt the validity of the measurement information from the scale.  You could easily come the the conclusion that if repeated measurement attempts result in different values when, in fact, the thing being measured does not change, then any belief that the resulting measurements are accurate and true, simply can not be supported.  Stability of measurement is what is known as measurement reliability.

     What all this means when security clearance adjudication decisions are being considered is that it would appear that this particular decision making process (which in some cases can take one or two years and involve the efforts of literally dozens of different people) results in decisions that are a bit 'wobbly' or unstable (i.e., not overly reliable).  A person, who is vetted for potential granting of a high-level security clearance, it would seem likely and not surprising, might end up with very different adjudication decisions (i.e., success vs. denial) depending upon a large number of variables.  These might be : the agency doing the processing, what processing persons involved, subject individual's gender, age, physical appearance, ethnic/racial membership, if a critical shortage of the subject's expertise exists, etc. etc. etc.  In other words, a subject individual granted the hoped for clearance status might surprisingly be denied for the same level clearance status at some other time or with another agency.  Why?  Simply because the argument presented herein (which is support by strong and well accepted psychometric understandings) suggests that the security clearance vetting process may not be overly reliable, and hence may suffer some very meaningful resultant degree  of non-validity. 

     Therefore, if one is informed that the government has denied him/her the granting of a hoped-for and needed security clearance, once can have some doubt as to whether the whole clearance evaluation process, including the final adjudication decision, was carried out so that only a completely correct and valid decision was accomplished.  This is to encourage those who end up believing some significant error, on the part of the government, was committed in their case, to contact appropriate legal counsel so as to possibly appeal the government's decision.  For those in the 'business,' it is well understood and believed that many mistakes (i.e., in both directions) are committed that sometimes seriously affect the final adjudication decision process.

     One way that one can attempt to ascertain whether such a mistake has been made is to be administered the PSSPQ (and most importantly, respond to the PSSPQ's items in an entirely
honest and candid manner) and then see if the PSSPQ-based prediction regarding success/failure for being granted a high-level security clearance corresponds with or disagrees with the government's made decision.  You can be assured that the prediction, based strictly on PSSPQ-based information, was arrived at in a purely objective fashion - this is something that may not apply to the governmentally made decision.
 
 

Cronbach, L. J. (1960). Essentials of psychological testing (2^nd ed.), New York: Harper &
    Brothers.
Guilford, J. P. (1959). Personality. New York: McGraw-Hill.
Gulliksen, H. (1950). Theory of mental tests. New York, Wiley.
Nunnally, J. C. (1967). Psychometric theory. New York, McGraw-Hill.
Strong, E. K., Jr. (1943). Vocational interests of men and women. Stanford: Stanford University
   Press.
Ulrich, L., & Trumbo, D. (1965). The relationship of validity coefficients to the practical
   effectiveness of tests in selection: Discussion and tables.  Journal of Applied Psychology, 23,
   565-578.
 

Readers of this presentation are strongly urged to clink on the following provided web address, which is the "index/home" page for the Personnel Security Standards Psychological Questionnaire (PSSPQ):
http://www.home.earthlink.net/~lastone2/psspq.html