Pros and cons of computer testing. Computer testing

Personnel assessment has been and remains one of the most important elements of the personnel management system: it is impossible to do without assessment either during personnel selection, certification, creation of a personnel reserve, or personnel rotation. Often the effectiveness of the entire HR system depends on the effectiveness of personnel assessment. The effectiveness of personnel assessment directly depends on the adequacy of the methods and approaches used. Is testing, so fashionable in the early 90s, always an adequate method?

At one time, when HR management in our country was taking its first steps, the majority of HR managers were recruited from psychologists, who directly brought with them to new area activities habitual scientific activity working methods - tests. This is quite understandable - in those days they didn’t know or know how to do anything else, information about Western technologies for working with personnel leaked out “a teaspoon at a time”, their own methods had not yet been developed.

To maintain their authority and not lose their jobs, some psychologists, when hiring, gave candidates 300–600 questions each to fill out a battery of clinical tests. Of course, such a selection made an indelible impression. Both for candidates and employers. And on the “HR managers” themselves. In addition, the output is “objective” data. Apparently this is where the myth about the omnipotence of tests originates.

Unfortunately, this is just a myth. The use of tests for scientific purposes has a number of limitations, while the use of testing in business is doubly limited.

Traditionally, the advantages of testing include the standardization of methods, the presence of a normative result, and its reproducibility. It is believed that the data obtained during testing are objective. Also, many managers are impressed by the scientific nature of the assessment procedure in the case of testing.

However, almost all of these advantages have a “other side of the coin.” Let's start with standardization. Not all methods used by HR managers are truly standardized (tested on a large, reference sample, which confirmed that for people with the same expressed trait being tested, the test results will be the same); amateur and popular science tests are very often used in HR work. Moreover, standardization in itself is not a guarantee of quality: as a rule, tests are standardized on students, and no one can guarantee that the norm of, say, anxiety among students, accountants and, for example, customs brokers will be the same.

The objectivity of data obtained through testing may also be questioned. Most of the tests used in personnel assessment are questionnaires; not all of them are equipped with a lie scale. The bulk of these questionnaires were designed for research purposes, testing took place voluntarily, or on the initiative of the subject, so the lie scale was not provided, or was poorly protected: the subjects had no need to lie. Therefore, for a person with higher education(which means it's enough high level intelligence) “bypassing” such a test is not a problem, especially if the success of passing the test determines whether he will be accepted for a promising job.

In addition, cumbersome questionnaires require a lot of time to complete, process and interpret. Naturally, a person who spends a lot of time and effort filling out tests begins to feel irritated towards the company and the people who subjected him to such a “test”. As a result, the company’s image deteriorates and employee loyalty decreases.

By and large, psychological testing in personnel work makes sense in two cases: when assessing the professional suitability of specialists in a number of fields that place special demands on cognitive functions (attention, memory, thinking, emotional sphere etc.) professional (accountant, dispatcher, pilot, etc.) and with a large flow (mass recruitment or certification of similar specialists), when speed of assessment and great importance gains the ability to compare results.

At the same time, many characteristics that are in great demand in the labor market (corporatism, loyalty, constructiveness, customer orientation, etc.) cannot be reliably identified using tests. And it is impossible to determine whether a candidate will fit into the organizational culture of the company using any methods other than observation and conversation. In addition, it is not always possible to establish a direct connection between the presence of certain psychological qualities in a candidate and his professional success, and the absence of a number of professionally important qualities can be compensated for by experience and individual style of activity. In general, a focus on identifying a predetermined set of characteristics limits the range of information that can be obtained during a survey.

In general, the use of questionnaire tests requires the HR manager to have less competence in the field of psychology than projective techniques, observation and interviews, since the results of testing as a method minimally depend on the skill of the researcher. However, the lack of proper competence can lead to the fact that what is measured is not what was planned due to an inadequate choice of method. Often the test that the researcher is good at or is used to using is used, rather than the one that fits the situation. Many have probably encountered the fact that the MMPI clinical test, created to identify severe mental pathologies from the field of major psychiatry, was used to select and evaluate managers, sales representatives, insurance agents, and bank employees. Even if we ignore ethical issues, the adequacy of using this method outside the clinic raises, to put it mildly, great doubts. And the use of the Rorschach test (an even more complex projective clinical test, which takes several years to master) in marketing focus groups (imagine, this happens) is simply shocking. As practice shows, much more adequate and informative results when assessing professionalism can be achieved with the help of specially designed, structured interviews, the case method and an assessment center.

In terms of the variety of information provided, testing as a method is significantly inferior to such methods as conversation and observation. For all its apparent simplicity, ingenuousness, bias and “unscientific” nature, a half-hour conversation can give an experienced psychologist or manager more information about a person than a half-hour test.

However, there are three main categories of tests that can be used successfully by HR departments. These are projective, professional and cognitive tests. Projective tests provide a lot of varied information about a person, do not require much time to complete, and are very difficult to “deceive”, since these methods rather appeal to the unconscious, having little contact with our conscious attitudes and beliefs. That is why projective techniques, among other things, are the best method for identifying serious mental pathologies of an organic nature that may not be revealed in observation and conversation. Cognitive tests allow you to evaluate features cognitive functions: distribution of attention, resistance to stress, reaction speed, etc. Professional tests, as a rule, are not strictly psychological. They allow you to assess the level of professional knowledge of a specialist.

In conclusion, I would like to remind you that testing data, as well as refusal to undergo testing, according to current legislation, cannot be the reason for denying a job to an applicant or employee.

Based on materials from “New Markets”

Computer testing is increasingly used in pedagogical practice. Perhaps soon it will almost displace traditional methods(such as "pencil - paper"), since it has clear advantages over them. What are they?

1) The computer version of testing saves a lot of time (this is probably the most important thing). The test taker's task is to simply press the key corresponding to the selected answer. The received data is automatically calculated, processed, evaluated and interpreted. As a result, the computer produces a finished report, often accompanied by charts, graphs and other visual images. The entire procedure, including processing and interpretation of results, takes significantly less time than with conventional testing. This time saving is especially valuable when working with a group of test takers - you can sit down at the computer at the same time big number people and quickly obtain the necessary data.

2) The tester's energy is saved - he does not have to do very tedious routine work (instructing the test taker, issuing tasks, keeping records, counting and processing results).

3) If you have a well-debugged program, computer testing practically eliminates errors when processing results - the machine always uses the same algorithm, it does not get distracted or get tired.

4) It becomes possible to accumulate and save an electronic database. A unified database is convenient for analysis and replaces huge piles of experimental forms, reports and conclusions.

5) When using a standardized computer program testing conditions do not depend on the individual characteristics and psychological state of the experimenter, which undoubtedly increases the “purity” of the diagnostic procedure.

6) During computer testing, the subject, left alone with the computer, can allow himself to be more frank and natural. He has no one to be ashamed of - the hardware cannot react either evaluatively or emotionally to answers that are not the most successful, from the standpoint of social desirability.

7) Thanks to computer testing, it is possible to increase information and prevent declassification of the test due to the high speed of information transfer and special protection of electronic files. The procedure for calculating the resulting scores is also simplified in cases where the test contains only multiple-choice tasks.

8) The advantages of computer testing are also manifested in current control, in self-control and self-preparation of students; Thanks to the computer, you can immediately issue a test score and take immediate measures to correct the assimilation of new material based on the analysis of protocols based on the results of corrective and diagnostic tests. The possibilities of pedagogical control during computer testing are significantly increased by expanding the range of measured skills in innovative types of test tasks that use the diverse capabilities of the computer when including audio and video files, interactivity, dynamic formulation of problems using multimedia tools, etc.

9) Thanks to computer testing, the information capabilities of the control process are increased, it becomes possible to collect additional data on the dynamics of test passing by individual students and to differentiate between missed and unachieved test tasks.

10) Finally, the most routine part of the work is eliminated - preparing forms, providing methodological material etc., since the entire technique is presented in the form of a computer program. It's convenient in every way.

In addition to its advantages, computer testing has a number of disadvantages:

1) Typical psychological and emotional reactions students for computer testing.

Communication between a person and a computer has its own specifics, and not everyone is equally calm about computer testing. For example, if the testing procedure is delayed or the content of the test is not of interest to the person, the positive attitude may be replaced by the opposite: the monotony and monotony of the work, the “stupidity” of questions and tasks will tire and irritate. Sometimes a negative attitude towards computer testing is caused by the lack of feedback. And when the person being tested does not receive feedback, the likelihood of erroneous answers increases (you can misunderstand the instructions, confuse the answer keys, etc.).

Special studies have been conducted to determine how people feel about computer testing. It turned out that some people experience the so-called psychological barrier effect, and some people experience the overconfidence effect. It happens that a person is not able to cope with a task at all because he is “afraid” of the computer. It is also possible to include psychological defense mechanisms associated with the reluctance of the test taker to open up, the desire to avoid excessive frankness, or the deliberate distortion of results.

Negative reactions usually cause various restrictions, which are sometimes imposed when issuing tasks in computer testing. For example, either the order in which tasks are presented is fixed, or the maximum possible time completion of each task, after which, regardless of the desire of the subject, the next test task appears. In adaptive testing, students are unhappy that they do not have the opportunity to skip the next task, review the entire test before starting work on it, and change answers to previous tasks. Sometimes students object to computer-based testing because of the difficulty that comes with doing and writing math calculations, etc.

2) The impact of prior level of computer experience on test performance.

The results of foreign studies have shown that the experience of working with computers that schoolchildren have, in many cases, significantly affects the validity of the test results. If the test includes tasks without innovation with multiple choice answers, then the influence of computer experience on test results is insignificant, since such tasks do not require students to do anything. complex actions when running the test. When innovative types of tasks are presented on the screen, making extensive use of computer graphics and other innovations, the influence of previous computer experience on the test score becomes very significant. Thus, when using computer testing, it is necessary to take into account the level of computer experience of the students for whom the test is intended.

To reduce the impact of computer experience on test scores, it is recommended to include special instructions And training exercises for each innovative form of tasks. It is also necessary to first familiarize students with the interface, conduct a rehearsal, and allocate students who do not have sufficient experience with a PC into independent groups in order to further train them or give them a blank test.

3) The influence of the user interface on the results of computer testing The user interface includes the functions available to the student and the ability to move through test tasks, elements of placing information on the screen, as well as the general visual style of presenting information. A good user interface should have a clear and correct logical sequence of interaction with the examinee, reflecting general principles graphic information design. The more elaborate the interface, the less attention the student pays to it, concentrating all his efforts on completing the test tasks.

4) During computer testing, specialists deal only with the results obtained. They do not see the person being tested, do not communicate with him, which means they do not have additional information about him, and cannot find out his actual amount of knowledge. So the results obtained using computer testing should be trusted with some reservations.

Computer testing is a recently emerged area of ​​psychodiagnostic research (examinations) associated with the use of electronic computer technology. The emergence of computer psychodiagnostics is due to the development information technologies. Attempts to automate the presentation of stimulus material to the subject and subsequent processing of the results have been made since the 30s. XX century But only since the 70s. The true development of computer psychodiagnostics begins in connection with the advent of personal computers. Since the 80s computer tests are being developed on a massive scale. First, as computer versions of already known blank methods, and in the 90s. as special techniques that take into account the possibilities modern technology and not used in blank form, since they are designed for complex stimulus material that varies in space and time, specific sound accompaniment, etc. Beginning of XXI century is marked by the fact that testing management is increasingly transferred to the computer. If in past years certain stages of research were automated, for example, presentation of material (it is very convenient to use a computer instead of a tachistoscope), data processing (especially cumbersome tests such as MMPI, 16PF, sociometry), interpretation of results (Lüscher test), then modern stage More and more often you can find programs that take care of the entire examination, right up to the diagnosis, which reduces the need for the presence of a psychologist to a minimum. And this has its pros and cons.

Unconditional advantages computer tests (CT) are: rapid; high speed and error-free processing; the ability to immediately obtain results; ensuring standard testing conditions for all subjects; clear control of the testing procedure (questions cannot be skipped; if necessary, the time of each answer can be recorded, which is especially important for intelligence tests); the possibility of excluding the psychologist as an additional variable (which has special meaning during the examination); clarity and entertainment of the process (support of attention with the help of color, sound, game elements, which is especially important for educational programs); easy archiving of results; the ability to combine tests into batteries (software packages) with a single final interpretation; mobility of the experimenter (all instruments on one floppy disk); the possibility of conducting mass research (for example, via the Internet).

Flaws computer tests: complexity, labor intensity and high cost of program development; the need for expensive computer equipment; the difficulty of using computers in the field; the need for special training for the subject to work with CT; difficulties in working with non-verbal material, the particular difficulty of translating projective tests into computer form; lack of an individual approach to the person being tested (loss of some of the psychodiagnostic information obtained in conversation and observation); latency of the data processing and interpretation stages (the quality of these procedures depends entirely on the program developers). Some subjects may experience the effects of a “psychological barrier” or “overconfidence” when interacting with a computer. Therefore, data on the validity, reliability and representativeness of blank tests cannot be automatically transferred to their computer counterparts, which leads to the need for new standardization of tests.

The disadvantages of CT cause psychologists to be wary of them. CT scans are rarely used in clinical psychology, where the cost of error is too high. L. S. Vygotsky identified three levels of psychodiagnostics: symptomatic (identification of symptoms), etiological (identification of causes) and typological (holistic, dynamic picture of personality, on the basis of which a prognosis is based). Computer psychodiagnostics today is at lowest level– the level of symptomatic diagnosis, providing practically no material for identifying the causes and making a prognosis.

But, apparently, CT scans have a great future, where many of the listed shortcomings of computer psychodiagnostics will probably be solved thanks to the further development of electronic technology and the improvement of psychodiagnostic technologies. The key to such optimism is the growing interest of science and practice in computer diagnostics, which already has over 1000 CT scans in its arsenal.

If we try to classify the currently existing CT scans, we can distinguish the following types:

1. By structure

a) analogues of blank tests;

b) CT scan itself.

2. By the number of people tested

a) CT individual testing;

b) CT group testing (for simultaneous submission of identical material on computers connected to a local network).

3. According to the degree of testing automation

a) automating one or more stages of the examination;

b) automating the entire examination.

4. By task

a) diagnostic CT scans;

b) educational CT (simulator tests, developmental programs).

5. By addressee

a) professional psychological;

b) semi-professional;

c) unprofessional (entertaining).

by the user professional CT is a psychologist, so they are developed by specialized laboratories or centers of computer psychodiagnostics (among which it is worth mentioning the Center “Humanitarian Technologies” in Moscow, JSC “Imaton-M”, the laboratory of clinical psychology of the V. M. Bekhterev Psychoneurological Institute in St. Petersburg and etc.). These tests have a number of specific features: 1) the presence of an archive (database); 2) the presence of a password to enter the test or database to ensure the confidentiality of the results; 3) detailed interpretation of the results using professional terms, coefficients, with the construction of graphs (profiles); 4) availability of information about the developers of the methodology, information about validity and reliability, reference materials about the theoretical principles underlying the methodology. Semi-professional CT is aimed at specialists in related professions, for example, teachers, personnel managers. Such tests are often provided with a reduced interpretation without the use of special vocabulary, and are easy to learn and use. Tests of this level can also be intended for a non-specialist, an ordinary personal computer user interested in psychology. Finally, there is also a large number unprofessional CT aimed at popularizing psychological ideas or pursuing entertainment purposes.

When using professional or semi-professional CT scans, the same ethical principles must be followed as for blank testing. It is important not to distribute test results and protect your files with a password, especially if there are multiple users on the computer. And the main thing is “not to create an idol for yourself.” Remember that CT is only a tool, only an assistant and has its own limits of application (the knowledge of which distinguishes a professional psychologist from a charlatan from psychology).

The test method is one of the main ones in modern psychodiagnostics. In terms of popularity in educational and professional psychodiagnostics, it has firmly held first place in world psychodiagnostic practice for almost a century.

In this section, tests should be understood as methods that consist of a series of tasks with a choice of ready-made answer options. When calculating test scores, the selected answers receive an unambiguous quantitative interpretation and are summed up. The total score is compared with quantitative test norms, and after this comparison, standard diagnostic conclusions are formulated.

The popularity of the test method is explained by the following main advantages:

1) standardization of conditions and results. Test methods are relatively independent of the qualifications of the user (performer), for whose role even a laboratory assistant with secondary education can be trained. This, however, does not mean that in order to prepare a comprehensive conclusion on a battery of tests it is not necessary to involve a qualified specialist with a full-fledged higher psychological education;

2) efficiency and efficiency. A typical test consists of a series of short tasks, each of which usually takes no more than half a minute to complete, and the entire test usually takes no more than an hour. A group of subjects is tested simultaneously, thus saving significant time on data collection;

3) quantitative differentiated nature of the assessment. The granularity of the scale and the standardization of the test allow us to consider it as a “measuring instrument” that gives a quantitative assessment of the measured properties. The quantitative nature of the test results makes it possible to apply a well-developed psychometric apparatus that allows one to assess how well a given test works on a given sample of subjects under given conditions;

4) optimal difficulty. A professionally done test consists of tasks of optimal difficulty. In this case, the average subject scores approximately 50% of the maximum possible number of points. This is achieved through preliminary tests - a psychometric experiment (or aerobatics). If during aerobatics it becomes known that approximately half of the examined contingent can cope with the task, then such a task is considered successful and is left in the test;

5) reliability. This is probably the most important advantage of tests in educational psychodiagnostics. The lottery nature of modern exams with lucky or unlucky tickets drawn has long become the talk of the town. The lottery for the examinee here results in low reliability for the examiner - the answer to one fragment of the curriculum, as a rule, is not indicative of the level of mastery of the entire material. In contrast, any well-constructed test covers the main sections of the curriculum (the area of ​​knowledge being tested or the manifestation of some skill or ability). As a result, the opportunity for “tail-leaders” to become excellent students, and for an excellent student to suddenly fail, is sharply reduced;

6) justice. It is the most important social consequence of the above advantages. It should be understood as protection from examiner bias. A good test puts everyone on an equal footing. The examiner’s subjectivity is most strongly manifested, as is known, not in the interpretation of the level of solution of the problem (it is not so easy to call black white, or a solved problem unsolved), but in the tendentious selection of tasks - easier for one’s own, harder for someone else’s. Tests provide the most important function of the school as a social filter - the function of “social and professional selection”. How fair such selection turns out to be is of enormous importance for the development of society. Therefore, it is so important for everyone who has access to tests and their results to learn the culture of competent and humane use of tests, because only a conscientious and qualified attitude of users towards tests turns them into a tool that increases, and not decreases, the level of justice in society;

7) the possibility of computerization. In this case, this is not just an additional convenience that reduces the human labor of qualified performers during a mass examination. As a result of computerization, all testing parameters are increased (for example, with adapted computer testing, testing time is sharply reduced). Computerization is a powerful tool for ensuring information security (diagnostic reliability). The computer organization of testing, which involves the creation of powerful information banks of test tasks, makes it possible to technically prevent abuse by unscrupulous examiners. The selection of tasks offered to a particular subject can be made from such a bank by the computer program itself directly during testing, and the presentation of a particular task to a given subject in this case is as much a surprise for the examiner as for the subject.

In many countries, the implementation of the test method (as well as resistance to this implementation) is closely related to socio-political circumstances. The introduction of well-equipped testing services in education is a vital tool in the fight against corruption that affects the ruling elite (nomenklatura) in many countries. In the West, testing services operate independently of issuing (schools) and receiving (universities) organizations and provide the applicant with an independent certificate of test results, with which he can go to any institution. This independence of the testing service from issuing and receiving organizations is an additional factor in the democratization of the process of selecting professional personnel in society, giving a talented and simply efficient person an extra chance to prove themselves.

Most students in the process of learning English will undergo knowledge tests: TOEFL, IELTS, school exams etc. Tests are a way to quickly and easily check a large amount of information, and, like any type of testing, they help psychologically prepare the student for upcoming exams. Also, the test is a statistical proof of the student’s progress, since here he gets a clear result, for example, 80% out of 100.

At the same time, it should be taken into account that tests are not suitable for testing all language skills, especially the main one - speaking. In addition, tests give an idea of ​​the student’s passive rather than active knowledge. Many, for example, have no problem choosing the correct verb forms in a test, but at the same time use them incorrectly in communication and writing.
In the article we will consider different kinds test items, their advantages and disadvantages, and what skills they are best suited to test.

Types of test items:

• Multiple choice
  
• Transformation
  
• Gap-filling
  
• Matching
  
• Close
  
• True/False
  
• Open questions
  
• Error correction
  

1.Multiple choice

Choose the correct word to complete the sentence.
I….. tennis every Sunday morning.

• playing
  
• play
  
• am playing
  
• am play
  

The essence of the task: The student must choose the correct answer among several options.
Suitable for testing: grammar, vocabulary, reading and listening.
Advantages:
– the fastest way to test a large amount of information
– Suitable for preparing students for standardized tests
Flaws:
– Creating high-quality multiple choice tests takes a lot of time.

2.Transformation

Complete the second sentence so that it has the same meaning as the first.
We paid some people to landscape the garden for us last year.
had
We _____________________________________ last year.

The essence of the task: The student must remake the sentence using a specific word.
Suitable for testing: grammatical forms

3. Matching

The essence of the task: The student matches the items in one column with the corresponding items in the other. These can be individual words, words and their definitions, parts of a sentence, pictures and words, etc.
Suitable for testing: vocabulary
Advantages:
– reduce the student’s “test stress”, since there is support for answers
Flaws:
– do not always provide a clear picture of the student’s knowledge. Even if the student does not know the correct answer, he has a chance to guess. This problem can be solved by including in the second column more words or pictures than in the first one.
– one mistake can “advance” the entire task and it will be completed incorrectly

4. Completion (fill in the blank tests, cloze tests)

Complete the text by adding a word to each gap.

Word Bank: feast, turkey, parade, football, cranberry, pumpkin pie, thanks

Thanksgiving is a special time of year when families come together and give ___ for all they have.
There is a popular ___ in New York, and you can also enjoy the ___ games on TV.
Thanksgiving dinner is a ___ that includes ___, potatoes, ____ sauce and much more.
There are also many desserts after Thanksgiving dinner, including the most popular ___.

The essence of the task: The student fills in the gaps in the sentences.
Suitable for testing: grammar and vocabulary

5.True/False

Decide if the statement is true or false.
The currency of the USA is the dollar. T/F

The essence of the task: every question is a statement. The student must determine whether it is correct or not.
Suitable for testing: reading and listening.
Advantages:
– very quickly and easily checked
Flaws:
– in this type of task, a student may choose the wrong answer not due to ignorance of the topic, but due to ordinary inattention to some details
- do not always indicate gaps in knowledge, since the student has a 50% chance of guessing correctly. This problem can be solved with the help of an additional task - the student must not only mark incorrect statements, but also correct them.

6. Open questions

Answer the questions.
Why did John go to the shop?

The essence of the task: the student gives written answers to questions
Suitable for testing: all linguistic aspects.
Advantages:
– tests how well a student can use knowledge, not just how much information he has memorized.
Flaws:
– takes more time to check
– the test is more subjective than with other types of test tasks. One teacher may agree with a student's choice of words, another may not. This problem can be solved by using a word bank - a set of vocabulary that students can use when answering. It is advisable to provide students with more options than required in the answer. For example, if there are 20 questions on a test, the word bank may contain 30 words.

Error Correction

Find the mistakes in the sentence and correct them.
I would rather to work from home than come to the office.

The essence of the task: The student must find and correct errors in the sentence. Errors may be unnecessary words, irregular shapes verbs, missing words, etc.
Suitable for testing: grammar, vocabulary, reading and listening

To summarize, it should be noted that the main advantages of tests are that they are a structured and objective way to quickly check a large amount of information.
The main disadvantage is that the tests are not suitable for testing the main aspect of study - communication. Therefore, to fully check the level of language proficiency, test tasks should be combined with such forms of testing as essays and oral interviews.