THE ANALYSIS OF THE EFFECTIVENESS OF THE INDONESIAN NATIONAL POLICE’S INTELLIGENCE SPECIALIZED DEVELOPMENTAL EDUCATION USING KIRKPATRICK TRAINING EVALUATION MODEL

This study examines the effectiveness of the implementation of the Security Intelligence Specialized Developmental Education (Dikbangspes Intelkam) in the Indonesian National Police (INP)’s Security Intelligence Education Center in 2018 using Kirkpatrick training evaluation model. Conceptually, training center is important to improve the ability of human resources (HR) in the organization and in actual fact the INP’s security intelligence human resources constitute the critical and strategic assets as well as leading component in supporting the police organization especially in presenting information related to the investigation, security and promotion in order to support the main tasks and functions of the Police (Tupoksi) in the field of security and public order (Kamtibmas). This research was conducted with a mixed method, beginning with the collection and analysis of quantitative data then proceeded with the collection and analysis of qualitative data. The analysis used Kirkpatrick's training evaluation model, namely reaction, learning, behavior, and result. The quantitative analysis results show that the effectiveness of the INP’s Security Intelligence Specialized Developmental Education (Dikbangspes) is high; however, based on the qualitative analysis, the implementation of the Security Intelligence Specialized Developmental Education (Dikbangspes) in 2018 faces 3 (three) problems, namely: the attainment of educational goals has not yet optimal, the training management system having been developed in the yearly period has not yet had a strong reference, and the evaluation of educational outcomes has had no significant follow-up.


INTRODUCTION
The importance of conducting a study of an evaluation of the Indonesian National Police (INP)'s Security Intelligence Specialized Developmental Education (Dikbangpes Intelkam Polri) can be identified from the following points. First, the intelligence personnel as police human resources constitutes the critical and strategic asset as well as leading component in supporting the police organization, especially in presenting information related to the investigation, security and promotion in order to support the main tasks and functions of the Police in the field of security and public order (Kamtibmas). Second, Meyer and Smith (2000) in Elnaga (2013: 142) emphasize that Study on human resources education and development related to organizational culture exists in a training institution, as the research by Kissack & Callahan (2009: 365) noticed that there is a reciprocal relation between organizational culture and training effectiveness. Caro (2011: 367) also found that organizational climate and culture can influence the evaluation process.
Human resources education and development greatly affects the quality of employees' work (Alipour, 2009: 63), the results expected by the organization (Kunche, et. Al. 2011: 1), and their improved technical skills and knowledge to upgrade services (Maduka, 2014: 327). This is also in line with the research conducted by Borate, et.al. (2014: 2) revealing that good training and development can improve the professionalism of participants. Another study was conducted by Mellahi (2000: 338), which made curriculum education relevant to the needs of the labor market to escalate the expansion of job opportunities and job skills. Another act done by Prabawati, et. al. (2017: 1) develops human resources with competency-based training (CBT).
Training evaluation has become attention to a lot of researches in the development of human resources (Devi & Shaik, 2012). Devi & Shaik (2012: 726)  Among the six models above, the most widely accepted model for the effectiveness of education and training is the model developed by Kirkpatrick that classifies training evaluation into 4 (four) categories, namely: (1) participant 'reactions to the training program; (2) changes in attitude, knowledge or skill levels; (3) changes in work behavior; (4) changes in organization's desired outcomes (Leach, P. Mark & Liu, H. Annie, 2003;in Devi & Shaik, 2012: 727).
The four-level method of the Kirkpatrick model represents a sequence of stages of evaluating the training program (Meghe, et.al: 2013) in Ramadhon, (2016: 45). The purpose of the sequence is that each level should be carried out in stages. That is because each level in the four level model is important and one level impacts the next level (Abdulghani, et. Al: 2014) in Ramadhon, (2016: 45). The four levels are: Level 1 -Reaction, Level 2 -Learning, Level 3 -Behavior, and Level 4 -Result.
Evaluating reaction (Level 1) is similar to measuring the level of consumer satisfaction (Kirkpatrick & Kirkpatrick, 2006: 21). Measuring the level of participant satisfaction in training activities is important, because it involves their motivation in learning. This is consistent with what was stated by Holton (1996: 5) that learning motivation is directly related to learning. Evaluation at level 1 does not measure what participants have learned, but assesses the interest, motivation, and level of attention of the trainees (Smidt, et.al., 2009: 272).
Learning evaluation (level-2) relates to measuring the improvement of participant competencies, both in terms of knowledge, skills and attitude in accordance with the objectives of the training. Learning is defined as the extent to which participants change their attitude, increase knowledge, and / or improve skills as a result of attending the program (Kirkpatrick & Kirkpatrick, 2006: 22). The purpose of implementing learning evaluation at level-2 according to Kennedy, et. Al (2013: 11) is to measure how well the participants take up the knowledge or skills delivered in the teaching learning activities. Behavior (level-3) according to Kirkpatrick & Kirkpatrick (2006: 22-23) identifies the extent of participants' behavior changes as a result of their participation in the training program. Level-3 evaluation is carried out to indicate the extent to which the material in the training is applied to the work and workplaces of the participants (Steensma and Groeneveld, 2010: 13). To be able to apply these behavioral changes, according to Kirkpatrick & Kirkpatrick (2006: 23), there are 4 (four) conditions that are needed, namely: a person should have a desire to change; he or she should know what to do and how to do it; he or she should work in the right work environment; and he or she should get an award because of the behavior change he or she has made.
Evaluation of result according to Kirkpatrick & Kirkpatrick (2006: 25) can be defined as an end outcome that is generated as a result of participants attending a training program. Rafiq (2015: 160) stated that evaluation at level 4 aims at finding out whether the training program is useful in achieving organizational goals. The final results in the context of evaluation at level 4 include increased production outcomes, customer satisfaction, increased employee morale, and increased company profits (Arthur et al.2003a;Landy and Conte 2007) in Praslova, (2010: 7).

RESEARCH METHODOLOGY
This study used a mixed method by using quantitative and qualitative research methods that can provide an understanding of comprehensive research problems and questions related to the study. When one form of research (qualitative or quantitative) is not enough to examine the research problems that are formulated or not complete enough to answer the research questions, then data, elaboration and further explanation are needed (Creswell, 2012: 22). The design of this study used "The Explanatory Sequential Design". Researcher collected quantitative and qualitative data in two ways, the quantitative data collection was firstly done and then followed by obtaining qualitative data.
The National Police Intelligence Specialized Developmental Education Program examined in this study was the implementation of Batch VII Specialized Developmental Education Year 2018 at the National Police Intelligence Education and Training Center, Jalan Terusan Soreang -Cipatik No.79, Pamekaran, Soreang, Gajahmekar, Kutawaringin, Bandung, West Java 40912 and in the West Java Regional Police.
The research subjects consist of respondents and informants. Respondents are the population (quantitative approach), who are the participants of the National Police Intelligence Specialized Developmental Education Year 2018 with proportional samples, namely 26 Intelligence Chiefs (KBO), 25 middle-rank Intelligence personnel, 30 personnel of Inteligence Collection Operations Unit, divided by 5 (five) commissioned officers and 25 non-commissioned officers, and 26 Security Investigation non-commissioned officers in the The National Police Intelligence Specialized Developmental Education Batch VII Year 2018. The informants of this study include: (1) Students of the the Specialized Developmental Education Batch VII, Year 2018; (2) the number of West Java Police Intelligence Personnel who have participated in the Intelligence Specialized Developmental Education was 214 personnel; (3) Head of the National Police Intelligence Education Center, leaders and educators, instructors and facilitators in the Police Intelligence Education Center. (4) Head of West Java Regional Police Human Resources Bureau, West Java Regional Police Intelligence Director, Head of Personnel Control Section (Dalpers) Human Resources Bureau West Java Regional Police Headquarters, West Java Regional Police Chief and Chief of Intelligence of Resort Police/City Police /Metropolitan City Police under West Java Regional Police, as Security Intelligence Personnel superiors in developing and carrying out personnel management functions.
(5) Key Officials of West Java Regional Police Directorate of Operational Functions; (6) Head of Functional Unit outside the intelligence function in the West Regional Police; (10) Head of Planning and Administration Bureau (Karo Renmin); and (11) the people of West Java, as the community who experience the impact of the main tasks, functions and roles (Tupoksiran) of Intelligence in the West Java Regional Police. The data collection technique in quantitative research approach is carried out through questionnaires. Meanwhile, the qualitative data collection technique include observation, interviews, and study of documents that are relevant and in accordance with the research objectives.

THE EFFECTIVENESS OF SPECIALIZED DEVELOPMENTAL EDUCATION
The calculation results of the dimensions of the Reaction variable is a parameter of the level of participant satisfaction towards the implementation of the National Police Intelligence Specialized Developmental Education Program Batch VII Year 2018. Reaction variable dimension includes the needs, materials, and methods of Specialized Developmental Education, trainers, media, facilities and environment of Specialized Developmental Education, time, and teaching and learning process.  Table 1. above shows that the Reaction variable has a minimum value of 77 and a maximum value of 200 with an average value of 155.30 with different variations in the perception of the standard deviation, which is 22.91. Reaction variable score tends to be high. This indicates that the Intelligence Specialized Developmental Education Year 2018 is perceived high by the respondents, which means that it has been held effectively.
Level 2 evaluation regarding the improvement of participant competencies in terms of knowledge, skills, and attitude can be seen in Table 2 below.  Table 3 below.  Table 4 below presents descriptive statistic of the Result variable obtained from the superiors' assessment of intelligence personnel as the impact of the results of the National Police Intelligence Specialized Developmental Education Batch VII Year 2018, with the dimensions of the Result variable, which include the outcomes of intelligence products, user satisfaction, and discipline. The final results in the context of evaluation level 4 include the improved work outcomes, user satisfaction, employee's morale improvement, and increased company profit (Arthur in Praslova, 2010). Result variable has a minimum value of 16 and a maximum value of 48 with an average value of 37.79 with different perceptual variations of standard deviation of 5.38. The score of the Result variable score tends to be high. The results of simple regression test on various aspects of the education and training program can be seen in Table 5 below.   The table above shows that the value of Fcount which is processed using SPSS is 50,930 with a significance value of 0,000. Thus, it can be said that the significance value <0.05 means that the regression model is appropriate.

The Dominant Factors Affecting the Effectiveness of the Management of Specialized Developmental Education (Dikbangspes) in the National Police Intelligence Education Center
The Dominant Factors Affecting the Effectiveness of the Management of Specialized Developmental Education in the National Police Intelligence Education Center take the view of Kirkpatrick (2006: 1) about the 10 (ten) steps of Training management which are carried out with careful consideration to each factor when planning and implementing an effective Training program. Such factors include: (1) Determining needs, (2) Setting objectives, (3) Determining subject content, (4) Determining participants, (5) Determining the best schedule, (6) Selecting appropriate facilities, (7) Selecting the appropriate instructors, (8) Selecting and preparing audiovisual aids, (9) Coordinating the program, and (10)  Developmental Education follows the curriculum of teaching learning materials set by the National Police Education and Training Center. The subject content was still concrete towards the tangible abilities, while towards characters / morale, it did not exist. 4. The selection of participants used a bottom-up approach and in its implementation violated many existing regulations as many of the participant's requirements were not met when the Regional Police chose its personnel to take part in the the the National Police Security Intelligence Specialized Developmental Education.

The Education Schedule or Calendar is prepared by the National Police Education and
Training Center and specified with the decision of the Indonesian National Police Chief. Based on the structure and all the matters managed for the implementation of the National Police Intelligence Specialized Developmental Education in the National Police Intelligence Education Center, the organization of the program was designed through the education and training department, teaching staff/instructors, and student coaching. 6. In general, the facilities of Specialized Developmental Education are already fulfilled by the National Police Security Intelligence Education Center, but what remains an obstacle is the availability of books, modules, and water. 7. Instructors consist of instructors and educational staff. However, not all of them have certificates as instructors and educational staff. Even though in terms of quantity, the instructors and educational staff are sufficient in number but in terms of quality, they are not enough. 8. Not all of the educational facilities and infrastructure have been fulfilled, especially water, books and modules. These three are the main problems in the facilities and infrastructure of the National Police Intelligence Education Center. 9. Coordination in the management process still overlaps with the development education implementation and experiences ineffective bureaucracy. The supervision and control system in the implementation of the National Police Intelligence Specialized Developmental Education in the National Police Intelligence Education Center has been running as routines. 10. Quantitative evaluation results from the comparison of pre and post-tests shows high scores for each Specialized Developmental Education. However, the results of qualitative data indicate that the quality of training results are much not in accordance with the objectives of the National Police Intelligence Education Center which has a morale of devotion, insightful, perceptive, and skillful.

Model (Hypothetical) Evaluation of The National Police Intelligence Specialized Developmental Education Using the Integration of Kirkpatrick and Countenance Stake
Improving the quality of the implementation of the National Police Intelligence Specialized Developmental Education is important in order to realize the development and prevention of regional and national security. However, the results of the study showed that there were obstacles in various management components of the training. In order to overcome various problems in the future Specialized Developmental Education, researcher tried to make an evaluation model of the National Police Intelligence Specialized Developmental Education so that the quality improvement can be carried out thoroughly, precisely, and in accordance with needs based on the results of an evaluation of the implementation of the National Police Intelligence Specialized Developmental Education. The evaluation model is as follows: Figure 1 below illustrates that the management of education development specialization which is carried out through a 10-step process of management training in the Kirkpatrick model, namely 1) determining needs, 2) setting goals, 3) determining subject content, 4) selecting participants, 5) determining the best schedule, 6 ) choosing the appropriate facilities, 7) choosing the appropriate instructor, 8) selecting and preparing audiovisual aids, 9) coordinating the program, and 10) evaluating the program. As for the program evaluation stage, this stage is carried out through 7 (seven) evaluation components, namely the context of Education and Training, Reaction, Learning, Process, Behavior, Result, and Impact. The seven evaluation components are an integrated model of Kirkpatrick and Coutenance Stake. The context evaluation analyzes 3 (three) things, namely the reasons why education and training is needed, the analysis of education and training needs, and participant recruitment. Reaction Evaluation analyzes participants' satisfaction with the implementation of the Education and Training they experience. Learning Evaluation analyzes changes in attitude, knowledge and skills of the participants. Process Evaluation analyzes the effectiveness of the environmental influence of the Specialized Developmental Education on the participants. Behavior Evaluation analyzes the level of changes in the behavior of Specialized Developmental Education alumni towards the presence of National Police Intelligence, changes in the behavior of Specialized Developmental Education alumni to the development of the National Police Intelligence, and changes in the behavior of Specialized Developmental Education alumni towards the National Police Intelligence services after returning from training. Result / Impact Evaluation analyzes the improvement in morale, discipline, organizational productivity, and user satisfaction, where the participants work after attending the education and training. In terms of time and place, the context evaluation is carried out, especially the training of policy making and the analysis of the needs of the National Police Intelligence Education Center, which is conducted before the Education and Training is held at the National Police Education and Training Center. The integration of the Kirkpatrick model and the Countinance Stake model is the model proposed by researcher because based on the experience of this study, where researcher used a four-level evaluation model from Kirkpatrik, much information could not be obtained by the evaluator, specifically related to the context (pre-training), the training process, and the impact of Education and Training. Integration is done by adding 1) evaluation of the context, namely how the evaluation of the recruitment process of the intelligence officers who join the Specialized Developmental Education should be done, and 2) how to evaluate the process of Intelligence Specialized Developmental Education. In addition, the component of the result in Kirkpatrik evaluation is considered the same as the stage of impact (outcome) of the countenance stake model. The purpose of developing the National Police Intelligence Specialized Developmental Education Evaluation Model using the integration of Kirkpatrick and Countenance Stake is to (1) obtain comprehensive, accurate, and up-to-date information on the implementation of the National Police Intelligence Specialized Development Education, (2) provide inputs to the Education and Training organizers and other related parties regarding the effectiveness of the National Police Intelligence Specialized Developmental Education and how to improve its quality systematically and sustainably, and (3) guide the the Indonesian National Police Training and Evaluation team to make operational and technical guidelines in accordance with needs.

CONCLUSION
This study concludes the following three things: 1) Evaluation of the four stages of Kirkpatrick model on the Indonesian National Police's Intelligence Specialized Developmental Education Batch VII Year 2018 through a quantitative approach shows a high level of effectiveness of the training; 2) The effectiveness of the National Police Intelligence Specialized Developmental Education Batch VII Year 2018 was analyzed by the process of the 10-stage training management model developed by Kirkpatrick, namely Determining needs, Setting goals, Determining subject content, Selecting participants, Determining the best schedule, Selecting the appropriate facilities, Selecting the instructors accordingly, Selecting and preparing audiovisual aids, Coordinating programs, and Evaluating programs. Through a qualitative approach, there are several problems faced, namely: a) Achievement of educational goals is not optimal, b) The training management system having been prepared in a yearly period does not yet have a strong reference, and c) Evaluation of training outcomes has not been followed up significantly, and 3) Based on the research findings, an understanding is obtained, Evaluation of the Kirkpatrick Model (2006) training program does not accommodate the evaluations of context and process. A new model is needed to evaluate an effective training program, especially for the Intelligence Specialized Developmental Education, which is comprehensive, appropriate, and in accordance with the needs. The intended model is the (Hypothetical) Evaluation of the Indonesian National Police Training Program by Integrating the Evaluation of the Kirkpatrick four-stage training program model (reaction, learning, behavior, and result) with the Countenance Stake Model (antecedents, transactions, and impacts). This model was developed based on the research on participants of the National Police Intelligence Specialized Development Education Batch VII Year 2018.
International Review of Humanities Studies www.irhs.ui.ac.id, e-ISSN: 2477-6866, p-ISSN: 2527-9416 Vol. 5, No.1, January 2020, pp. 77-92 SUGGESTION 1) An effective management method through the 10 (ten) stages of Kirkpatrick model needs to be formulated operationally as one that can analyze the interrelationship between the training components; 2) A hypothetical model of Education and Training evaluation developed by researcher requires the testing and creating of need of the research instruments, and 3) The need for the development of online / internet-based information and communication technology (ICT) is necessary to be developed both in training services and training evaluation.