{"id":243719,"date":"2017-07-27T18:20:15","date_gmt":"2017-07-27T21:20:15","guid":{"rendered":"http:\/\/revistapesquisa.fapesp.br\/?p=243719\/"},"modified":"2017-07-27T18:20:15","modified_gmt":"2017-07-27T21:20:15","slug":"gears-in-motion","status":"publish","type":"post","link":"https:\/\/revistapesquisa.fapesp.br\/en\/gears-in-motion\/","title":{"rendered":"Gears in motion"},"content":{"rendered":"

\"\"Imagezoo\/GettyImages<\/span><\/a>The University of S\u00e3o Paulo (USP), Brazil\u2019s largest public institution of higher education\u2014and the most prominent in international rankings\u2014begins 2017 with a new model for measuring the performance and quality of the work of its teaching staff that numbers nearly 6,000.\u00a0 On November 8, 2016, the University Council approved a revamped assessment system designed to coordinate existing evaluation mechanisms and, more importantly, establish objectives to be achieved by professors, departments and units working together.<\/p>\n

The change was implemented through an overhaul of the Permanent Assessment Committee (CPA) and the Faculty Bylaws. The assessments will be carried out every five years, based on the guiding principles of the academic plans established for each faculty member.\u00a0 These plans will define the expected performance in teaching activities at the undergraduate and graduate levels, student orientation, research, extension and business administration courses, and must be approved by the department to which the faculty member is assigned. \u00a0Each department, in turn, will have a collective academic plan whose objectives apply to its entire teaching staff.\u00a0 And the Department projects will be coordinated with larger projects carried out at each school or institute.<\/p>\n

In putting together their academic plans, professors may, through negotiations with the department, define specific tasks, such as their overarching interest in conducting research or devoting themselves to teaching activities. \u00a0\u201cThis is a major change because it addresses the variety of backgrounds among professors that a university like USP needs to have,\u201d says President Marco Antonio Zago. \u201cIt corrects distortions such as requiring that faculty members who are researchers become involved in an inordinate number of administrative or teaching activities, or that call for a professor recognized for her excellence in teaching to produce a long list of scientific papers.\u201d<\/p>\n

If the faculty member fails to comply with what is specified in her individual plan, she can sign an agreement to commit to redirecting her work within a three-year period and re-establish the expected academic level.\u00a0 If she fails again, she would be subject to an administrative procedure that may result in a warning, reprimand and dismissal. Faculty members who sign such an agreement once will not be entitled to use this resource in the next two assessment cycles. \u201cThere need to be consequences for those cases in which academic plans were not fulfilled, but the focus of the new system is not to punish,\u201d says Alexandre Nolasco de Carvalho, a professor at the Institute of Mathematical Sciences and Computation (ICMC) in S\u00e3o Carlos, chairman of the committee tasked with proposing the model.\u00a0 \u201cThe idea is to have goals that foster the development of academic quality among professors, departments, schools and the university as a whole.\u201d<\/p>\n

One source of inspiration for the USP model is the triennial assessment process for graduate programs that has been carried out for more than 20 years by the Brazilian Federal Agency for the Support and Evaluation of Graduate Education (Capes), under the Ministry of Education (MEC). \u201cThe Capes model is recognized for its ability to promote continuous improvement in the quality of graduate programs. It is performed by peers and the requirements are increased in subsequent cycles,\u201d says Maria Paula Dallari Bucci, a professor in the USP Law School, member of the committee charged with proposing the system and former secretary of higher education at MEC from 2008-2010.<\/p>\n

\"\"Imagezoo\/GettyImages<\/span><\/a>The model is intended to refine the previous system, which consisted of three distinct assessment bodies.\u00a0 One of them was the university\u2019s institutional assessment, which the CPA performed every five years and submitted to the State Department of Education.\u00a0 That assessment is conducted through opinions written by advisors who evaluate quantitative indicators, self-assessment forms and interviews with staff, students and university employees. \u201cIt is the university\u2019s obligation as a public entity,\u201d Nolasco says. \u201cBut, on its own, this assessment has never had the power to guide institutional development at USP.\u201d\u00a0 Now, the purpose of the CPA has been expanded.\u00a0 It is presently composed of a general assembly and two specific chambers\u2014one focused on teaching activities and the other on institutional assessment (of departments and units)\u2014and its mission is to formulate the assessment process of USP entities and establish guidelines and the timetable for assessments.<\/p>\n

There was also an assessment committee focused on career development of faculty members as well as one for monitoring the employee probation period for staff and faculty who passed the civil service exam and were admitted to the university. The probation period used to be six years\u2013during which time, the faculty member was evaluated to be confirmed in the position and earn tenure.\u00a0 In the new configuration, the probation period has been reduced to three years.\u00a0 According to Alexandre Nolasco, the three stages were somewhat flawed in that they did not operate in a mutually related way. \u201cThere was no connection between the assessments and the university\u2019s institutional development,\u201d he says.\u00a0 \u201cAssessment cannot be an end in and of itself.\u00a0 It has to be part of a process to improve quality, in which objectives are planned and periodically measured so as to determine what has or has not been achieved, and then allow for course corrections.\u201d<\/p>\n

Discussions about the model have elicited various critiques from the academic community.\u00a0 Some have led to modifications in the original proposal, but others remain.\u00a0 What Maria Arminda do Nascimento Arruda, the director of the USP School of Philosophy, Literature and Human Sciences (FFLCH) finds concerning is centralization of the assessment process for the entire university in a single body: the CPA. \u00a0\u201cThe assessment process has the ability to synchronize the departmental plan with the faculty member\u2019s main focus: teaching, research and extension activities that will serve as their assessment parameters. But the university model we are accustomed to grants broad autonomy to the various schools and each of them has their own criteria for judgment and assessment.\u00a0 Centralization introduces the risk of expanding standardization and bureaucratization at the university and constricting a system that is already rigid,\u201d she says.\u00a0 According to the professor, although the project provides for the presence of representatives from the social sciences and the humanities on the assessment committee, there is real concern that criteria so unlike the tradition in these fields will be used to measure the quality and performance of faculty members.\u00a0 \u201cI\u2019m completely in favor of assessment because USP is a public university and has to be held accountable.\u00a0 But the question that really needs to be asked is: will excessive standardization compromise cultural, scientific and academic activities at the university?\u201d<\/p>\n

Discussions concerning the changes in the evaluation system began back in 2014, when President Marco Antonio Zago established a committee tasked with updating the system.\u00a0 In 2016, at least three versions of the regulations were produced through a process of public hearings, incorporating contributions made during unit meetings and in meetings held on the Ribeir\u00e3o Preto, S\u00e3o Carlos and S\u00e3o Paulo campuses. \u201cWe received more than 800 pages of suggestions, which were in large part incorporated into the final text approved by the University Council, says Bucci.<\/p>\n

Initially, linking assessments to the type of work arrangement was considered\u201386% USP faculty fall under the full-time teaching and research arrangement (RDIDP). But that idea was abandoned.\u00a0\u00a0 \u00a0\u201cThere was a completely mistaken interpretation in some quarters of the university that suggested that the President\u2019s office might be interested in reducing the number of RDIDP professors to save money,\u201d says President Marco Antonio Zago. \u201cSince this was not true and the RDIDP plays an essential role in the university, we decided to take type of work arrangement off the table, even though the system clearly needs to be updated.\u201d\u00a0 If the notion of tying evaluations to type of work arrangement had gone forward, it would not have been the first time. Back in the 1990s, the University of Campinas (Unicamp), for example, instituted a faculty performance assessment format that could have resulted in the elimination of the RDIDP, and a resulting reduction in salary for professors whose assessments indicated that their performance had not reached the level determined through established criteria.\u00a0 There are always vacancies for part-time faculty at Unicamp.\u00a0 Those who pass civil service exams, after submitting an activity plan, may be authorized to enter the full-time track, but that does not ensure them the right to definitely remain in that category. \u00a0Another sensitive topic was the makeup of the CPA, which initially would have had all its members indicated directly by the Office of the President. The approved text specifies that only three of the nine members will be appointed by the President.\u00a0 The rest will be named by units and faculty groups.<\/p>\n

\"\"Imagezoo\/GettyImages<\/span><\/a>According to Elizabeth Balbachevsky, a professor in the Department of Political Science at the School of Philosophy, Literature and Human Sciences at USP, the new system is an improvement on several fronts. \u201cFor the first time, it includes something that was always considered taboo at USP: accountability for faculty performance,\u201d says the professor, remembering that back in 1988, the leaking of a list of USP professors who had written no papers in previous years left a lasting impression.\u00a0 \u201cThis accountability is very clear and specifies that the faculty member will be assessed according to solid and predefined parameters,\u201d says Balbachevsky, who takes part in research studies about the academic profession in Brazil and in other countries.<\/p>\n

Renato Pedrosa, a professor in the Department of Political Science and Technology of the Geosciences Institute of Unicamp, is critical of one provision in the new USP Bylaws\u2014the one which states that at the end of the three-year probation period, the recently-hired faculty member will be evaluated by a committee made up of three people, at least one of whom is associated with the department to which the staff member is assigned. \u201cIf the committee has at least one member who is a colleague of the professor being evaluated, that means there is the possibility of forming a committee with two or three members from the same department.\u00a0 This seems unlikely to prevent the endogeny that is a feature of many Brazilian universities and the formation of interest groups that maintain the departmental status quo,\u201d he says.<\/p>\n

Pedrosa also believes that there need to be more criteria about what constitutes acceptable performance in assessing faculty and departments. Elizabeth Balbachevsky agrees that the new system is vague regarding these criteria, but says that this was done intentionally. \u201cClearly, it is a solution that contemplates the diversity of academic backgrounds that make up the units at USP, which attracts faculty from professional schools to international level professional enclaves,\u201d says the professor.\u00a0 President Marco Antonio Zago asserts that the objective of the reform was to establish the system\u2019s general framework, which can now be discussed and refined by the new assessment bodies. \u201cIt would make no sense to establish every last detail because it would become more complicated to correct what does not work and refine the model over time,\u201d he says.\u00a0 The presence of representatives from various fields on the CPA, says Zago, is important for reaching consensuses and lending validity to the process.\u00a0 \u201cThere is a tendency for researchers from some fields to fail to recognize the assessment criteria used by others, or even to reject such criteria. We have to get beyond that.\u201d<\/p>\n

Elizabeth Balbachevsky points out that the solution adopted by USP follows university governance models used in other countries, based on mechanisms for valuing and monitoring faculty performance. \u201cThe difference is that at British, U.S. and Australian universities, the assessment processes are authorized by experts from outside of the institutions, many of whom are from other countries.\u00a0 This is missing from the USP model and could serve as a counterweight to possible corporate dynamics that the new type of evaluations could engender.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"Assessment system at USP ties faculty goals to those of departments and units","protected":false},"author":11,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[166],"tags":[226,215],"coauthors":[98],"class_list":["post-243719","post","type-post","status-publish","format-standard","hentry","category-policies-st-en","tag-education","tag-scientometrics"],"acf":[],"_links":{"self":[{"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/posts\/243719","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/comments?post=243719"}],"version-history":[{"count":0,"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/posts\/243719\/revisions"}],"wp:attachment":[{"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/media?parent=243719"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/categories?post=243719"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/tags?post=243719"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/revistapesquisa.fapesp.br\/en\/wp-json\/wp\/v2\/coauthors?post=243719"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}