THE OPERATIONALISATION OF THE R & D ASSESSMENT FRAMEWORK IN MAGNETI MARELLI SERBIA

This paper has twofold purpose. Firstly, an assessment method aimed to evaluate R&D strategic performance indicators is introduced. The proposed framework is based on fuzzy set theory which represents an adequate tool for quantitatively describing the vague and ambiguous nature of the R&D function. The second purpose of the paper is to analyze relevant R&D performance indicators in Magneti Marelli S.p.A. Serbia, a leading supplier in Serbian automotive industry. Specific findings of this empirical analysis are discussed and suggestions for further research are provided.


INTRODUCTION
The measurement and evaluation of R&D performance has always posed a very challenging task to managers, primarily because of the long gestation period and difficulties in identifying a tangible output.According to Wang and Hwang (2007), long lead times of R&D and market and technology dynamics which are reflected in short life cycle of technologies and very fast speed of innovation could lead to unavailable and unreliable information.Moreover, since the effort levels are not easy observable and since there is high uncertainty in success, influenced by various uncontrollable factors, R&D has a larger extent of problems associated with performance measurement than manufacturing and services (Banwet & Deshmukh, 2006).
Due to the nature of the R&D activity that is characterized with high degree of uncertainty and with often highly fuzzy completed output, in many cases R&D has been treated as an expense centre (Baglieri et al., 2001).In that sense, one of the crucial motives for measuring R&D performance is validation of the chosen investment level on research and development.In other words, there is necessity that R&D function proves its productivity and contribution to the whole organization (Ojanen & Vuola, 2006).Lee et al. also, pointed out that measurement of R&D efficiency must be conducted to determine whether the investment is justified and whether its maximum productivity is achieved (Lee et al., 1996).Although many researchers believe that such measurement may be counter-productive, because the mere act of measurement could reduce productivity, nevertheless, companies continue to evaluate R&D with crude methods available as they look for more effective quantitative methods (Pappas & Remer, 1985).Consequently, improvements in methods and processes for assessing and managing R&D function are gaining more and more significance.
Hence, in this paper we introduced an approach based on Fuzzy set theory, that aims to quantitatively describe the imprecise R&D performance indicators.
According to Kosko (1994), fuzzy systems are universal approximators to algebraic functions.As opposed to the classical sets in which objects must satisfy precise properties of membership, in a fuzzy set theory, membership of an object can be approximate.In that way, the notion of binary membership is extended to accommodate various ''degrees of membership'' on the real continuous interval [0,1].Generally, Fuzzy logic and fuzzy triangular numbers are utilized to deal with the myriad of situations where the inputs to a problem are vague and ambiguous (i.e. when inputs are not specific but amorphous or when there are too many choices and contradictories).Ross (2004), stated that "vagueness can be used to describe certain kinds of uncertainty associated with linguistic information or intuitive information''.
Examples of vague information in the R&D context are: the quality of investments is ''good,'' or percentage of innovation projects is ''acceptable.''In this paper, we used characteristic traits of fuzzy sets to adequately represent vague aspects of R&D performance indicators.
The paper is organised as follows: Second section reviews the specific literature on R&D performance, whereas third section describes the research methodology and introduces a framework for R&D performance indicators analysis based on fuzzy logic.In the section four, we present a real-world case to demonstrate the proposed approach.Namely, the paper specifically addresses the issue of how to deploy the fuzzy model to effectively improve evaluation of R&D performance indicators in the company -Magneti Marelli S.p.A. Serbia.Finally, we discuss the findings and propose directions for future empirical work.

LITERATURE REVIEW
Summarizing previous research findings Werner and Souder (1997), synthesised the following approaches to the R&D performance measurement: Collection and aggregation of multiple judgements of quantitative and qualitative nature; joint inter functional estimates of R&D performance; set comparative performance standards for benchmarking of measures.In attempt to further specify measurement approaches, Kerssens-van Drongelen and Bilderbeek (1999), provide additional classification of generic measurement techniques.They formulated four main techniques as: Subjective assessment of superiors; assessment by an independent third party; feedback from the internal and external customers; and objective score on quantitative criteria.Chiesa and Frattini (2009), presented the taxonomy of research stream on performance measurement in R&D and argued that at the first level there is a focus on the choice of the relevant indicators and metrics for R&D performance measurement.According to Lager (2011), some of relevant indicators could be defined as follows: Knowledge of production structure and market conditions in the industry, collaboration with research institutes, skill and competency profiles of R&D managers, using formal innovation processes to manage business relevant innovation projects, etc.
Addressing the problem of selection of R&D measures, Ojanen and Vuola (2006), argued that the most successful approach appears to be integrated metrics that contain quantitative and qualitative techniques.Similarly, Pappas and Remer (1985), emphasized that semi-quantitative techniques are generally viewed to be the most appropriate for evaluating R&D productivity since many problems of quantitative and qualitative techniques can be avoided using this approach.Semiquantitative techniques include qualitative judgements that are quantified according to different ratings.Still, there are limitations regarding the use of traditional numbered scales and making qualitative judgements.Therefore, in order to enhance the efficiency and the quality of the assessment process, an alternate measurement models based on the fuzzy logic are introduced.
Since Zadeh presented Fuzzy set theory as a strict mathematical framework in which vague phenomena can be precisely and rigorously studied, the utilization of fuzzy logic continuously arises in management and organizational decision models (Zimmerman, 2001).Bhattacharya et al. (2011), also stressed that Fuzzy set theory has been extensively used in different areas of application in order to model uncertainty and vagueness.Similarly, Lin and Yang (2015), discussed that fuzzy logic enables optimal quantification of imprecise information and effective decision-making based on vague and incomplete data.Recently, various models based on fuzzy logic are primarily used in evaluation of individual R&D projects.In that sense, Carlsson et al. (2005), developed a model for valuing options on R&D projects using trapezoidal fuzzy numbers.Coffin and Taylor (1996), addressed multiple criteria R&D project selection and scheduling using fuzzy logic.Huang et al. (2008), developed a fuzzy AHP application in governmentsponsored R&D project selection.Summing up the studies on R&D portfolio management, Wang and Hwang (2007), also cited several notable examples of Fuzzy set theory applications in R&D project selection process.
Based on the findings drawn from the vast literature on R&D performance measurement and specific applications of fuzzy logic, a framework for appraisal of R&D strategic performance indicators is developed.

MODELLING FUZZY ASSESSMENT FRAMEWORK
Developing fuzzy model of R&D assessment framework inevitably involves identification of strategic performance indicators.In that sense, we used the list of "success factors" or "leading indicators" proposed by Lager (2011).
Negative numbers from the interval [-1,0) are used to estimate degree of weakness and positive numbers from the interval ( 0 (1) Graph of given membership function is presented in Figure 2.
In a given questionnaire, the level of expert assurance in estimation of each relevant performance indicator a i , i = 1,...,n, with values ranging from 0 (0%) to 1 (100%) is also requested.The boundaries of the a i -confidence interval for the indicator I i are: (Figure 3) -SiCA -Significant competitive advantage.
Each term is defined by an appropriate membership function (Figure 4).

CASE STUDY-MAGNETI MARELLI SERBIA
In preparation of items for research questionnaire we decided to use the list of leading performance indicators proposed by Lager (2011), although there were other relevant lists appropriate for our purpose.When reviewing similar empirical studies it is evident that the most frequently used scales are graded scales such are five-point Likert scale or seven-point Likert scale.In this paper we developed a model in which the R&D performance indicators are estimated by fuzzy numbers.The main reason for this approach was in the fact that fuzzy model of assessment could handle uncertain and flexible parameters more effectively than traditional quantitative scales.
We have chosen Magneti Marelli S.p.A. for our research because this company represents well known, worldwide supplier for automotive sector.We have also taken  (Pešić, 2015) into account that automotive industry in Serbia plays an important role in the manufacturing sector of domestic economy.Since 2010.Serbian Government and Magneti Marelli (as subsidiary of Fiat Group) signed an agreement for the opening of a new industrial site in the Kragujevac area, Magneti Marelli production facility in Serbia has supplied the local market with components mainly aimed at the operations of Fiat Auto Serbia (FAS).
In this case study we opted for the most commonly used approach to R&D function assessment that is based on the personal judgment of relevant managers.
According to the list of leading indicators proposed by Lager, team of 8 operative managers of Magneti Marelli identified 10 relevant performance indicators I i , i = 1,...,10.
Those indicators were assessed in the way described previously in the paper (using Equation ( 2)).
The average values of individual assessments for performance indicators are shown in Table 1.
On the basis of this data, following confidence interval for this organization is obtained (using Equation ( 3)): [0.368, 0.564].
From the Figure 5 it is evident that this organization belongs to the set SmCA from the degree 0.736 (and at the same time it belongs to the set NoCA with the degree 0.264) to the degree 0.872 (and at the same time it belongs to the set SiCA with the degree 0.128).
The degrees are found by substituting 0.368 and 0.564 for arguments into the membership function of the terms SmCA and  SiCA.As additional data, we used the arithmetic mean of the confidence interval (0.466) to calculate the average value of the competitive advantage of Magneti Marelli.In this case, the organization belongs to the set SmCA with the degree 0.932, while it belongs to the set NoCA with the degree 0.068 (Figure 5).

DISCUSSION AND SUGGESTIONS
Since the aim of this case study was to obtain an expert opinion of the current state of R&D performance indicators we conduct an investigation only with executives in Magneti Marelli.The needed information for the study was collected by a questionnaire survey.
The questionnaire was mailed to the quality manager in Magneti Marelli who distributed it by hand to the other operational managers within the company.Each manager received a written note outlining the purpose of research and explanation of the measuring scale.The data analysis was based on their replies.The individual assessments of R&D performance indicators were followed by the calculation of the average values of those assessments.By analyzing the answers it is evident that the indicators I 3 (Knowledge of production structure and market conditions in the industry) and I 8 (Experience and creativity of R&D professionals) are generally the best ranked and thereby represents strengths contributing to achieving competitive advantage.On the other side, indicators such as I 5 (Average cost reductions) and I 9 (Willingness to undertake risky projects) have the lowest value and possible increasing their values could play an important role in improving efficiency in the upcoming period.
Generally, the state of leading indicators or key success factors indicates a favorable competitive position of Magneti Marelli.That is, considering the fact that survey options include two unfavourable positions (e.g.position of significant competitive disadvantage and position of small competitive disadvantage) with one neutral position (no competitive advantage), belonging to the two remaining positions designated as "small competitive advantage" and "significant competitive advantage" represents a desirable outcome for an organization.Competitiveness of the R&D function of Magneti Marelli is mostly encompassed by the set "small competitive advantage".Still, at the same time, it is also encompassed (to a very small extent) in the set "no competitive advantage" and in the set "significant competitive advantage".To be more specific, the average value of the competitive advantage the organization was calculated.It was found that competitive position of Magneti Marelli belong to the set "small competitive advantage" with very high degree -0.932, while it belongs to the set "no competitive advantage" with almost insignificant degree -0.068.
Further research could be performed in order to analyze differences of R&D performance evaluations in Magneti Marelli branches from different countries in Eastern Europe.Practical applicability of using models based on fuzzy logic could also be tested in process of evaluation of specific R&D projects in Magneti Marelli.

CONCLUSION
Increasing complexity and intensive growth of technologies continuously forcing organizations to rely on research and development function in order to adapt to technology changes and achieve a strong competitive position (Abbassi et al. 2014).In that sense, measurement of R&D performance indicators is very important task for managers because it provides information needed for the effective management and improvement of the overall R&D function.However, measuring R&D performance has always been tricky due to the nature of R&D activities and the difficulties in identifying a tangible output (Pappas & Remer, 1985).Namely, although R&D may have measurable input, the main problem in measuring productivity is that R&D output is often intangible and difficult to quantify.
Due to characteristics of the R&D function, such as uncertainty and ambiguous of the strategic R&D factors, some standard and traditional quantitative methods cannot provide satisfactory level of assessment (Pešić et al., 2015b) Thereby, in this paper we introduced an alternate quantitative approach that incorporates fuzzy logic to assist managers in reaching a better understanding of the overall competitiveness of R&D function.
The proposed framework applies fuzzy set theory and involves fuzzy triangular numbers in assessment process in order to deal with measurement on a more flexible level comparing to the traditional quantitative methods.Additionally, since proposed assessment model do not require advanced mathematical knowledge it provides a fast way of assessing the extent of strengths and weaknesses of R&D function.
Fuzzy model has been tested by means of a real case application, which refers to an Italian company -Magneti Marelli S.p.A. operating in the Serbian automotive industry.
Empirical research has shown that the implementation of the fuzzy assessment framework creates
Кључне речи: Процена R&D, индикатори перформанси, фази троугаони бројеви, фази логика, "Magneti Marelli" framework creates no difficulties to the managers of the company and that it can be successfully adopted in management planning and control functions.
Figure 1.Questionnaire for the fuzzy assessment of the R&D performance indicator(Pešić, 2015a) Figure 3. a i -confidence interval of the R&D performance indicator

Table 1 .
Obtained data of the case study