COLLABORATIVE VENDOR MANAGED INVENTORY MODEL BY USING MULTI AGENT SYSTEM AND CONTINUOUS REVIEW (R, Q) REPLENISHMENT POLICY

In the vendor-managed inventory (VMI) system, the vendor takes over responsibility for managing customer inventory so that delivery is not based on the order but the customer's inventory condition. It makes the vendor becomes a dominant entity, and customers are supplied by its own vendor exclusively. That is why most studies in VMI implement a single-vendor-single-customer or single-vendor-multi-customer scenario. In certain conditions, this exclusiveness can increase lost sales. Besides, most of them implement a single product scenario. In this work, we develop VMI model for the multi-vendor-customer-product scenario. This model is developed based on the collaborative multi-agent system. The relationship between vendors and customers is many-to-many. This work aims to reduce lost sales and maintain efficiency in the inventory. The continuous review (r, Q) policy is used as the replenishment model. The simulation result shows that the collaborative model creates higher sales, lower lost sales, and competitive inventory than the non-collaborative one. The lost sales is 50 to 75 percent lower. The sales percentage is 17 to 27 percent higher. The total retailers' stock is 20 to 38 percent higher. The total vendors' stock is 11 to 30 percent lower. The total stock in the supply chain in the collaborative model is 2 to 16 percent higher. The number of retailers is directly proportional to the total vendor's stock and total supply chain stock gaps; inversely proportional to the lost sales gap; and not related to the sales percentage and total retailers' stock gaps.


INTRODUCTION
Vendor-managed inventory (VMI) is a popular model in supply chain management (SCM). This model has been adopted widely in many industrial areas: fast moving consumer goods, error sensitive industries, perishable goods industries, high value, and unpredictable demand industries, and low margin competitive industries [1]. There are many studies in VMI with various industrial cases, such as chemical products [2], sawmill in Sweden [3], timber [4], and instant noodle in Thailand [5]. There is a difference between the VMI system and the traditional supply chain model. Rather than based on the customer's purchasing order, supply is based on customer's inventory and sales condition [6]. The vendor has direct access to the customer's inventory information [7]. The obligation in managing customer's inventory is transferred from customer to vendor [8]. The customer plays a passive role in managing its inventory [9]. The vendor then becomes the dominant entity in the VMI system. In many VMI studies, the vendor-customer relationship can be divided into two groups: one-to-one and one-tomany. The example of the one-to-one is work conducted by Hadiguna et al. [8] which proposed one-to-one-based VMI model for infinite production rate and fuzzy demand. The example of the one-to-many is work conducted by Salem and Elomri [10] which studied several one-to-many-based VMI model where the customers are retailers. Studies in VMI that implemented many-to-many relation-ships are rare. One of them is work conducted by Casino et al. [7] which developed blockchain-based information sharing in multi-vendor-multi-retailer based VMI model. Unfortunately, this work focused on information security, not on inventory dynamics. In both models, customers are supplied only by its vendor so that other vendors cannot serve them. When the customers need products and its vendor's inventory is not available, the lead time will increase. It also triggers potential lost sales. Meanwhile, any other vendors may be more ready to supply them.Our research aims to improve sales and maintain low inventory level by developing collaborative many-tomany-based VMI model. This model consists of multiple vendors and multiple customers. It adopts a collaborative approach and eliminates exclusive vendor-customer relationship. The customers can be served by any vendors in the system who are ready. There are several critical points in this work. This model is developed by using multi-agent system (MAS). A multi-product scenario is applied. The customers are retailers. (r, Q) the policy is used as a replenishment model. (r, Q) the policy is a popular replenishment policy, and it was also used in several studies in VMI, for example, a study conducted by Taleizadeh et al. [11]. Our contributions are as follows. 1. We propose a many-to-many based VMI model, which is very rare in VMI studies.
2. In our model, the exclusive relationship between vendor and customer is eliminated. This paper is organized as follows. The background, problem statement, research purpose, and paper organization are explained in the first section. In the second section, we describe the related studies, which includes studies in VMI, (r, Q) policy, and multi agent system. In the third section, we describe the method used in this work. In the fourth section, we describe the simulation result and the discussion. In the fifth section, we conclude the work.

Vendor-managed Inventory
Vendor-managed inventory is a supply chain management model which implements collaboration between vendor and customer [6]. Collaboration means both vendor and customer try to develop mutual benefit among entities in the supply chain system [7]. This concept is different from the traditional vendor-customer relationship, where each party acts based on its interest [12]. Traditionally, the customer only shares the purchase order with the vendor [6] so that vendor only serves the customer only based on the purchase order. Sometimes, there is a bias between the actual condition in the retailer's inventory, sales data, and purchase order. Sometimes purchase order does not represent the real market and inventory conditions [6]. Based on this problem, VMI becomes a solution. The vendor takes over the responsibility in managing customer's inventory and shares its inventory condition [8]. It gives the vendor have a clear view to make the decision [7]. In the VMI system, the customer does not send purchase orders anymore. Many studies stated that VMI provides advantages compared with the traditional way. Sari [6] noted that VMI offers higher product availability, lower inventory cost, and lower lead time for retailers. Joseph et al. [1] stated that VMI could improve production plan and delivery, prevent stock-out, and reduce inventory cost. Khajehnezhad [12] stated that VMI can maximize revenue and minimize cost. Hadiguna et al. [8] stated that VMI can improve service level and inventory turnover. Casino et al. [7] stated that VMI can increase sales because of better product availability and avoid overstock and shortage condition.Various replenishment policies were used in studies about VMI. Sari used (R, S) where R is reviewing interval and S is order-up-to level [6]. Hadiguna et al. [8], Poorbagheri, and Niaki [13] used economic order quantity policy, and this policy was implemented in uncertain demand conditions. Taleizadeh et al. [11] and Guan and Zhao [14] used (r, Q) policy. Multi-vendor-multi-customer VMI system can be developed based on single-vendor-multi-customer VMI system. This system can be modelled by integrating vendors and customers in one VMI system. An example of this system is illustrated in Fig. 1. In Fig. 1, there are three vendors and six customers. Vendor one maintains customers one and two.
Vendor two maintains customers three, four, and five. Vendor three maintains customer six.

Figure 1: Multi-vendor-multi-customer VMI architecture
Unfortunately, this relationship is exclusive. The vendor can access information that belongs to their customers. This vendor cannot access customers who do not belong to it to not supply products to them. The problem arises when its vendor cannot provide for customer's needs. On the other side, other vendors who are more ready cannot supply them. Based on this problem, our idea is to develop a collaborative VMI model. In the collaborative VMI model, the relationship is not exclusive so that any vendors in the system can supply every customer.

Continuous Review (r, Q) Replenishment Policy
(r, Q) the policy is a replenishment model widely used in many studies in stochastic-based inventory management [15]. This model contains two notations: r and Q. r represents the reorder level, and Q represents the fixed size order quantity or batch size [16]. These variables have their purpose or behavior. Higher r can avoid stockout probability but increase higher inventory space [17]. Larger Q can reduce replenishment frequency but increase inventory level [17]. There are two types of inventory review in this model: continuous (real time) and periodic [18]. In the continuous review, a sophisticated warehouse management system is needed [18]. In their study, the orders arrival was random [16]. The inventory capacity was limited so that stock could not surpass the inventory capacity [16]. In other work [15], the shortage was allowed, replenishment was instantaneous, and demand was also stochastics. There are several arguments about demand distribution. Castellano [17] stated that it is difficult for decision makers to know the distribution type of the demand in the real world, although they know the mean and the variance. Sung and Oh [19] noted that demand arrival follows the Poisson process, and demand size follows exponential distribution. Moon and Gallego [20] stated that the demand follows normal distribution based on the assumption that individual demand is independent and identically distributed (IID) random variables. On the other side, Castellano [17] stated that in reality, normal distribution in demand arrival is hard to validate. Besides, Andersson et al. [21] stated that individual demands are not IID random variables. Gallego et al. [22] did not recommend normal distribution for demands with highly uncertain demand. There is a relationship between (r, Q) policy and economic order quantity (EOQ) policy. Similar with (r, Q), EOQ also implements fixed order quantity [23]. The difference is that in EOQ, the time interval between successive orders is fixed too [24] because EOQ is developed based on an idealized inventory with the assumptions that demand is known exactly, continuous, and constant over time; shortages are not allowed; and lead time is zero [23]. That is why (r, Q) policy is more practical than the EOQ policy in real inventory systems where demands are uncertain [17]. Fortunately, EOQ was still used in (r, Q) policy in determining the optimum Q [15]. Several studies in the VMI system also used (r, Q) policy as the replenishment policy. Taleizadeh et al. [11] focused on comparing (r, Q) and (R, T) policies in the VMI system. Meanwhile, Guan and Zhao [14] focused on developing a contract between vendor and customer based on the ownership status of the stock. Both studies used retailers as the customer. The difference is that the first study implemented a single-vendor-multi-customer scenario [11]. The second study implemented single-vendor single-customer scenario [14]. This explanation shows that (r, Q) the policy is appropriate to develop VMI model under uncertain demand. (r, Q) the policy is practical and straightforward to be implemented in the real inventory system. Besides, several studies in VMI also used (r, Q) policy in their replenishment model.

Multi Agent System
There are several definitions of the agent. Russel and Norvig [25] defined that an agent as an entity that perceives its environment through sensors and acts based on its understanding of its environment through actuators. Wooldridge [26] described an agent as a computer system that can serve autonomously based on its design purpose. Glavic [27] stated that the agent could be a physical entity or a virtual entity. Autonomy is the ability of the agent to act independently [28]. Multi-agent system (MAS) can be defined as a group of autonomous agents that acts in an environment to achieve a common goal [28]. The interaction among agents can be cooperative or competitive [28]. MAS can be used to model the self-organizing system [29] due to its automation and adaptation capabilities. MAS was also used in many studies in supply chain management. Gamoura et al. [30] proposed a multi agent-based supply chain architecture that consists of multiple suppliers and multiple customers. In their work, every customer or every supplier is represented by an agent [30]. Pal and Karakostas [31] proposed a MAS for a collaborative material procurement system in a supply chain. Zgaya et al. [32] developed a negotiation model in a multi-agent supply chain system. Based on this explanation, there is a potential in developing a VMI model by using MAS so that this VMI system can run autonomously. In the VMI system, the vendor takes an active role, in the VMI system that consists of many vendors. This system can also consist of multi-agents where every agent represents a vendor. Then there should be an agent that becomes an intermediary among vendors.

Architecture
This work develops a VMI model based on the multiagent system. The model consists of three entities: vendors, customers, and dispatchers. Customers provide their inventory information. Vendors provide their inventory, machines, and preference information. The dispatcher's role is to match the customers' needs for the selected vendors. There is a scheduler embedded in every vendor. The scheduler's role is managing its vendor's production process. Based on it, there are two types of agents. The first agent is the dispatcher. The second agent is the scheduler. The model illustration is shown in Fig. 2.

Figure 2: Collaborative vendor-managed inventory architecture
This system can run autonomously. The dispatcher agent controls the dispatching process by matching three aspects: customers' inventory, vendors' inventory, and vendors' customers preference. By using vendors' customer preferences, a vendor can prioritize some customers rather than the other ones. The scheduler agent controls the scheduling process. The scheduler works by determining the product that must be produced based on its vendor's inventory and product preference. Using the vendor's product preference, a vendor can prioritize some products rather than the other ones. In this system, vendor's customer preferences and vendor's product preferences may be different among vendors.

Model
We use some notations and assumptions in this work. These notations are as follows. : status that there is at least one product held by retailer j that its current stock is below minimum stock st lackve,i : status that there is at least one product held by vendor i that its current stock is below minimum stock st Q,i,j : status that order quantity between vendor i and retailer j can be achieved q totpos,i,j : total possible quantity of products supplied by vendor i to retailer j q pos,i,j,k : possible quantity of product k supplied by vendor i to retailer j Δs max,j,k : gap between maximum stock and current stock of product k held by retailer j q totdel,i,j : total delivered quantity of products supplied by vendor i to retailer j q del,i,j,k : delivered quantity of product k supplied by vendor i to retailer j q al,i,j,k : allocated quantity of product k supplied by vendor i to retailer j q pr,i : remaining production quantity in vendor i q p,i : production quantity in vendor i q req,k,b,j,d : quantity of product k requested by buyer b that visits retailer j in day d q pur,k,b,j,d : quantity of product k purchased by buyer b that visits retailer j in day d q treq : total requested quantity q tsal : total sales quantity q tlost : total lost sales r success : success percentage ΔQ i,j : gap between order quantity of retailer j and current allocated quantity of products supplied by vendor i to retailer j We also use several functions in this work. These functions are shown and explained in Table 1.

find() find in a list
Assumptions used in this model are as follows. 1. Every vendor has willingness to supply to all retailers in certain prioritization. 2. Every vendor has willingness to produce all products in certain prioritization. 3. Shortage is allowed. 4. Lost sales is allowed. 5. Inventory capacity is limited. 6. The interval review is daily.

The buyer's arrival rate follows Poisson distribution
while the requested quantity follows exponential distribution [19].The model is split into two groups. The first model is the dispatcher action model. The second model is the scheduler action model. The first model is the dispatcher model. The vendor will prioritize retailers with higher preference first and they need supply. When the number of retailers who needs supply and has the same preference level is more than one, the vendor prioritizes the retailer with the lowest total stock. The vendor sorting mechanism is based on a stochastic approach with equal opportunity. This concept is used to develop the matching algorithm. This algorithm runs in every retailer replenishment session, and it is daily. The retailer replenishment process is shown in Algorithm 1. .
, , ( + 1) = , , ( ) − , , , ( ) (12) , , ( + 1) = , , ( ) + , , , ( ) (13) The next model is the scheduler model. As the scheduler is embedded in its vendor, then the number of schedulers is same as the number of the vendors. The scheduler's goal is to optimize vendor's inventory based on two aspects: the vendor's product preference and inventory condition. The scheduler review interval is also daily. The scheduler has three main actions: produce, plan, and wait. The production process in every review and the scheduler's actions are shown in Algorithm 2. When the scheduler decides to produce, it will prioritize products with a higher vendor's product preference that meets the requirement. If the number of products in the same preference that meets the requirement is more than one, then the scheduler will choose a product with minimum stock.

Simulation
This proposed model is implemented into a simulation to evaluate the model performance. This model is compared with the existing non-collaborative VMI model [11]. The observed variables are lost sales, success percentage, total retailers' stock, total vendors' stock, and total stock in the supply chain. They are chosen based on the general goals of the SCM which are increasing service level and maintaining low stock. These variables calculation is formalized by using Equation 18 The simulation scenario is as follows. In the beginning, retailers and vendors are generated. The initial value of some variables is set. This setting is formalized by using Equation 25 to Equation 30. , , , = ( ) (28) , , = ( , ) , , = ( , ) Every day, buyers come to retailers to buy products in a certain quantity. If the product and quantity requested by the buyer are available, the buyer purchases this product at the requested quantity. On the other side, the buyer purchases it at the available quantity. Lost sales is the difference between the requested quantity and the purchased quantity. The total retailers' stock, total vendors' stock, and total stock in the supply chain is based on the stock at the end of the simulation. The number of buyers that visit per day, the number of products that requested, and requested quantity are also generated randomly and formalized by using Equation 31 to Equation 33.

RESULT
The simulation result is shown in Fig. 3 to Fig. 7. Fig.  3 to Fig. 7 represent the lost sales, sales percentage, total retailers' stock, total vendors' stock, and total stock in the supply chain consecutively. Fig. 3 shows that the lost sales increases due to the increasing in the number of retailers. The reason is the increasing in number of retailers increases the demand. The lost sales of the collaborative model is 50 to 75 percent lower than the non-collaborative one. This lost sales gap decreases due to the increasing in the number of retailers. It means that the number of retailers is inversely proportional to the lost sales gap.   5 shows that the total retailers' stock increases due to the increasing in the number of retailers. The reason is more retailers means more inventory nodes that must be managed. The total retailers' stock in the collaborative model is 20 to 38 percent higher than the non-collaborative one. This total retailers' stock gap fluctuates, neither increases nor decreases, due to the increasing in the number of retailers. It means that the number of retailers is not related to the total retailers' stock gap.   Fig. 7 shows that the total stock in the supply chain increases due to the increasing in the number of retailers. The total stock in the supply chain in the collaborative model is 2 to 16 percent higher than in the non-collaborative model. This total stock gap increases due to the increasing in the number of retailers. It means that the number of retailers is directly proportional to the total supply chain stock gap. Based on this simulation result, we then process this result to predict the condition when the number of retailers is higher. We process it by using linear regression. In this process, we use only the result of lost sales and total stock in the supply chain because these two aspects can represent two important parameters: sales and inventory. In this process, we expand the number of retailers to 100 units. This linear regression-based prediction result is shown in Fig. 8. The analysis of the prediction is as follows. The collaborative model still creates lower lost sales rather than the non-collaborative one and the gap between models is wider. The collaborative model still creates higher total stock in the supply chain rather than the non-collaborative one and the gap between model is also wider. When the number of retailers is 100 units, the lost sales of the collaborative model is 51 percent compared with the non-collaborative one. On the other side, the total stock in the supply chain of the non-collaborative model is 72 percent compared with the collaborative one. Now, we will discuss the simulation result deeper and connect it with the research purpose. This discussion will be focused on two important aspects in supply chain management: sales and inventory. In the sales aspect, the collaborative model is better than the non-collaborative one. The main cause is the elimination of the relationship exclusiveness so that retailers can be supplied by more vendors. In the collaborative model, all vendors can access the inventory of all retailers and all retailers are served by the vendor which is more ready. It makes the retailer's inventory is more available. This is relevant to Casino, et al.'s statement which said that VMI can improve sales because of better product availability [7], especially on the retailers' side [6]. In the inventory aspect, the collaborative model still maintains a low inventory level. Although the collaborative model creates higher total retailers' inventory, its total vendors' stock is lower on other side. In the total stock in the supply chain, the collaborative model is just a little bit higher. Lower total vendors' stock can be seen that vendors' inventory is more liquid because of the faster product flow from vendors to retailers. This condition is linear with the purpose of VMI in improving service level and inventory turnover [8].

CONCLUSION
The collaborative model performs better than the existing non-collaborative one in the VMI system. The key factor is the elimination of the exclusiveness. The product flow from vendors to retailers is more liquid. In general, this collaborative model has achieved goals in increasing sales, product availability, service level, and inventory turnover, especially on the vendors' side. Meanwhile, total stock in the supply chain still cannot be reduced but it is still low. The simulation result shows that the collaborative model creates higher sales, lower lost sales, and competitive inventory than the non-collaborative ones. The lost sales of the collaborative model is 50 to 75 percent lower. The sales percentage in the collaborative model is 17 to 27 percent higher. The total retailers' stock in the collaborative model is 20 to 38 percent higher. The total vendors' stock in the collaborative model is 11 to 30 percent lower. This total vendors' stock gap increases due to the increasing on the number of retailers. The total stock in the supply chain in the collaborative model is 2 to 16 percent higher. The number of retailers is directly proportional to the total vendor's stock and total supply chain stock gaps; inversely proportional to the lost sales gap; and not related to the sales percentage and total retailers' stock gaps. This work has shown that the improvement of the VMI model is needed and important. It is not only to make this model becomes better than the non-VMI model but also the newer VMI models become better than the older ones. This collaborative model can also be improved and enriched in the future to solve more complex problems or problems with different circumstances, for example, vendors and retailers with prioritization.