It is well-known that combining life annuities and death benefits introduce opposite effects in payments with respect to the unsystematic mortality risk on the lifetime of the insured. In a general multi-state framework with multiple product types, such joint effects are less trivial. In this paper, we consider a multivariate payment process in multi-state life insurance, where the components are defined in terms of the same Markovian state process. The multivariate present value of future payments is introduced, and we derive differential equations and product integral representations of its conditional moments and moment generating function. Special attention is given to pair-wise covariances between two present values, where results closely connected to Hattendorff type of results for the variance are derived. Numerical examples are shown in a disability model and a spouse model to illustrate applicability of the results on insurance of a single life as well as two dependent lives.