Being able to measure the success of online shopping interactions is crucial in order to evaluate and optimize the performance of e-commerce systems. We consider the domain of voice shopping, supported by digital voice-based assistants, where measuring successful interactions poses a challenge. Unlike Web shopping, which offers a rich amount of behavioral signals such as clicks, in voice shopping a non-negligible amount of shopping interactions frequently end without any immediate explicit or implicit user behavioral signal. Moreover, users may start their journey using voice, but finish elsewhere, for example by using their mobile app or Web. We explore the challenge of measuring successful interactions in voice product search based on users’ feedback, and propose a medium-term reward metric named Extended ConVersion (ECVR). ECVR extends the notion of conversion (purchase action), which is a clear and natural indication of success for an e-commerce system. The strength of this new metric, is that it does not only capture immediate conversion, but also a conversion that is part of the same user shopping journey, but is performed at a later stage, possibly using a different medium. We provide multiple ways of evaluating the quality of a metric, and use these to explore different parameters leading to different variants of ECVR. After finalizing these parameters, we show that a ranking system optimized for the proposed ECVR leads to an improvement in long-term engagement and revenue, without compromising immediate gains.