Stretchable electronics has received major attention in recent years due to the prospects of integrating electronics onto and into the human body. While many studies investigate how different conductive fillers perform in stretchable composites, the effect of different elastomers on composite performance, and the related fundamental understanding of what is causing the performance differences, is poorly understood. Here, we perform a systematic investigation of the elastomer influence on the electromechanical performance of gold nanowire-based stretchable conductors based on five chemically different elastomers of similar Young’s modulus. The choice of elastomer has a huge impact on the electromechanical performance of the conductors under cyclic strain, as some composites perform well, while others fail rapidly at 100% strain cycling. The lack of macroscopic crack formation in the failing composites indicates that the key aspect for good electromechanical performance is not homogeneous films on the macroscale but rather beneficial interactions on the nanoscale. Based on the comprehensive characterization, we propose a failure mechanism related to the mechanical properties of the elastomers. By improving our understanding of elastomer influence on the mechanisms of electrical failure, we can move toward rational material design, which could greatly benefit the field of stretchable electronics.