This study proposed a real-time deep fusion framework for RF positioning ensembles by merging heterogeneous sensor sources and positioning algorithms to create a better combination for a localization solution. The current incarnation of the proposed fusion framework uses video localization, Wi-Fi positioning, motion filters, and a dynamic matching and weighting mechanism in an asynchronous manner. Real-world experiments show that the resulting ensemble reduces the mean error to 0.83 m in real time and 0.62 m for off-line use.