A New Look at the Ensemble Kalman Filter for Inverse Problems: Duality, Non-Asymptotic Analysis and Convergence Acceleration