Flexible ureteroscopy is a routinely performed surgical procedure to treat renal disorders such as tumors and stones, but it gets trapped in precisely orientating the ureteroscope in the complex kidneys. To facilitate ureteroscopic procedures, we propose employing deep learning techniques for preliminary data processing and propose a new ureteroscopic navigation framework that uses deeply learned bilateral 2D-3D registration. Specifically, a new structural intensity-position similarity function is formulated to characterize the difference between 2D ureteroscopic video sequences and preoperative computed tomography urography images. While we propose a small deep learning model of deformable large-kernel convolutional networks without any transformer blocks to segment the urinary collecting system from preoperative images, we employ dense prediction transformers and a color model of hue-saturation-value to extract structural regions from ureteroscopic video sequences. The new cost function is designed by the dice similarity coefficient and structural similarity index to calculate the pixel intensity and position (coordinate) differences. We validated our method on clinical data collected from different patients in the operating room, with the experimental results showing that our method outperforms state-of-the-art registration approaches, reducing the navigation errors from (7.8 mm, 10.7°) to (7.1 mm, 9.7°).