Multi-sensor fusion is important in real-world robotics systems, but aligning different sensors through calibration is challenging and requires hours of human efforts. To this end, we propose SemAlign that does not require ground-truth calibration annotations and automates the process of camera-3D calibration.