No, you don’t have to connect the kit with the APP when you are riding. The assist system is written in the controller, not in the APP. But before the first time you use it, you must calibrate the gyro angle for the bike.
You can set the assist grade in the APP and put your phone in your pocket. The kit will provide proper assist power for you automatically.