You can´t feed the main board 6V, it needs at least 7-8 to work reliably. Max is 18V. Do you have a good reason not to feed the main battery directly to the FC? (like using 5S or more?)
You can feed 6V to PWM header on ESC32 if you have the powerselection jumper set to "5V". In this case the voltage you feed to the PWM or UART header is then fed to a 3.3V LDO that in turn powers the logic side of ESC32.
I would not leave out the ground wire on the ESC´s and servos, but otherwise its OK.
Depending on how much stuff you need to drive, you could consider replacing the Recom between the uSD slot and the Ublox with a beefier 6V recom. The current one is rated 5V/1.5A and is used to drive the servos and the supply for the uSD slot. So there is no problems in adding a 6V here, in fact Bill conceived this as an option to begin with.