If I'm connecting my controller via MIDI port, can I skip installing the SD card slot and just use the USB port with a flash drive to install the OS and store the samples?
AFAIK you need to have an OS installed on an SD card (or at least some of it) for the Pi to even know the USB ports exist. You can however keep all your samples on a USB flash drive: http://www.samplerbox.org/faq#wheresamples
Found an article in the raspberry docs where booting from USB is explained.
As the article explains, this action is one-time: booting from SD is impossible forever.
However, I can only see disadvantages in running from your sample stick(s).