Pt Vic would be my pick if it was me. It is in the Middle of Yorkes so no matter what way you want to go fishing you are half way there. It is a great little town that is pretty easy going.. Not a lot there just the kiosk and general store which i reckon has fuel, no big servo, the pub is bloody fantastic, I've had many great nights there. The fishing there is fantastic in close then if you go out wide past Wardang Island (no issues in your boat) there are some big fish and squid to be had. Only down side there is an Aboriginal camp at Pt Pierce and they come into town to get booze, sometimes cause trouble but the locals usually sort them out. You have to be on the ball at the ramp or end up on rocks.. But any experienced captain will be fine.
Second would be Ards.. It is a bit closer to Adelaide and has a lot more stores if that is your thing... Some great snapper there on a handful of drops but every knows them and they get hit hard from the locals and the people from Adelaide as it's only an hour and a bit out of Adelaide on the road and about the same if you head over the gulf in the boat from St Kilda, Which i have been lucky enough to do a few times with members on here. The servo has the best fried chicken and sausage rolls too!!
Third would be Wallaroo. Plenty of stores and services, good fishing but from what i have been told there is a fair drug problem in these parts.. Kadina is the ICE capital of Yorkes apparently which is spreading like wildfire and invading WallyWorld... Just this week on the news there was a massive issue there where someone pulled a gun on the cops and they locked the town down pretty much.
If you want total isolation Marion Bay.
Hope this helps and makes sense