Frankly you don’t even need to use AM/PM for casual conversation… “I’ll see you at four” almost certainly doesn’t mean 4am unless it’s very clear from contextual information, same for verbally booking appointments

It’s just things like travel timings etc which are likely going to be written anyway where you actually need that clarification