The players.
More specifically, the 11th Edition rules for Ultimate state:
XVI. Violations and Fouls
H. Fouls (II.E): It is the responsibility of all players to avoid contact in every way possible .
1. A foul can be called only by the fouled player and must be announced by loudly calling foul immediately after it occurs.
Chat with our AI personalities
Players call their own fouls so they can occur at anytime. When a foul does occur play is stopped, people stay right where they were, person throwing the disc gets is back, and play resumes as if nothing happened.