We have 87 camera angles on every play these days, and I'm convinced that each of us sitting on our couches at home have a better view of what's going on in the game than do the refs. Why not take six of the seven refs off the field, stick them in a booth/trailer with a bank of video screens, and let them call the game from there? Leave the head ref (with a headset) on the field to blow the whistle, announce penalties, and spot the ball. Thoughts?
To answer your question directly: yes, we do need refs.
But I think your question is really this: do we need "on the field" refs?
In my opinion, we only need 2 or 3 "on the field" refs for a football game. Their only duties would be to spot the ball and signal timeouts.
All other refs would be remote and assigned to one particular part of the field and/or player (e.g., a ref for: the offensive backfield; the QB; each of the sidelines; each of the WRs; each RB/TE that goes out for a pass; the line of scrimmage; the goal line; etc.).
Each "remote" ref would have a dedicated camera for his particular player or part of the field. (For the offensive backfield, we could have 2 refs watching, each ref having a different angle, since there are at least 10 different players in that area and that's a lot to watch over.)
All of the "remote" refs would be required to make calls in real time as the play happens. We DON'T want to review every play after it occurs.
The 2 major areas for improvement in refereed plays during a football game are: 1) holding by OL; and 2) pass interference (both offensive and defensive).
In my opinion, the "remote" refs could do a MUCH better job of refereeing the game than "on the field" refs because: 1) the dedicated camera would better enable each ref to focus on his job; 2) each ref is seated (instead of running around on the field) and does not have to worry about getting into position to make a proper call; and 3) each ref is isolated and protected from improper influence by coaches, players and/or fans.