Today on the Scuttleblog a post was published by Ernie Bain, Jr., Senior Judge and Umpire: Certification process
I realize I'm not an US-Sailing judge nor umpire. If that disqualifies me from commenting, please ignore the rest of this post.
I agree with the need to maintain the skill to Judge or Umpire.
Rules knowledge is an important part of that. But not that's not all. You need also to improve abilities like dealing with people in the room or on the water. Professionalism goes further than tests.
However you want to achieve that first part is in the organization, be it trough a points system or annual testing. For the second part you need a support system. Feedback on your teamwork, the opportunity to talk about what has happened and learn from mistakes, without prejudice.
It should be part of the chief or chairman to give that (or delegate to other experienced Judges and Umpires). But also we should expect to hear from our fellow Judges and Umpires every time we work together. Not to criticize, not to put down, but to improve and think about what we are doing.
It should be perfectly normal to talk a few minutes before getting back to shore, or before we rush into the next protest-hearing. Or find someone in a quit moment and tell him how he or she can improve dealing with that particular difficult party in the protest-room. Or how to put a young sailor at ease before starting with those demanding questions.
Ernie's idea is a good one, I'm all for improving rules-knowledge, but not without an equal part attention to 'people-skills'.