The governor may remove a member of the board for misconduct, incompetency, or neglect of duty.