If you haven't already done so, I suggest having a conversation with the principal about your concern. I had a similar concern when my daughter started kindergarten. In our case, there were gates but they were being left open during the day, and I worried about unwanted visitors coming in, or children finding their way out. The principal at my daughter's school informed me that there were staff watching (or patrolling) the gates, that guests on campus without a badge would be quickly approached and guided to the office, and that there had been a fire safety concern with keeping gates closed (that it would be difficult to escape during fire). Your school's principal may likewise be able to explain and/or reassure you about the gate situation at your school. I've linked below to some tips for having positive discussions with educators.
I would also suggest that you consider walking your son to his classroom, rather than dropping him off without making sure he enters his classroom. If that's not possible, perhaps you could make arrangements with another parent who does wait with their child, and ask that they keep an eye to make sure that your child makes it into the classroom. Discuss your concerns with the teacher as well. And of course, have a discussion with your son about not wandering off campus without adult supervision (specifying which adults it is ok for him to leave with). These are just some ideas off the top of my head. Your principal, teacher or other parents may have more. Good luck!