Saying that someone doesn't speak very good English isn't racist. I don't speak great French. Someone in Quebec or France observing that in my conversations with them wouldn't be racist either.
there's nothing wrong with describing how well a person speaks a specific language, but it's kind of weird and fvcked up to bring up a foreign person's language proficiency with a foreign language as literally the first thing you use to describe them when you're asked about them
i'll be honest, i haven't watched the clips so i can't say. if he was having a conversation with that club about how they navigate dealing with ESL players as a baseball organization and then he describes how well they speak the language........fine. but i know you know there's a huge difference if he was randomly asked what he thinks about X player or Y player and the first thing that comes out of his mouth is 'well his english fvcking sucks so yeah, his english is bad'